Use Drupal FAPI #tree only visually, without affecting the (already nested) form value structure

The Drupal Form API (FAPI) #tree property is helpful and needed when using form containers like details, fieldset or others. Also it's helpful if you need to nest form values and also want to nest their results.

On the othe hand it becomes tricky and painful, if you just need a visual container, but don't want to change the array structure.

The general functionalities of the #tree attribute, #parents #array_parents are described here: https://www.drupal.org/docs/7/api/form-api/tree-and-parents

But how can we add a container to an existing tree form structure, while preserving the form submit values structure?

A good workaround I found for the Homebox (3.0.x) module, is using  #parents explicitly for the form items, overriding the tree changes. This way you set the expected form structure manually and ensure it's not affected by the additional tree.

The old structure, whose form value structure we may not change was this:

 $form['label'] = [
      '#type' => 'textfield',
      '#title' => $this->t('Label'),
      '#default_value' => $this->getConfigurationPortletLabel() ?? '',
      '#description' => $this->t('Allows overriding the title. Leave empty to use the default title.'),
      '#placeholder' => $this->getLabel(),
      '#weight' => 999,
    ];
    $form['color'] = [
      '#type' => 'radios',
      '#title' => $this->t('Color'),
      '#default_value' => $this->getThirdPartyConfigurationValue('color') ?? '',
      '#options' => $this->getColorOptions(),
      '#weight' => 998,
      '#description' => $this->t('Select a color for this portlet.'),
      '#attributes' => ['class' => ['fieldgroup--homebox-color']],
    ];

The new structure, with the added details element, looks like this:

    $form['individualizations'] = [
      '#type' => 'details',
      '#title' => $this->t('Individualizations'),
      '#collapsed' => TRUE,
    ];
    $form['individualizations']['label'] = [
      '#type' => 'textfield',
      '#title' => $this->t('Label'),
      '#default_value' => $this->getConfigurationPortletLabel() ?? '',
      '#description' => $this->t('Allows overriding the title. Leave empty to use the default title.'),
      '#placeholder' => $this->getLabel(),
      '#weight' => 999,
      // We need to override the parents because of the details tree:
      '#parents' => ['portlet_form', $this->getDelta(), 'label'],
    ];
    $form['individualizations']['color'] = [
      '#type' => 'radios',
      '#title' => $this->t('Color'),
      '#default_value' => $this->getThirdPartyConfigurationValue('color') ?? '',
      '#options' => $this->getColorOptions(),
      '#weight' => 998,
      '#description' => $this->t('Select a color for this portlet.'),
      '#attributes' => ['class' => ['fieldgroup--homebox-color']],
      // We need to override the parents because of the details tree:
      '#parents' => ['portlet_form', $this->getDelta(), 'color'],
    ];

Especially note the new details wrapper:

    $form['individualizations'] = [
      '#type' => 'details',
      '#title' => $this->t('Individualizations'),
      '#collapsed' => TRUE,
    ];

and the #parents attribute:

      '#parents' => ['portlet_form', $this->getDelta(), 'color'],

In this example, the resulting array strucuture is the same like without the new details container. We couldn't use '#tree' => FALSE or '#tree' => TRUE here, as the parents already define it and this would then also change the form values structure.