Skip to content!

SOEP-Core v34 (2017) - Changes in the Dataset

Änderungen am Datensatz

Dataset Information

SOEP-Core soep.v34

1. New, user-friendly integrated data format

The new wave of the SOEP-Core study incorporates our “wide” and “long” data formats, which used to be provided to users separately. Our aim is to eliminate any confusion about what is available in which format and to make data use easier overall. After several years of testing SOEPlong as an additional service designed to facilitate analysis for both experienced and new users, we will now be providing all datasets in the “long” format as a standard part of our SOEP data release. This means that you will find the different SOEP data formats listed below in your data file, some of which will be contained in separate subdirectories.

Please make sure that you unpack the entire directory structure when unpacking your data.

1.1. SOEP in “long” format on the top level

In the top-level (or root) directory, you will find all of the datasets provided up to now with SOEPlong (pl, ppfadl, etc.) as well as all of the additional datasets formerly provided only in our classic “wide” format (biographical or spell data such as bioparen, artkalen, etc.). All of the data in the main SOEP-Core study are therefore contained in the datasets in the top-level directory.

Feedback from experienced and beginning users over the past several years shows that the “long” data offer significant advantages in ease of use, particularly for beginners. We have therefore decided to use this as our primary data format in future data releases.

All available individual year-specific datasets are pooled into a single dataset (e.g., all $P datasets are integrated into the PL dataset). In some cases, this means that we have to harmonize variables in order to be able to define them consistently over time. For instance, income information is given in euros up to 2001 and not in deutschmarks, and in cases where questionnaires have changed, the categories are modified over time. All changes are presented to users in a clear and understandable way, and if harmonization is necessary, all input variables are provided in their original form (see below _v*-variables). SOEPlong thus significantly reduces the number of datasets and the number of variables.

A more detailed description of the format of our SOEP-Core data release can be found in our new SOEPcompanion.

1.1.1. Most important changes to v33 in the long format

  • The following new files have been added:
    • HBRUTT: long file of the HBRUTT$$ files
    • PLUECKEL: long file of the $PLUECKE files
    • VPL: long files of the $VP files
  • Data sets PL and PL2 are being provided again in one combined file (PL).
  • The variable scheme with c-variables (cross-sectional) and l-variables (longitudinal) has been modified as follows:
    • If the variables on which a variable in the long format is based changed in the cross-section, then corresponding _v*-variables will be created for each version. A harmonized _h-variable is provided as well. Further information can be found in the SOEPcompanion (general description, examples)
  • All of the long datasets generated from the various cross-sectional datasets contain the new variable: INPUTDATASET.
  • Due to adjustments to the new joint data release format, some files with “long”-specific names are no longer included in the data release: CDESIGN, CSAMP, CSAMPFID, KIDL, PBREXIT.
  • The following datasets have been renamed to avoid conflicts with the data names in the raw directory:
    • PPATH replaces PPFAD
    • PPATHL replaces PPFADL
    • HPATH replaces HPFAD
    • HPATHL replaces HPFADL

1.2. Classic format in the subdirectory raw

Since we know that many users have existing scripts that are based on the original data format, and to enable users to understand the process of generating the “long” data, we provide all of the datasets in their original SOEP format in the directory raw.

Users who want to continue using the old format simply need to switch into subdirectory rawand use the datasets there.

The only change is that there are now additional identifiers in all of the datasets in the raw directory with the name in the long format (PID and PERSNR or HID and $HHRNAKT) and a survey year variable (SYEAR) so that users can easily merge variables from the two data formats.

1.3. New EU-SILC clone in the subdirectory eu-silc-clone

Many users are undoubtedly aware that the SOEP supports cross-national analysis with CNEF through the dataset PEQUIV. We have now produced a data product that allows you to use the SOEP data in comparative analyses with the EU-SILC (European Union Statistics on Income and Living Conditions) data. EU-SILC, which is provided by Eurostat upon request, offers cross-sectional and longitudinal information for many European countries. Up to now, only cross-sectional information has been available for Germany. The EU-SILC clone offers longitudinal information on private households in Germany based on the SOEP data. All of the information contained in it can be directly compared with the EU-SILC longitudinal information on other European countries.

The EU-SILC clone is integrated into the standard SOEP data release (in subdirectory eu-silc-clone).

Documentation on the 2005-2016 EU-SILC clone can be found here.

2. New samples in the main SOEP study

The new SOEP data release (v34) will be the first to contain data from the IAB-BAMF-SOEP Survey of Refugees in Germany as Sample M5, as well as the continuation of the PIAAC-L Survey, as Sample N.

2.1. IAB-BAMF-SOEP Survey of Refugees (M5)

The SOEP, in cooperation with the Institute for Employment Research (IAB) and the Federal Office for Migration and Refugees (BAMF), has succeeded in integrating a third sample of refugee households (M5) into the SOEP study. The survey was launched in 2017. The population of M5 covers adult refugees who have applied for asylum in Germany since January 1, 2013, and are currently living in Germany. M5 added another 1,519 households of refugees who have migrated to Germany since 2013 to the SOEP framework.

2.2. Integration of respondents from PIAAC-L as Subsample N

Sample N integrated 2,314 households of former participants of the Program for the International Assessment of Adult Competencies (PIAAC and PIAAC-L) in 2017. This is the most recent addition to the SOEP-Core samples. Fieldwork in sample N was conducted between mid-March and mid-August and thus slightly later than the majority of samples A–L1. More information on the PIAAC-L project can be found on the project homepage.

3. Translation errors in some questionnaire languages

In the IAB-BAMF-SOEP Survey of Refugees (M3-M5), there were translation errors in some some of the questions on income components in translated versions of the household questionnaire. Answers for these variables are therefore not comparable with other answers. The corresponding variables were set to -3.

4. Deletion of interviews not conducted in line with the standards of the IAB-BAMF-SOEP group in the IAB-BAMF-SOEP Survey of Refugees (M3/M4)

In the process of data preparation, three interviewers were identified who had not conducted interviews in line with the standards of the IAB-BAMF-SOEP group (more information here). The interviewers in question were responsible for 88 households in 2016 and 112 households in 2017. The households affected in the first wave of the survey (2016) were completely removed from the dataset. The households affected in 2017, who were supposed to be interviewed for the second time, were deleted for 2017 but left in the dataset for 2016. There are no indications that the first interviews (by a different interviewer) were not conducted in line with IAB-BAMF-SOEP standards. The interviews and cases deleted from the data release may be accessed upon request from a guest work station at the SOEP-RDC for survey methodological analysis. After these lines were deleted from all datasets, the following adjustments were made:

  • The deletion of the household and individual interviews required an update of the weights (dataset HHRF and PHRF), which now take account of the slightly reduced case numbers in survey years 2016 and 2017.
  • Update / inclusion of the new weights in the datasets BGPEQUIV and BHPEQUIV.

5. Extended variable naming convention

The extended variable naming convention is applied only to data sets from wave BH onwards and only applicable for the datasets $P, $H, $KIND. We added underscores between unit of analysis, question identifier, and item identifier to clearly separate the analysis unit, question, and item visually. In addition, a questionnaire identifier was introduced, which is also separated by an underscore from the item. This new version of naming variables is only used if the survey instrument differs from the “original” SOEP-Core instrument.

Due to our different samples in the SOEP, there are some respondents that receive sample-specific questions, such as the refugee sample that started in 2016. For that specific group, we created an extended individual questionnaire with some specific questions along with the standard SOEP questions that are asked every year. For the specific questions, you can use the instrument variable to see the source of the variables.

Examples and more detailed descriptions can be found in the chapter on this subject in the SOEP Companion.

6. Changes in specific variables

  • New variables for interview year: HIYEAR in HGEN, HPATHL, and PIYEAR in PGEN, PPATHL. These new variables indicate, for all survey years, the household and individual interviews that were finalized after (or before) the survey year (variable syear), which is the reference year for the questionnaires and for data collection.


6.1. Dataset PPATH / PPATHL (in raw: PPFAD)

6.1.1. SEXOR

  • The previous data release was the first to include the variables SEXOR (sexual orientation) and SEXORINFO (source of information on sexual orientation). The value -1 “insufficient information” has been changed to 2 “insufficient information”.

6.1.2 PARINFO

  • The value -1 “unclear” has been changed to 5 “unclear”.

6.1.3 Migration information

  • The coding of GERMBORN, CORIGIN, IMMIYEAR, and MIGBACK was changed for inconsistent cases (for more information, see the PPATH/PPFAD documentation).

6.1.4. Asylum-Seekers and Refugees

  • The variables for asylum-seekers and refugees [AREBACK, AREFINFO] have been renamed (in v33: REFBACK, REFINFO) and revised. The variable AREFINFO now also allows identification of specific subgroups (more information is available in the documentation).

6.2. Dataset PGEN

6.2.1 Partner pointer

  • For the variable PGPARTZ (PARTZ$), the value -1 (“no answer”) has been replaced by the correct value 5 (“unclear”).
  • Starting with wave BH, the new quality control processes implemented in generating the partner indicator have improved the quality of data from previous waves:
    • Contradictory answers between partners regarding their relationship have been identified and corrected.
    • Partnerships with differing partner indicators (1 “spouse” or 2 “life partner”) within a relationship have been identified and corrected.
    • Errors in the assignment of PARTZ values (1 “spouse” and 2 “life partner”) due to different filter routing in the different survey instruments have been corrected. Marriages were asked differently in the individual biography questionnaire for Sample J+K and in the individual questionnaire for Samples A-I. Separating out samples J and K played a key role in this correction, since these two led to errors due to their different filter routing.
    • Partnerships with recently deceased individuals were identified and deleted.
    • Respondents’ data on divorce, separation, or the death of a life partner within the past year have been taken into account for the first time in the process of generating the data.
    • For the first time, family status, civil status, partner’s first name (permanent partner number) and place of residence of the partner in the case of refugees’ partnerships (Samples M3-M5) have been taken into account in addition to interviewer given relationships between the different houshold members.

6.2.2. Volunteer work and side jobs

  • The PGEN (raw: $$PGEN) files contain nine new variables. In 2017, the SOEP fundamentally revised how respondents were surveyed about side jobs. Now, for the first time, respondents can provide answers on three different side jobs. They can also now differentiate the type of side job, whether volunteer work (variables HONOR1, HONOR2, HONOR3) and whether they are working for an employer or working freelance (SNDTYP1, SNDTYP2, SNDTYP3). The amount of gross additional income from side jobs is provided as imputed information (SNDJOB1, SNDJOB2, SNDJOB3).
    • SNDTYP117 : First side job occupational status
    • SNDTYP217 : Second side job occupational status
    • SNDTYP317 : Third side job occupational status
    • SNDJOB117 : Current gross additional income from side job 1 (gen.) in euros
    • SNDJOB217 : Current gross additional income from side job 2 (gen.) in euros
    • SNDJOB317 : Current gross additional income from side job 3 (gen.) in euros
    • HONOR117 : Volunteer work 1
    • HONOR217 : Volunteer work 2
    • HONOR317 : Volunteer work 3


6.2.3. Educational degrees

  • In v34, CASMIN and ISCED are based on additional information on educational degrees obtained abroad. Hence, some individuals with degrees from abroad display higher ranks in v33 than in v34.
  • The error in the CASMIN variable in v33 is fixed: In v33, individuals with 2c_voc (vocational maturity certificate) were mistakenly categorized as 2c_gen (general maturity certificate).

6.2.4. AUTONO

  • The generation of autono was discontinued in 2017 due to the difficulty in comparing this variable with the usual models of autonomy. Work is currently underway to introduce comparable definitions of autonomy.


6.3 Dataset PEQUIV

  • The PEQUIV (raw: $$PEQUIV) files contain six new variables. These are:
    • IAUS117 : Pensions from another country
    • AUS217 : Widows / orphans pension from another country
    • ASYL17 : Asylum-seeker benefit
    • FASYL17 : Imputation flag: Asylum-seeker benefit
    • EDUPAC17 : Benefits from the educational package
    • FEDUPAC17 : Imputation flag: Benefits from the educational package
    For more details, see the SOEP Survey Paper: Codebook for the $PEQUIV File 1984-2017.

6.4. Dataset BIOAGEL and BIOPUPIL

  • Variables from questionnaires given to 12-year-olds and 14-year-olds are now provided in BIOPUPIL dataset to reflect the differences in survey mode (parents being asked questions about their children vs. children being surveyed directly).
  • Variables from additional questions in refugee samples are integrated in BIOAGEL and BIOPUPIL datasets.


6.5. Dataset HGEN

A number of changes have taken place in recent years in questions on home rental. The first change took place in the hosehold questionnaire of wave BF (2014). The question asked about the costs of utilities in such detail that respondents were not able to provide correct answers. This led to underestimation of both base rent and utilities.

It emerged that this led to a slight break in the time series. Rent has increased continuously over the years since 1984. In 2014 and 2015, however, rental costs fell and have been increasing again sharply since 2016. This break can be explained by the change in the questionnaire.

Starting with wave BH, respondents are being asked about rent in the same way as in wave BG (2016) and in wave BD (2013) in order to maintain long-term comparability. In addition, with wave BH, the new migration sample M5 and the new refresher sample N are part of the SOEP. Since Sample M5 was not surveyed on utility costs in a comparable way and since many of these respondents probably live in group housing or receive subsidies to cover living costs, no rent variable was generated for them.

v33 - rent

v34 - rent

2010: 486.25

2010: 486.21

2011: 484.93

2011: 485.64

2012: 491.01

2012: 490.75

2013: 505.00

2013: 505.59

2014: 470.95

2014: 473.74

2015: 507.06

2015: 508.57

2016: 545.53

2016: 541.90

 

2017: 550.67

6.6. Dataset BIOIMMIG

  • The population of BIOIMMIG shrunk due to a change of coding of BIIMGRP (for more information, see the BIOIMMIG documentation)
     

6.7. Dataset HHRF/PHRF

  • New variables in HHRF: BHHHRF, BHHBLEIB, BHHHRFAM4, BHHHRFM5, BHHHRFN
  • New variables in PHRF (and ENUMHRF, available on request): BHPHRF, BHPBLEIB, BHPHRFAM4, BHPHRFM5, BHPHRFN
  • Please note that with our new integrated data format, you’ll find all weighting variables now directly in PPATHL or HPATHL.
  • On request, we provide stand-alone weighting variables (BHPHRFM35, BHHHRFM35) for the refugee samples M3, M4, and M5.

6.7.1. Revisions and Bugfixes

  • Due to confusion in the country codes for Iran and Russia in the sampling frame (Central Register of Foreign Nationals, AZR), design weights for Samples M3 and M4 as well as their cross-sectional weights for wave BG had to be updated.
    In Wave BG, we interpreted the population of samples M3 and M4 as refugees who immigrated to Germany between January 2013 and January 2016. In fact, only those refugees whose registration at the Central Register of Foreign Nationals (AZR) took place until April 2016 were included in those samples. In Sample M5, among others, those refugees were interviewed who, although they had immigrated in the same period, were registered later. For this reason, the total for the post-stratification of the second wave of M3 and M4 has been reduced by the number of refugees with a later registration date.
keyboard_arrow_up