The Social Surveys Work Package (WP06) of the WorldFAIR project is focussed on the improvement of FAIR practices in the management of harmonised content in cross-national social surveys. The first report from the Work Package (Deliverable 6.1) provided an overview of the practices of comparative (cross-national) social surveys, through case studies of: (1) the European Social Survey (ESS) and (2) a satellite study, the Australian Social Survey International – European Social Survey (AUSSI-ESS). The focus of this Deliverable 6.2 is oriented towards progress on three recommendations (Rs) from that first report – the use of the DDI Lifecycle and variable cascade (R6.1 and R6.2), and requirements for formal registries of variables and reusable content (R6.5). To achieve this, this paper explores the development of recommended practices for the management and processing of cross-national survey data for the establishment of harmonised social science datasets.
In this deliverable, we outline a proposed workflow for the processing of data harmonisation of social surveys, that takes account of the practical steps required to bring diverse content together in a machine-actionable way, and that could best take advantage of external registered, persistent content. This workflow considers the core steps involved in the harmonisation process, key issues that occur in the processing of data during this process, and potential resolutions of these issues. These resolutions are all oriented towards improving FAIR practices in the harmonisation process – through the use of reusable, accessible metadata structures that can both improve processing consistency for current projects, and be applied to future harmonisation projects.
The key conclusions are two-fold. Firstly, there is a key need for the application of standardised workflows to enable consistent interaction with registry content held across multiple data repositories. The proposed workflow detailed in this report is a first effort at such a workflow model. Secondly, there is a need for consistent pre-processing of data and metadata within repositories to reduce error handling in the harmonisation process. The final section of the report provides an initial set of processing rules that could be used in such circumstances. The final third phase of this Work Package will then focus on testing this workflow and rules on new waves of data coming from the AUSSI and ESS projects.
The report is available on Zenodo.
