feat(script): Add validity method explanation and appendix

scoping_review.qmd

@@ -282,11 +282,12 @@ There is also low evidence for return to work being increased by education, work
 
 # Methodology and data
 
-<!-- {{++ TODO: besides scoping, introduce systematic review considerations applicable: Cochrane, PRISMA ++}} -->
-
-## The search protocol
-
 This section will discuss the systematic scoping review methodology that is proposed to conduct the review of the literature on policy interventions that are expected to address inequalities in forms of work and labour market outcomes.
+This study follows the principles of a systematic review framework, to systematically assess the impact of an array of policies on inequalities in the world of work.
+It strives to follow the clear and reproducible method of identification prior to synthesis of relevant research,
+while limiting "bias by the systematic assembly, critical appraisal and synthesis" through applying scientific strategies to the review itself [@Cook1995].
+It thereby attempts to provide an improved basis for comparative analysis between studies through the rigorous application of systematic criteria and thus to avoid the potential bias of narrative reviews.
+
 Unlike purely systematic reviews which typically focus on specific policy questions and interventions, systematic scoping reviews focus on a wider spectrum of policies, where different study designs and research questions can be investigated.
 Since scoping reviews allow both broad and in-depth analyses, they are the most appropriate rigorous method to make a synthesis of the current evidence in this area [@Arksey2005].
 
@@ -295,6 +296,8 @@ It does so through a breadth-first approach through a search protocol which favo
 Its purpose, clearly mapping a body of literature on a (broad) topic area, is thereby useful on its own or in combination with a systematic approach [@Arksey2005].
 With an increasingly adopted approach in recent years, with rigorous dichotomy of inclusion and exclusion criteria it provides a way of charting the relevance of literature related to its overall body that strives to be free of influencing biases which could affect the skew of the resulting literature sample [@Pham2014].
 
+## The search protocol
+
 <!-- TODO need correct above definitions -->
 The search protocol was carried out based on the introduced areas of policies as well as the possible combination of definitions and outcomes in the world of work.
 For each dimension of definitions, a cluster containing possible utilized terms will be created, that is for: definitions of work and labour, forms of work, definitions of inequality, forms of vertical and forms of horizontal inequalities, labour market outcomes, and definitions of policy.
@@ -302,10 +305,10 @@ Each of the clusters contains synonymous terms as well as term-adjacent phrase c
 
 <!-- TODO Why WOS database? -->
 The search protocol then follows a three-staged process of execution: identification, screening and extraction.
-First, in identification, the above categorizations are combined through Boolean operators to conduct a search through the database repository Web of Science.
+First, in identification, the relevant policy, inequality and world of work related dimensions are combined through Boolean operators to conduct a search through the database repository Web of Science and supplemental searches via Google Scholar to supply potential gray literature.
 While the resulting study pools could be screened for in multiple languages, the search queries themselves are passed to the databases in English-language only.
-<!-- TODO will we be using gray lit? -->
-Relevant results are then complemented through the adoption of a 'snowballing' technique, which analyses an array of published reviews for their reference lists to find cross-references of potentially missing literature.
+Relevant results are then complemented through the adoption of a 'snowballing' technique,
+in which an array of identified adjacent published reviews is analysed for their reference lists to find cross-references of potentially missing literature and in turn add those to the pool of studies.
 
 To identify potential studies and create an initial sample, relevant terms for the clusters of world of work, inequality and policy interventions have been extracted from the existing reviews as well as the ILO definitions.
 Identified terms comprising the world of work can be found in @tbl-wow-terms,
@@ -446,6 +449,16 @@ flowchart TD;
 All relevant data concerning both their major findings and statistical significance are then extracted from the individual studies into a collective results matrix.
 The results to be identified in the matrix include a study’s: i) key outcome measures (dependent variables), ii) main findings, iii) main policy interventions (independent variables), iv) study design and sample size, v) dataset and methods of evaluation, vi) direction of relation and level of representativeness, vii) level of statistical significance, viii) main limitations.
 
+Finally, following @Maitrot2017, the relevant studies are ranked for their validity.
+Here, a 2-dimensional approach is taken to separate the external validity from the internal validity of the studies.
+The ranking process then uses the representativeness of a study's underlying dataset,
+from a non-representative survey sample, through a sub-nationally representative sample, a nationally representative and the use of census data,
+to arrive at a ranking between 2.0 and 5.0 respectively.
+Similarly, the studies are ranked for internal validity using the study design,
+with only quasi-experimental and experimental studies receiving similar rankings between 2.0 and 5.0 depending on the individually applied methods due to their quantifiability,
+while observational and qualitative studies go without an internal validity rank (0.0) due to the more contextual nature of their analyses.
+For a full list of validity ranks, see @sec-appendix-validity-rankings.
+
 ## Data
 
 ```{python}
@@ -1720,4 +1733,27 @@ print(f"""
 """)
 ```
 
+## Validity rankings {#sec-appendix-validity-rankings}
+
+| Representativeness                  | Ranking |
+| ---                                 | ---     |
+| non-representative survey           | 2.0     |
+| subnationally representative survey | 3.0     |
+| nationally representative survey    | 4.0     |
+| census-based dataset                | 5.0     |
+
+: External validity ranking. Adapted from @Maitrot2017.
+
+| Method                                         | Ranking |
+| ---                                            | ---     |
+| ordinary least squares                         | 2.0     |
+| discontinuity matching                         | 3.0     |
+| difference in difference (& triple difference) | 3.0     |
+| propensity score matching                      | 3.5     |
+| instrumental variable                          | 4.0     |
+| regression discontinuity                       | 4.5     |
+| randomised control trial                       | 5.0     |
+
+: Internal validity ranking. Adapted from @Maitrot2017.
+
 {{< pagebreak >}}