Soc pdf

Two reviewers will examine full-text versions of the remaining, potentially relevant articles, for eligibility. See Appendix 2 for our full-text hierarchical exclusion criteria tool.

We will resolve disagreements regarding study eligibility through discussion, referring to an independent arbiter. JBI Separate forms will be used for differing types of studies, including: 1 studies with a control group; 2 studies without a control group; 3 studies with a qualitative design; 4 studies using systematic review methods; and 5 studies providing economic analysis.

The forms will be selected and adapted after conducting an interactive workshop with several academics with relevant methods and domain expertise. We will extract data on key study components relating to the research question, including features of the study design, population characteristics, funding instrument characteristics, study outcomes, benefits, and adverse events and perceptions.

We are also exploring the suitability of machine learning tools to facilitate data management and extraction. Missing data will be requested by the study authors via email. Where data remain unavailable, we will analyse the available data and discuss the potential impact of the missing data.

Disagreements between reviewers will be resolved by discussion; a third reviewer will be consulted where necessary.

As the evidence base is still developing, studies will not be excluded on the basis of quality. Regardless, the quality appraisal step will be integral for assessing any potential bias and considering the extent to which it is possible to have confidence in study findings Gough et al.

For quantitative studies, we will use Cochrane recommended risk of bias tools. We will assess the risk of bias in each randomised study using a tool such as Version 2 of the Cochrane risk-of-bias tool for randomized trials RoB 2 Sterne et al. For both randomised and non-randomised studies, we will compare outcomes reported in the study protocol where available and the published report to further assess publication bias and outcome reporting bias.

Where a protocol is unavailable, we will compare the outcomes reported in the methods and results sections. For the qualitative component, we will conduct quality assessment using a critical appraisal tool developed specifically for qualitative research studies, as recommended by Hannes Further, it is likely that there will be significant variation in the types of outcomes measured and the intervention, settings and populations are also likely to differ considerably.

The heterogeneity across studies may make it inappropriate to pool the results. Only where studies have used a similar intervention programme and funding contract , along with the same outcome measure, will we pool the results.

Specifically, we will pool the results of randomised controlled trials, if possible, using a random-effects meta-analysis. Non-randomised studies may be included in the meta-analysis if we identify fewer than three randomised or quasi-randomised trials.

We will use RevMan 5 software to conduct the meta-analyses. For continuous outcomes, we will calculate standardised mean differences and for binary outcomes, we will calculate risk ratios RRs. If the effects of clustering have not been taken into account in a study, we will adjust the standard deviations SDs for the design effect, using intra-class coefficients, if they are provided in the study reports.

If they are not provided, we will use external estimates obtained from similar studies. Where required, we will log transform skewed data. To identify common categories and themes, we will undertake thematic content analysis. This will involve the following three stages outlined by Thomas and Harden : 1 coding text; 2 developing descriptive themes; and 3 generating broader analytical themes.

Each of these steps will be independently undertaken prior to consolidating content; a sample will be validated by a second reviewer with any disagreements to be discussed and resolved by a third reviewer.

The approach to coding and analysing qualitative research will be refined in response to recommendations from the PAG and the nature and scope of identified studies.

Our approach is informed by Gomersall et al. We do not expect to generate an average generalisable incremental cost- effectiveness measure. We instead aim to identify the range and quality of available studies Social Science Protocols, September , We propose to adopt the JBI critical appraisal checklist for studies reporting economic evaluations and the three-by- three dominance ranking matrix tool for synthesising and interpreting findings from economic evaluations.

We will compare the findings from the qualitative review against the findings of the quantitative review and the findings of the economic review, using a conceptual and methodological matrix. This will allow us to identify synergies, conflicts, and research gaps. We will explore a range of methods that go beyond measuring and reporting on programme effectiveness.

The realist approach to research synthesis is aligned to many questions of importance to policymakers, as it aims to explain the link between the intervention context, the mechanisms by which it works and the outcomes produced Pawson et al.

We will repeat the primary statistical analysis or meta-analysis, substituting alternative decisions or ranges of values for decisions that were arbitrary or unclear. We will present the results of the sensitivity analysis in a summary table. For studies reporting individual person- level outcomes, we will treat the individual study participants as the unit of analysis.

If any multi-arm studies meet the inclusion criteria, we will combine the groups in order to create a single pairwise comparison Higgins et al. Where this is not possible, we will select the treatment group receiving the most intense level of intervention, and the control group receiving the least intervention, from each study.

Studies reporting contract or system level outcomes will include a comparator of at least one other, more conventional funding approach, allowing for a comparison of the effect of the different funding mechanisms. We will present the assessments in a summary of findings table.

Discussion This mixed-methods systematic review will provide a detailed synthesis of the existing evidence on the effectiveness of SOC approaches, thereby informing future commissioning decisions and improving a broad range of individual and system level outcomes.

It will provide empirical insights on whether, when, and where and if possible, how SOC approaches deliver improved impact when compared to more conventional funding arrangements. This will support policymakers to make more informed decisions in relation to commissioning and funding approaches. Competing interests: Senior members of the review team are regularly invited to present at international conferences and events on the application of SOC, including SIBs and PbR.

Ecorys conducts research and evaluations internationally in the field of SOC and holds contracts with a range of public and private organisations to undertake this work. Similarly, the GO Lab has been funded by government departments and independent foundations to investigate the application of impact bonds and outcome contracts.

Some members of the review's PAG, who are invited to make recommendations on the review scope and reporting, are previous or existing clients and some could potentially be clients for future opportunities. The funders have not been involved in developing this protocol. Authors' contributions: EC is the guarantor. VP drafted the manuscript drawing on background notes prepared by EC.

EC provided guidance on various components of the manuscript, as required and contributed to the background section and economic analysis section. EC reviewed the search strategy. RW developed the strategy for identifying grey literature. JR led the establishment of the PAG. All authors read, provided feedback, and approved the final manuscript.

Acknowledgements: This review is jointly conducted by the Blavatnik School of Government at the University of Oxford and Ecorys research and consultancy company. We thank all of the PAG members who have agreed to contribute to this review. We also thank Professor Carolyn J. References Albertson, K. Policy Press. Bovaird, T. Outcome-based service commissioning and delivery: Does it make a Difference?

Van De Walle Eds. Emerald Group Publishing Limited. Disappointment by results. Institute for Government. Development, testing and use of data extraction forms in systematic reviews: A review of methodological guidance.

Building the tools for public services to secure better outcomes: Collaboration, prevention, innovation. Government Outcomes. Payment by results in international development: Evidence from the first decade. Development Policy Review, 37 6 , CASP qualitative studies checklist. Deakin, N. The enabling state: The role of markets and contracts. However, because a title such as "painter" could be classified in more than one occupation, including Fine Artists, Including Painters, Sculptors, and Illustrators, Painters, Construction and Maintenance, or Coating, Painting, and Spraying Machine Setters, Operators, and Tenders, the title "painter" is not a direct match.

Requests for additions to the DMTF. Interested parties should submit their request and materials in support of their recommendation to the SOCPC. Supporting materials should include the specific job title in question and the specific SOC code and title believed to be a direct match. The SOC is a task-based classification that does not differentiate occupations based on education or certification, but rather on the work performed.

Interested parties should submit their request and materials in support of their recommendation to the SOCPC. Supporting materials should include the specific job title in question and the specific SOC code and title believed to be a direct match.

The SOC is a task-based classification that does not differentiate occupations based on education or certification, but rather on the work performed. Therefore, the SOCPC will require information regarding the work of individuals with that particular job title.

Once all of the materials have been received, the SOCPC will consider the activities involved in the specific job, and whether these activities are unique to that job.