Download the framework

Download the framework into your preferred offline format.

Use this to help you assess your skills or to produce a development plan.

Download as PDF Download as CSV

Evidence Synthesis Skills

Framework last updated: Tue Sep 16 2025

Printed on:

Available at:

Category: Conducting research

Evidence synthesis involves systematically developing a protocol and research question, comprehensively searching and screening literature, managing references, extracting and assessing data for bias, and synthesizing findings to answer a specific question.

This process ensures transparency, rigor, and a clear summary of available evidence.

Skill: Developing a research question

The research question is the crucial first step that sets the direction for the entire review.

It involves clearly defining what you want to investigate, often using frameworks like PICO (Population, Intervention, Comparison, Outcome) to structure the question and specify its scope.

The question must be specific enough to be answerable yet broad enough to be meaningful.

This process ensures the review is focused and methodologically sound, guiding decisions on which studies to include and how to analyze the evidence.

A well-crafted research question is essential for producing a relevant and reliable evidence synthesis.

Level	Descriptor
1	Understands that evidence syntheses should be based on an initial, clearly defined research question.
2	Demonstrates basic knowledge of one or more structured frameworks for developing research questions that are suitable for evidence synthesis e.g. PICO, SPIDER, PECO, ECLIPSE, SPICE, PerSPEcTiF or PCC. Can suggest an appropriate review type (e.g. evidence map, systematic review, umbrella review) to answer a question.
3	Can explain how research questions map onto structured frameworks for evidence synthesis.
4	Demonstrates ability to refine a given topic into specific research question(s) which the evidence synthesis project will seek to answer (working with a team comprising topic experts and evidence synthesis specialists).
5	Advises on ways to improve the ability of others’ proposed research question to answer the basic problem, including splitting or broadening the review question. Develops and/or delivers training to help people formulate good research questions and identify the appropriate structured frameworks to frame those questions.
6	Approves or authorizes research questions for evidence syntheses within your organization.
7	Works with partners and other stakeholders, to set priorities around research questions to be investigated. Leads and promotes the development of good research questions on a national and international basis.

Skill: Designing a protocol for evidence syntheses

The evidence synthesis protocol is a foundational step that serves as a blueprint for the research team.

It clearly defines the research question, search strategy, screening methods, criteria for inclusion and exclusion, data extraction methods, and synthesis approach.

It ensures transparency, reproducibility, and reduces bias by detailing all steps before the review begins.

Often protocols are registered to avoid duplication of effort and to support open science practices.

Level	Descriptor
1	Understands and can describe in general terms the process of evidence synthesis. Can explain the purpose of a protocol as a foundation for evidence synthesis. Demonstrates awareness of different evidence synthesis types such as scoping reviews, systematic reviews, rapid reviews, systematic maps, and umbrella reviews.
2	Follows instructions to develop one or more protocol sections. Seeks guidance from mentors or peers to refine plans. Understands the difference between methodology and reporting guidelines and uses each appropriately.
3	Develops methods section of protocol for evidence synthesis in collaboration with a multidisciplinary team.
4	Provides guidance to research team members in protocol development. Designs research that follows a methodology, meets reporting standards and prioritizes integrity and transparency.
5	Drafts high-quality protocols ready for approval. Takes responsibility for the protocol moving forward to registration/publication.
6	Can adapt knowledge of evidence synthesis methodology to different situations and suggest alternative approaches and important differences between these. Able to lead the design, development and quality control of research methodologies across a team or organization.
7	Leads and promotes good protocol design on a national and international basis.

Skill: Searching the literature

Literature searches are systematic and comprehensive efforts to identify all relevant studies on a specific research question.

The process involves developing a detailed search strategy, using keywords, subject headings and Boolean logic to optimize search sensitivity and specificity.

Searches typically include multiple databases and grey literature sources to reduce publication bias and ensure a thorough evidence base.

The quality and scope of the literature search are crucial, as they directly impact the completeness and reliability of the resulting synthesis.

Level	Descriptor
1	Describes the basic concepts of literature searching for evidence synthesis. Understands that the search should be comprehensive and reported transparently to enable reproducability Understands that the search is part of the foundation of evidence synthesis.
2	Understands the difference between a search engine such as Google Scholar and a database/index. Creates basic searches using search terms and Boolean logic. Understands what grey literature is and its importance.
3	Drafts comprehensive search strings with supervision, identifying, consulting and adapting other people’s search strings where appropriate. Liaises with topic and searching experts to refine the search strategy. Demonstrates ability to construct basic search strings to conduct grey literature searches on organizational websites. Can explain why benchmark papers are a useful component of literature searching. Contributes towards protocol methods regarding literature searching.
4	Independently creates and documents transparent, reproducible search strings, based on eligibility criteria and terms provided by subject experts, using appropriate field tags, subject headings, and Boolean operators. Demonstrates ability to adapt search strings for different sources (e.g. databases, indexes or websites). Understands conduct and reporting standards for literature searching and drafts high-quality literature searching methods. Interrogates lists of benchmark papers against search results to determine why any specific paper may not have been included in the search results. Either adapts the search to ensure they are returned or explains why the search cannot, or should not, be adapted. Includes detailed information on the use of a range of platforms and sources within the subject domain. Carries out forward and backward citation chasing on included papers.
5	Advises and guides evidence synthesis teams on constructing searches and best practice in searching for literature. Provides robust and insightful comments on other people’s search strategies including peer review. Leads on protocol methods regarding literature searching. Designs and/or delivers training programmes on literature searching within evidence synthesis projects.
6	Sets standards within your organization regarding the transparent and robust searching of literature.
7	Leads and promotes the development of literature searching as a rigorous and reproducible practice on a national and international basis.

Skill: Reference management

References and citations of identified literature should be systematically collected, organized, and tracked during the search process.

Citation management software may be used to import, store, and de-duplicate search results from multiple databases, ensuring a clean and efficient workflow.

These tools also facilitate collaboration among team members, allow for easy sharing of reference libraries, and support the automatic generation of in-text citations and bibliographies in manuscripts.

Effective reference management is essential for maintaining transparency, accuracy, and reproducibility throughout the evidence synthesis process.

Level	Descriptor
1	Aware that different forms of reference management software exist to assist with managing search results including citation management tools (e.g. Zotero or Mendeley) and screening software (such as Rayyan or Covidence). Explains why screening results of literature searches needs to be done accurately with the records managed and recorded carefully.
2	Uses reference management software to store and organize the results of database searching.
3	Understands the range of tools available along with their pros and cons. De-duplicates search results using reference management tools. Finds and removes retracted papers.
4	Drafts record management section of protocols and final evidence synthesis report, including search results at each stage of the screening process, following an agreed standard format (e.g. PRISMA, ROSES).
5	Provides high-quality advice and guidance on reference management sources and best practices for managing and presenting search results to teams. Designs and/or delivers training on reference management.
6	Sets standards and devises workflows within your organization regarding the transparent and robust management of search results.
7	Leads and promotes the development of reference management on a national and international basis.

Skill: Screening the literature

Literature screening in evidence synthesis is a two-stage process used to identify studies that meet pre-defined inclusion and exclusion criteria.

First, titles and abstracts are screened for relevance. Then, full texts of potentially eligible studies are reviewed in detail.

At least two independent reviewers typically conduct screening to minimize bias and ensure consistency. Disagreements are resolved by consensus or a third reviewer.

Screening tools help to streamline the process, help manage references, and track decisions.

This rigorous approach ensures only relevant studies are included, maintaining the focus, transparency, and reliability of the evidence synthesis.

Level	Descriptor
1	Understands the importance of screening as part of the foundation for the evidence synthesis process. Aware that reliability across raters is important and can be tested.
2	Demonstrates ability to perform title and abstract screening using a screening software or spreadsheet against eligibility criteria under close supervision.
3	Demonstrates ability to perform full text screening using screening software or spreadsheets. Aware of reporting requirements (e.g. presenting a list of reasons for exclusions). Contributes towards literature screening section of protocol. Engages with topic experts to refine inclusion and exclusion criteria where needed.
4	Organizes the screening process, tests for inter-rater reliability and makes decisions around switching from screening by two independent people to screening by one person only. Drafts screening sections of protocol and drafts results of the screening for the final report.
5	Keeps up to date with, and advises on, emerging methods using machine learning and AI for evidence synthesis screening. Designs and/or delivers training on literature screening.
6	Evaluates and promotes screening methods and provides input across your organization.
7	Leads and promotes the development of screening methods on a national and international basis.

Skill: Data extraction

Data extraction in evidence synthesis involves systematically collecting relevant information from included studies using a standardized template or form.

At least two reviewers generally perform this process independently to ensure accuracy and minimize errors. The extracted data is recorded in tables or spreadsheets, allowing for easy comparison and synthesis across studies.

Data extraction is a critical step, as it provides the foundation for analysis, synthesis, critical appraisal and ultimately answering the research question.

Level	Descriptor
1	Understands the importance of accurate data extraction and piloting to increase confidence in data extraction. Understands that there are different types of data (e.g. qualitative, quantitative and mixed methods) requiring different approaches to data extraction and analysis.
2	Follows instructions to extract data from eligible articles using pre-defined data extraction tools. Aware of different tools to record extracted data and has used one or more of these.
3	Confidently extracts data from eligible articles, requesting help from team members where needed. Adapts existing data extraction templates, or creates new templates, to enable extraction of data as defined in protocols. Contributes towards protocol definition regarding data extraction.
4	Manages the data extraction process, coordinating the team and checking for a suitable level of agreement in consensus checking. Arbitrates disagreements and seeks advice from a more senior topic expert where needed.
5	Design and/or delivers training on data extraction and analysis within evidence synthesis projects. Leads on drafting data extraction section of the protocol.
6	Designs ways to improve and standardize data handling across your organization and wider afield. Keeps up to date with, and evaluates emerging methods for data extraction including machine learning and AI for data extraction.
7	Leads and promotes the development of data extraction methods on a national and international basis.

Skill: Critical appraisal

Critical appraisal in evidence synthesis is the systematic evaluation of the methodological quality and external validity in each included study, focusing more on the methods used rather than just the results.

This process helps determine the reliability and validity of the evidence by identifying potential flaws or biases in study design, conduct, or reporting.

Critical appraisal is essential for establishing the trustworthiness of the evidence and informing the synthesis and interpretation of results.

Level	Descriptor
1	Understands that study design can affect the results of primary studies, presenting a risk of bias, and these risks of bias can be assessed using an appropriate tool.
2	Recognizes potential sources of bias in research studies. Aware of and applies one or more standardized critical appraisal tools to appraise research quality with supervision/guidance.
3	Aware of and can apply multiple critical appraisal tools to appraise research quality with a greater level of independence.
4	Evaluates the outputs of teams’ critical appraisals. Drafts summaries of the critical appraisal for the final report.
5	Designs and/or delivers training programmes on critical appraisal within evidence synthesis.
6	Evaluates and promotes critical appraisal methods and provide input across your organization.
7	Leads and promotes the development of critical appraisal methods on a national and international basis.

Skill: Data analysis

The data analysis part of evidence synthesis is where the extracted data from individual studies are combined, interpreted, and presented to answer the review question.

The approach to data analysis depends heavily on the type of data collected (quantitative, qualitative, or mixed) and the specific aims of the evidence synthesis.

The analysis aims to identify patterns, consistencies, or discrepancies in the evidence.

Level	Descriptor
1	Understands that there are a range of evidence synthesis methods for analysing different types of data.
2	Contributes to basic data analysis, for example selecting elements of extracted data to be tabulated for inclusion in the report. Creates graphs such as bar charts, or choropleths to display results under direction.
3	Creates a variety of tables and graphs to present results using own initiative. Analyses, tabulates and writes up characteristics of included studies to understand the studies in the review.
4	Cleans and transforms data ready for analysis. In quantitative synthesis, interprets a broad range of statistical analyses/measures of effect (e.g. odds ratios, standard mean difference, confidence intervals, heterogeneity).
5	Conducts synthesis of extracted data (e.g. meta-analysis, meta-regression, qualitative synthesis, narrative synthesis). Understands and interprets results from data analysis. Designs and/or delivers training on data analysis/synthesis.
6	Evaluates and promotes data evidence synthesis methods across your organization.
7	Leads and promotes the development of data analysis methods on a national and international basis.

Skill: Drawing conclusions

Drawing conclusions involves synthesizing and integrating the data from included studies to answer the research question and assess the overall strength and consistency of the evidence.

This step goes beyond simply summarizing findings. It requires identifying patterns, evaluating the reliability of results, and considering the implications for practice or policy.

Conclusions may highlight robust evidence, point out uncertainties, or recommend areas for future research, ultimately providing clear and actionable insights based on the body of evidence reviewed.

The process is guided by the planned synthesis approach and aims to ensure that conclusions are well-founded, transparent, and relevant to the intended audience.

Level	Descriptor
1	Understands that conclusions should be drawn from the evidence presented.
2	Uses synthesized data as a basis to draw conclusions for the evidence synthesis.
3	Draws conclusions from synthesized data and results from critical appraisal, acknowledging limitations and uncertainties perhaps using a tool that brings together risk of bias and outcome data such as GRADE.
4	Assesses the certainty and quality of the evidence (internal validity) and considers how generalizable it is (external validity). Makes decisions that prioritize integrity and transparency throughout the evidence synthesis process.
5	Designs and/or delivers training programmes for people in your organization or externally on drawing conclusions from evidence synthesis. Takes overall responsibility for the final evidence synthesis (all authors are responsible for approving reports for publication).
6	Shares conclusions from evidence synthesis with the research community e.g. through publications, posters, or presentations.
7	Translates complex scientific findings into clear implications for policy and management and for future research for diverse audiences.

Category: Working with others

Evidence synthesis projects are rarely carried out as a solo activity. They require good communication between team members and stakeholders.

They can also become quite complex; requiring careful management of conflicting priorities, budgets and people.

Skill: Project management

Project management in evidence synthesis is essential for coordinating the complex, multi-stage workflow and ensuring the project is completed efficiently and to a high standard.

It involves careful planning, organizing, and monitoring of each step, while managing time, resources, and team roles.

Effective project management helps maintain consistency, meet deadlines, and address any challenges that arise.

Level	Descriptor
1	Follows instructions to complete tasks carefully and accurately.
2	Provides clear and accurate task updates on a regular schedule. Typically meets deadlines and communicates issues promptly.
3	Leads a particular workstream within an evidence synthesis project. Provides project updates and coordinates with other workstreams leads. Handles conflicts within the team, escalating where necessary.
4	Manages a single evidence synthesis project’s scope and timelines within agreed constraints. Liaises with project stakeholders, ensuring that they are heard and are clear on the project’s objectives and parameters. Makes decisions within tight timelines and evolving contexts.
5	Manages and plans resources, scope and timeline for a single evidence synthesis project.
6	Manages and plans resources, scope and timeline for multiple, concurrent evidence synthesis projects.
7	Allocates resources, acts as the senior executive and signs-off projects as complete.

Skill: Communication

Good communication is vital for ensuring clarity, collaboration, and transparency throughout the project.

It involves team members, stakeholders and the wider audience; often through clear documentation, presentations, and dissemination of findings.

Effective communication ensures that everyone understands roles, progress, and decisions, supports peer review and feedback, and ultimately enhances the quality, reliability, and impact of the evidence synthesis.

Level	Descriptor
1	Able to communicate well with colleagues in writing and in person
2	Contributes to research papers and attends academic conferences.
3	You are an author on at least one published research paper. Engages with your community of practice and the wider public e.g. through informal channels such as blogs and social media, or presenting at academic conferences.
4	Mentors your team to improve the quality of their communications. You deliver presentations on your evidence syntheses at regional, national or international conferences.
5	Oversees the drafting of evidence synthesis products for your organization. Engages with media specialists, such as science journalists, to raise the profile of your evidence syntheses. Delivers keynote presentations at regional, national or international conferences.
6	You consider the wider political, societal and economic situation, articulating the requirements of evidence synthesis projects and their potential benefits. You validate/guarantee/sign-off evidence synthesis products for your organization, submitting final reports for publication. Deals with fallout arising from evidence synthesis produced by your team or organization. Contributes directly to broadcast communication channels such as podcasts, radio and TV.
7	Liaises directly with policymakers, practitioners and the wider public in the dissemination of evidence synthesis products’ conclusions.