Evidence-Based Practice

Is it peer-reviewed?

Peer-reviewed articles undergo a rigorous evaluation process by experts in the field, ensuring a higher standard of quality and reliability, but not all journals go through a rigorous peer-review process. If you are unsure, there are several ways to discover the peer review process. 

• Visit the website of the journal or publisher: Many reputable journals explicitly state that they use a peer-review system to evaluate and select articles. These can be found in the "About Us" or "Author Guidelines" sections on the journal's website. Peer-reviewed journals typically outline their editorial policies, including details about the review process.
• Utilise database tags and filters: When you find an article in a database, there may be indications of whether the journal is peer-reviewed. Look for symbols like a referee jersey icon, "Peer-Reviewed" tags, or similar indicators in the database interface. Some databases have filters or options to display only peer-reviewed content.
• Confirm via Ulrichsweb global serials directory: A comprehensive database that provides information about journals, including whether they are peer-reviewed. You can search for the specific journal title to find details about its editorial policies. The Library provides access to Ulrichsweb via databases search page. 

You may wish to take this process a step further by analysing the resource metrics to assess the quality, influence, and credibility of the resource.

Consider the elements of the resource

• Abstract: this is what the author wants the reader to take away from their article - what is the starting point? ​ ​
• Introduction: provides background and a starting point - how does it guide the reader?​ ​
• Materials and methods: often overlooked but very important - is the methodology understandable, reproducible, direct, and robust?
• Results: summary and analysis of the data but the statistical reporting is just as important as the words: What do the tables, figures, and legends actually report? what do you think the data means? Decide before reading the discussion​. 
• Conclusion: were the objectives achieved? Were the hypotheses tested? How do the results relate to other studies you have found?​ ​Do the authors openly discuss any limitations of their study?​ ​What else needs be studied in the future?

Question the research

The following questions may be asked to appraise the validity of research:

1. What is the research question? Are the objectives of the study clearly stated? Why was this research necessary?
2. Is the research original or important? Does the study have new findings? Is a treatment outcome clinically relevant?
3. Does the research question consider the group or population of patients, the intervention or therapy, and the outcome.
4. Did the authors use the relevant type of study for the research question?
5. Did the study design minimise the risk of bias in its methodology, reporting, and patient selection? Did the study use best practice design such as a randomised controlled trial or a systematic review? Blinding of patients and outcome assessors, randomization, concealment, intention-to-treat analysis, similarity of patients for known prognostic factors, and completeness of follow-up are indicators of validity.
6. Was the study designed in line with the original protocol? Is the focus of the report in keeping with the study objectives? Were changes made to the inclusion or exclusion criteria?
7. Has the study's hypothesis been tested?
8. Is the analysis of the data accurate? What level of uncertainty surrounds any results?
9. Are the conclusions based on the data and analysis? Do the authors draw conclusions that are supported by the data? Have the authors discussed other work that both supports and contradicts their findings? Have the authors identified any limitations to their study?
10. Does the study contribute to the understanding of the problem being investigated? What are the strengths and limitations of the study? Are the findings of the study useful for clinical practice? Do the risks of a treatment or diagnostic procedure outweigh the potential benefits?

Remember that while general principles apply, critical appraisal may be a little different for each type of study.

Critical appraisal tools

The following tools are available to assist in the critical appraisal of resources: 

Resource metrics 

Metrics are useful in assessing the quality, influence, and credibility of academic resources. They help researchers gauge the significance of a work, evaluate author expertise, and understand the reputation of the journals in which research is published, aiding in informed decision-making about resource reliability and relevance.

• Citation metrics measure how often a work is cited
• Author impact assesses an individual's scholarly impact
• Journal quality rankings evaluate the reputation of a publication.

These metrics provide valuable quantitative data that aids researchers in making informed decisions about the reliability and significance of the information they incorporate into their work.

Citation metrics

Citation metrics involve counting how often a particular work is cited in other academic publications. These metrics are used to measure the impact and influence of a specific piece of research within the scholarly community. High citation counts typically suggest that a work has been widely recognised, referenced, and deemed influential in its field. Researchers and institutions often use citation metrics to assess the significance and relevance of academic contributions.

There is no single citation analysis tool that collects all publications and their cited references. For a thorough analysis of the impact of an author or a publication, search multiple databases to find possible cited references.​ ​

A number of resources are available via the Library that help identify cited works. These include Web of Science, Scopus, Google Scholar, Murdoch Research Portal, and other databases with limited citation data. 

Author impact metrics

Author metrics evaluate the scholarly output and impact of individual authors. These may include metrics like the h-index, which considers both the number of publications and their citation impact. Author metrics help assess an author's contribution to their field and their standing in the academic community.

Locate an author on databases such as Scopus or Web of Science to discover:

• how many of their publications have been indexed by Scopus
• how many times their publications have been cited
• which of their publications are most highly cited
• who they have co-authored papers with
• their publication and citation trends for the past nine years
• compare the research output of more than one author
• their h-index.

The h-index

The h-index is an author metric that measures both the productivity and citation impact of an author's publications. This metric attempts to measure an individual's impact on the research community based upon the number of papers published and the number of citations these papers have received.

The method of calculating a h-index is the number of articles in a database that have received the same number or more citations over time. As an example, a researcher with an h-index of 20 has (of their total number of publications) 20 papers which have been cited at least 20 times each. Therefore, a h-index is not skewed by a single highly cited paper nor by a large number of poorly cited papers.

Note: The h-index may vary between databases as it is calculated by publications and metrics within that particular database. For a thorough analysis of the impact of an author or a publication, look in multiple databases to compare author h-indices.​ ​

Journal quality rankings

Journal quality rankings are used to evaluate an academic journal's impact and quality. These metrics measure the place of a journal within its research field, the relative difficulty of being published in that journal, and the prestige associated with it. Common metrics include the journal impact factor, which assesses the average number of citations to articles in a journal. 

Databases such as Scopus and Web of Science (including CAB Abstracts) can be used to determine the quality of journals in a discipline or field of research. Each database has its own way of measuring impact and you may need to consult and compare multiple database sources. 

CiteScore provided via Scopus calculates the average number of citations received in a calendar year by all items published in that journal in the preceding three years. The calendar year to which a serial title’s issues are assigned is determined by their cover dates, and not the dates that the serial issues were made available online.

Note: CiteScore and CiteScore percentile should not be used to compare journals from different subject areas as they are not field-normalised.

Library guides

The Library provides detailed methodology on a range of bibliometrics including citation metrics, alternative metrics, researcher impact, journal quality and impact, book quality and impact, and university rankings - including access to a number of resources via the below guides:

Note taking 

Taking clear, legible notes will help to focus your critical reading and analysis of your literature review sources. When taking notes, avoid plagiarism by:

•  keeping track of the difference between information from your sources and from your own ideas
•  providing clear references, including page numbers.

Note taking methods

Some effective methods of note-taking include:

• Outlining method: Use headings, sub-headings and bullet points to organize topics
• Cornell method: Use two columns - in one column write your summary of the authors' conclusions and evidence, and in the other column write down your own analysis and other comments
• Charting method: Create a list of topics or points you want to write about and use a column for each one. As you read, add references and make notes in the appropriate column
• Sentence method: Simply write down new ideas and bits of information as numbered sentences
• Mapping method: Write down key concepts and terms, with related ideas that radiate rom them.

You may consider using the matrix below for your note taking and analysis:

Critical Reading & Analysis Checklist

1. Does your literature review highlight flaws, gaps, or shortcomings of specific texts or groups of texts?
2. Have you identified areas that have not yet been researched or have not yet been researched sufficiently?
3. Does the literature demonstrate a change over time or recent developments that make your research relevant now?
4. Are you able to discuss research methods used to study the topic and/or related topics?
5. Can you clearly state why your research is necessary?

Your interpretation: 

• Read critically​ ​
• Note 2-4 bullet points for each study that summarises the main points and conclusions​ ​
• Use a matrix to analyse findings, relevance, and importance of each text​ ​
• Draw attention to studies that are important, influential, or that bring a new understanding or method of studying your area of research.