Critical Reading & Analysis
A critical reader:
- Does not believe everything they read
- Questions what they read
- Rereads if necessary
- Understands the influence of style
- Analyses arguments
- Discounts arguments that are unsupported or based on faulty reasoning
When reading critically, focus on the purpose of your literature review:
- Think about what you expect from the article or chapter, before reading it
- Skim the abstract, headings, conclusion, and the first sentence of each paragraph
- Focus on the arguments presented rather than facts
- Take notes as you read and start to organise your review around themes and ideas
- Consider using a table, matrix or concept map to identify how the different sources relate to each other
- Note four to six points for each study that summarises the main points and conclusions
- Be as objective as possible
Critical Appraisal
Critical appraisal is the process of carefully and systematically examining research to judge its credibility, its value and its relevance in a specific context.
The aim of critical appraisal is to understand the strengths, weaknesses, and potential for bias in the research.
Validity, applicability, and clinical importance should be considered during critical appraisal to ensure that research evidence is used reliably and efficiently and false conclusions are not drawn.
Why do we need to critically appraise the literature?
Critical appraisal is necessary to:
- Assess benefits and strengths for research against flaws and weaknesses
- Decide whether studies have been undertaken in a way that makes their findings reliable
- Make sense of the result
- Know what these results mean in the context of the clinical decision being made
- Assess the usefulness of the evidence for clinical decisions
Analysing your sources
Elements:
Abstract: this is what the author wants the reader to take away from their article - what is the starting point?
Introduction: provides background and a starting point - how does it guide the reader?
Materials and methods: often overlooked but very important - is the methodology understandable, reproducible, direct and robust?
Results: summary and analysis of the data but the statistical reporting is just as important as the words
- what do the tables, figures and legends actually report?
- what do you think the data means?*
* decide before reading the discussion
Discussion: author draws conclusions – how does this correlate with your conclusions?
Evaluation:
Consider the following criteria:
Currency |
* Is the source up-to-date? * Does it consider the latest research on your topic? |
Relevance |
* Is the article relevant to your topic? * Is the research methodology comprehensively described? * Does the work ultimately contribute in any significant way to an understanding of the subject? |
Authority |
* How reputable is the source? * Is the source peer-reviewed? * What is the source's impact factor? * Is the author from a reputable institution? * Have you seen the author cited in other sources? |
Accuracy |
* Does the data support the conclusions drawn? * Are the author's opinions and conclusions convincing? * Are the author's arguments supported by evidence (primary material, case studies, narratives, statistics, recent findings)? * Is the article properly referenced? |
Bias |
* What is the purpose of the article and its intended audience? * Can you detect any bias in the content? * Is the reporting objective? * Is contrary data considered or is certain pertinent information ignored to prove the author's point? * Which of the author's arguments are most/least convincing? |
Inference:
Were the objectives achieved?
Hypotheses tested?
How do these results relate to other studies you have found?
Do the authors openly discuss any limitations of their study?
What else needs be studied in the future?
Interpretation:
Read critically
Note 2-4 bullet points for each study that summarises the main points and conclusions
Use matrix to analyse findings, relevance and importance of each text
Draw attention to studies that are important, influential or that bring a new understanding or method of studying your area of research
Tools
Citation Analysis
Citation Analysis is the process whereby the impact or "quality" of an article is assessed by counting the number of times other authors mention it in their work.
Citation Analysis is the process whereby the impact or "quality" of an article is assessed by counting the number of times other authors mention it in their work.
Citation analysis involves counting the number of times an article is cited by other works to measure the impact of a publication or author.
The caveat however, is that there is no single citation analysis tool that collects all publications and their cited references.
For a thorough analysis of the impact of an author or a publication, look in multiple databases to find all possible cited references.
| A number of resources are available via the Library that help identify cited works including: Web of Science, Scopus, Google Scholar, and other databases with limited citation data. |  |
|
The Measure Research Quality and Impact Research Guide provides methodology on a range of bibliometrics including citation metrics, alternative metrics, researcher impact, journal quality and impact, book quality and impact, and university rankings. |
 |
Author Analysis
Databases such as Scopus and Web of Science can be used to:
- locate the papers of a specific author
- compare the research output of more than one author
The h-index
|
The h-index is a metric that allows you to compare the publications or research output of authors. The h-index is calculated by determining the number of articles (n) written by an author, in the database, that have received the same number or more (n) citations over time: The h-index is a useful metric for comparing rates of publication, as the value is not skewed by a single highly cited paper, nor by a large number of poorly cited papers. |
 |
Note:
- The h-index is not a static value – if discussing an author’s h-index, you need to specify the date on which the h-index was calculated.
- The h-index is also calculated by other databases/resources and may vary from the h-index given by Scopus – if discussing an author’s h-index, you need to specify the source of the h-index.
Scopus
To locate papers of an author in Scopus:
|
Go to the default Scopus search screen and select Authors tab.
|
 |
|
The author’s details and the documents that they have written, and which are indexed by Scopus, will be retrieved.
|
 |
|
The information about the author will also tell you:
|
|
Web of Science Core Collection
To locate papers of an author in Web of Science Core Collection (WoS CC):
|
Go to the default WoS CC search screen and select Researchers tab.
|
 |
|
The author’s details and the papers that they have written, and which are indexed by WoS CC, will be retrieved.
The information about the author will also tell you:
|
 |
|
The author's citation report will tell you:
|
 |
Source / Journal Analysis
Databases such as Scopus and Web of Science Core Collection (including CABI Abstracts) can be used to determine the quality of journals in a discipline or field of research.
Citation Analysis
Citation Analysis is the process whereby the impact or "quality" of an article is assessed by counting the number of times other authors mention it in their work.
Citation Analysis is the process whereby the impact or "quality" of an article is assessed by counting the number of times other authors mention it in their work.
Citation analysis involves counting the number of times an article is cited by other works to measure the impact of a publication or author.
The caveat however, is that there is no single citation analysis tool that collects all publications and their cited references.
For a thorough analysis of the impact of an author or a publication, look in multiple databases to find all possible cited references.
| A number of resources are available via the Library that help identify cited works including: Web of Science, Scopus, Google Scholar, and other databases with limited citation data. | |
|
The Measure Research Quality and Impact Research Guide provides methodology on a range of bibliometrics including citation metrics, alternative metrics, researcher impact, journal quality and impact, book quality and impact, and university rankings. |
|
To search for journals on a subject
Scopus
|
Go to the default Scopus search screen and select Sources screen.
The Results list includes journals indexed by Scopus on that subject. |
 |
|
Results can be sorted or ranked by:
|
 |
To compare journals or source publications
Scopus
|
Go to the default Scopus Search screen and select Documents tab.
|
 |
|
From the Advanced Search page, click on the link to |
 |
| A range of metrics for up to ten journals can be compared in either Chart or Table view |  |
Scopus
The Scopus Search Guide clearly describes how to search for papers on a subject or specific papers.
Web of Science Core Collection
The Web of Science Core Collection Quick Guide clearly describes how to search for papers on a subject or specific papers.
Note taking
Taking clear, legible notes will help to focus your critical reading and analysis of your literature review sources. When taking notes, avoid plagiarism by:
- keeping track of the difference between information from your sources and from your own ideas
- providing clear references, including page numbers
Note taking methods
Some effective methods of note-taking include:
- Outlining method: Use headings, sub-headings and bullet points to organize topics
- Cornell method: Use two columns - in one column write your summary of the authors' conclusions and evidence, and in the other column write down your own analysis and other comments
- Charting method: Create a list of topics or points you want to write about - use a column for each one. As you read, add references and make notes in the appropriate column
- Sentence method: Simply write down new ideas and bits of information as a numbered sentence
- Mapping method: Write down key concepts and terms, with related ideas radiating out from these
You may consider using the matrix below for your note taking and analysis:
Critical Reading & Analysis Checklist
- Does your literature review highlight flaws, gaps, or shortcomings of specific texts or groups of texts?
- Have you identified areas that have not yet been researched or have not yet been researched sufficiently?
- Does the literature demonstrate a change over time or recent developments that make your research relevant now?
- Are you able to discuss research methods used to study this topic and/or related topics?
- Can you clearly state why your research is necessary?