As veterinarians and animal health researchers, we regularly engage with new evidence and advances in the field to keep our clinical practice and research approaches up-to-date and effective. One of the common ways we do this is by reading peer-reviewed journal articles about new research studies and then reflecting on if and how we can apply the findings to our current practices.
Being able to critically evaluate a journal article is considered an essential Day-One level skill for graduate veterinarians as well as researchers.
The outputs from epidemiological research studies are what generally allow us to practice evidence-based medicine or, in other words, making clinical decisions on the basis of sound scientific evidence rather than just guessing or just continuing to do what we have always done. There are certain pieces of information that help us out at various stages of the consultation process to diagnose and treat disease. Many of the research questions and studies we tackle are geared towards collecting this type of information.
When you see a patient present for a consult, you will usually get some information on their signalment (species, breed, age, sex, desexing status), geographic location (most often from the same area as your clinic is located), and their presenting complaint (which could be something like cough, limping, vomiting, etc). There are many different diseases that can cause animals to have those clinical signs, but remember that common diseases happen commonly and so it is helpful for us to know what are typically the top differential diagnoses for each clinical presentation.
Remember that prevalence will tell us how likely it is that an animal currently has the disease while incidence will tell us how likely it is that a currently unaffected animal will get the disease. Both pieces of information are useful in discussions around preventative care because they can help owners understand the risks of their animal getting the disease. From a more population-based perspective, it is helpful for us to know what diseases are most common so that we can better allocate time and resources towards fixing them.
We are also often interested in understanding what factors are associated with an increased or decreased risk of an animal having a particular disease or outcome. This can further help us narrow down our differential diagnosis list when we ask more targeted questions in the medical history and it can also help us better predict how particular patients might respond to the disease or its interventions.
For example, we know that unvaccinated puppies who have been in contact with other dogs at the park or other locations outside the immediate household are more likely to have parvovirus and so when a puppy presents with diarrhoea and/or vomiting, we will ask the owners about the vaccination status and any places the puppy might have been in the last couple of weeks.
As we also know from other research, puppies with parvovirus that already present with fever and low white blood cell counts have a much worse prognosis and we can use that information to help counsel owners about the likely costs and outcomes from treatment.
It is also important for us to know what happens to an animal once it gets sick. Particularly for infectious disease epidemiology, we need to know how long the incubation periods are from the time an animal is exposed to the disease until it becomes infectious or starts showing clinical signs. We also need to know how long it spends being infectious and what typically happens as an outcome whether it recovering with long-term immunity, recovery with waning immunity, chronic infections, or death.
As we have saw with the COVID outbreak, one of the big challenges with controlling the disease is that it can take upwards of 14 days from the time someone was exposed to the virus until they are shedding enough virus to test positive (hence the reason for the 14 day quarantine period at the border and making decisions about lockdown levels in 14 day time windows) and many people who are infectious with COVID has no obvious clinical signs of being sick (hence the reason for lockdowns to prevent asymptomatic cases from unknowingly spreading the disease).
We also use clinical impact studies a lot in animal health economics because knowing how much the disease will affect things like growth, milk production, reproductive performance, mortality, and carcass quality helps us to estimate the total cost of disease to farmers and therefore justify why they should invest in disease control measures.
There are many different diagnostic tests that we can use to help us rule in or rule out different differential diagnoses and it’s important for us to have a good understanding of how well they perform so that we can better interpret the results. Diagnostic tests can include things like clinical signs, physical examination findings, clinical pathology, and imaging. We go into more detail around the performance of diagnostic tests in the Evidence-Based Medicine e-book.
For most diagnostic test studies, researchers apply the diagnostic test to one group of animals that are known (or at least strongly suspected) of having the disease and one group of animals that are known (or at least strongly suspected) of not having the disease. They then count the number of animals in each group that tested positive or negative to be able to generate the 2×2 table and estimate the Se, Sp, PPV, and NPV. Remember that we also need prevalence information to help us work out PPV and NPV.
We are also frequently faced with the challenge of deciding which treatment options to recommend for the disease, which could include different medications, vaccinations, and medical or surgical procedures. With intervention studies, we typically try to compare the outcomes from a treated group against a control group that is not receiving any intervention. However, in many cases, it is unethical to leave diseased animals untreated and so we may instead compare our new intervention against a group of animals that is given another acceptable treatment for the disease.
Another type of infectious disease study that you will commonly come across in the literature is simulation modelling studies. Briefly, these studies are based on building computer simulation models that replicate how disease spreads between individuals in a population, which lets us estimate how big a disease outbreak might be and how effective different potential control options could be in reducing disease transmission. I am probably dating myself here, but think “SimCity” for infectious diseases. These type of models have been used extensively to inform policy decisions around controlling Mycoplasma bovis and COVID-19 in New Zealand and integrate information from all of the research study types mentioned above.
Most journals will publish three main types of research studies provided that they fit with the journal aims and make a novel contribution to our understanding of a particular research subject.
A paper reporting new research conducted by the authors. This is often either an original research article that presents a complete study, describing its aims, methods, results, and significance or case study or report describing a single clinical case or a small number of clinical cases, often used to explore uncommon conditions or outcomes in depth.
Research that synthesises or evaluates findings from existing existing studies rather than generating new data. This commonly takes the form of systematic reviews, which answer focused research questions using predefined methods, or scoping reviews, which map the breadth and nature of evidence across a broader topic.
A paper expressing expert viewpoints or interpretations, such as editorials or perspectives, rather than presenting original research data.
When researchers conduct a study, one of the main ways they have of communicating their results with the wider world is to write an article about it for publication in a peer-reviewed scientific journal. The terms journal article and manuscript are used interchangeably to describe a scientific publication. For those of you planning to complete a clinical residency, you will usually be required to produce at least peer-reviewed journal article to demonstrate your ability to advance knowledge in the field. For those of you planning an academic career, research publications are the most widely used currency to demonstrate your expertise, productivity, and capability as a research scientist.
There are lots of different journals in the field of veterinary science that researchers can choose to submit their manuscript to with each journal typically focusing on a specific species (i.e. Journal of Dairy Science), practice area (i.e. Journal of Small Animal Practice), specialty field (i.e. Veterinary Anaesthesia), or geographic region (i.e. New Zealand Veterinary Journal). This website from the Ohio State University has key veterinary journals listed by subject area: (https://guides.osu.edu/vetmed/corejournals)
Each journal has a different impact factor, which is calculated as the average number of times that articles published in the journal over the past two years have been cited by other published journal articles and serves as a measure of how prestigious the journal is considered to be in its field.Some journals are open-access, which means that the full-text of all articles published in the journal are freely available to the general public. Other journals will make the article abstracts publicly available so that people can get a general idea of what the article is about, but then require readers to pay a fee to access the full text of the article. Most journals will charge authors around $4,500 to make their journal article open-access and some journals will have regular article processing charges as well.
Generally, we try to publish our articles in the highest-ranking journal we can afford that fits with the scope and objectives of our research studies.
When you submit an article to a journal, it first gets assigned an Editor who is usually an experienced scientist in the field. They will contact two or three other experts to serve as peer-reviewers for the article. It’s the reviewers’ job to critically evaluate the work and identify any potential problems with the way the study was designed or presented in the manuscript that would make it unsuitable for publication. The review process can be single-blind where the reviewers know the identity of the authors but the authors don’t know the name of the reviewers, double-blind where neither the authors nor the reviewers are aware of each other’s identities, or open review where the reviewers know who wrote the paper and the authors know who reviewed the paper.
The reviewers usually have 2 to 4 weeks to provide their feedback and comments on the manuscript. They will also make a recommendation about whether the manuscript should be accepted, rejected, or revised and re-submitted by the authors. The Editor will then forward the decision to the authors. If revisions are required, the authors usually have 1 to 3 months to revise and re-submit. The comments could be requests for further clarifications about how the study was designed or how the results were presented in the manuscript. The reviewers may also ask the authors to perform additional statistical analyses on the data. This process may continue for several back-and-forth rounds until the manuscript is deemed suitable for publication. Some journals also have slower turnaround times for reviews than others and sometimes it can take >18 months from the initial submission until the research gets published!
The main purpose of the review process is to serve as quality control mechanism for anything that gets published in the scientific literature, but it’s not a completely perfect system. Firstly, it relies on the authors having done a robust job of collecting, recording, and analysing the data. Mistakes can sometimes happen accidentally and most academics operate with integrity, but there have also been occasional instances where the authors have later been found to have “massaged” the study data to get outputs that are more publishable. Secondly, sometimes the reviewers are also not quite the best and may let some less than stellar work slide through. That’s why it’s really important for you to have a critical eye on the work whenever you are reading a journal article.
Another big limitation with the scientific literature is tends to be biased towards studies that found significant differences rather than those that found no effect or no relationship between variables. This means that the scientific literature is probably missing a lot of valuable information on things that didn’t work so well, which is problematic because future researchers could end up repeating the same mistakes again.
My favourite place to find journal articles is scholar.google.com. Similar to regular Google, you just type in keywords for the topic you are looking for and it will pull up hundreds of potential articles for you to look through. For CPD purposes, some folks will browse through specific journals every month or two to see what new articles of interest have come out.
Most universities pay a subscription fee to a large number of journals so that students and staff have unlimited free access to most published journal articles. It is a little bit more difficult for veterinarians out in practice to access article texts, but you can always e-mail the researchers directly to ask for a copy of the article. The contact details for the corresponding author are usually listed with the article or you can find the author online by searching for their name and institution.
Knowing how journal articles are typically structured can help you become a lot more efficient at reading through all those pages of text to identify and critically evaluate the key take-home messages. The good news is that most journal articles follow a very prescriptive format for how the information is presented.
The Abstract is a short summary of the journal article (usually 150 to 400 words) that is designed to provide readers with enough information to decide if the results are interesting or relevant enough to warrant reading the full text version. It usually contains information on the scientific rationale, methods, results, interpretation, and clinical relevance. The abstracts for journal articles are almost always freely available in the public domain.
The Introduction section provides a general background on the research study to help the reader better understand why there was a need to conduct the research study.
Be warned – sometimes the stated objectives don’t always match what the study did or found so you might have to read further before getting a good idea of what the research has contributed to the literature.
The Materials and Methods section is a descriptive narrative about how the researchers went about collecting and analysing the data to achieve their study objectives. There should be enough detail provided that another researcher could replicate the study or the statistical analyses if given the appropriate resources.
Somewhere in the methods section, there should also be a statement about any human or animal ethics approvals that were obtained prior to conducting the study. This is an important step in the research process to ensure that researchers have considered all the different ways that their study study may cause physical or psychological stress to participants and taken appropriate measures to reduce or mitigate those risks as much as possible. Ethics application must be submitted prior to conducting the research study.
The Results section presents an objective summary of the key research findings from the analyses. It usually starts with a descriptive summary of the demographic characteristics of the study participants (i.e. age, breed, sex, geographic location) followed by tables and figures depicting the key take-home points.
The Discussion section is where the authors critically review their findings in the context of other published research studies as well as the limitations of their study. Ideally, they should have paragraph for each key finding that explains:
The authors should also address broader issues around how well the study design answered the research question for the study sample (internal validity) and how well the results can be applied to other populations (external validity). It’s very rare to have a “perfect” research study it wouldn’t be called re-search if we always found the answer we were looking for on the first try. There will almost always be some issues that can affect the interpretation of the results and should be addressed in future research. This is very valuable information so that we don’t make the same mistakes again.
The Conclusions section is most often a brief paragraph summarising the clinical significance of the research study. This is a good place to navigate to if you want a quick overview of how the research findings could be applied in clinical practice.
The Reference list includes all of the peer-reviewed and non-peer-reviewed resources that the authors cited in their article to put their study background, methods, and findings in context of what is already known about the subject. As a reader, this section is particularly useful for identifying other relevant journal articles that you might want to dig out for further information on the subject. When I am peer-reviewing a journal article, I will usually check to make sure that the authors are citing recent literature (i.e. articles published within the last 5 years) as an indication that they have been staying up-to-date with advances in the field.
Reviewing a research article involves more than reading the results. This section introduces a three-step approach to interpreting evidence: assessing internal validity to determine whether the findings are reliable for the population studied, considering external validity to judge whether results may apply to other populations or settings, and using systematic reviews to understand how individual findings fit within the broader body of evidence.
The concepts of study design, bias, confounding, and error come together when assessing internal validity. This step focuses on whether a study was conducted in a way that supports reliable conclusions for the population under investigation. Evaluating internal validity involves determining whether key methodological details are clearly reported, identifying potential problems in how the study was carried out, and judging the likely impact of those issues on the study’s findings.
The following outlines key questions to ask yourself about the study as you are reading through the journal article:
Having established that the study produced valid findings for the population under study, the next question we must consider is if it is reasonable to generalise the result of the study to another population. When considering this issue, many people make the mistake of saying that as the study population differs (such as on with a different distribution of age, breed or sex) from the population they wish to extrapolate to or because the study population was not representative of the population, then we cannot generalize.
However, when deciding about generalisability, we need to ask if there is a reason to believe that the relationship between the exposure and disease would be different in a different population. When thinking of a biological agent, then the results are likely to be transferable. However, if the study was examining the effect of an educational package, then it may not be as straightforward to generalize the results.
The other issue that must be considered is if the underlying risk of the outcome is different in the study population to your own. For example, it is common for trials to be conducted in populations with a relatively high risk of developing the diseases that the treatment aims to prevent. This will not alter the relative risk measure, BUT it does mean that in a different population the number we would need to treat to avoid the poor outcome may be higher than in the study population. This means that the cost of the treatment may actually outweigh the benefits in your population, which means the treatment is not economic.
When evaluating the literature for evidence-based medicine or policy, we need to make use of all the available information from studies that have been published on the topic. Once all the information has been compiled, you can be sure that there will be conflicting and inconclusive findings. When we go through the process of compiling all the evidence, we are conducting a systematic review of the literature to ensure that we have gathered all the relevant literature upon which to base a decision. Some general guidelines for conducting a systematic review are:
Having systematically reviewed the data, we now need to summarize the results. In some cases, this can be as simple as tabulating the key findings. However, when there are a number of results, they can combine into a summary measure and the variability between results explored using a technique known as meta-analysis.
Advancing Animal Care through Shared Learning