Disclaimer: this blog post has been created as an example of
critical appraisal and doesn’t constitute medical advice. For expert advice, please
speak to your health professional!
With Easter approaching, you may be thinking of chocolate
eggs as an Easter treat. But did you know, some studies have explored the
health benefits of chocolate eaten in small amounts regularly? In the library
team, we encourage and provide training
for critical appraisal of research studies. So, we decided to present a critical
appraisal of a study about the cognitive effects of eating chocolate – how
reliable is the research? Read on to find out…
What is critical appraisal?
Critical appraisal is evaluating information to determine
its trustworthiness and its relevance to a particular context. There are
several published checklists that provide guidance for this evaluation, such as
those produced by CASP,
JBI and CEBM.
Each checklist is usually tailored to a particular type of research study. So
first we want to identify – what type of study is the article that we’re
looking at?
The article
We’ll be looking at Sub-Chronic Consumption of
Dark Chocolate Enhances Cognitive Function and Releases Nerve Growth Factors: A
Parallel-Group Randomized Trial by Sumiyoshi et al., published in 2019.
It’s a randomised trial so, if taking the CASP checklists as an example, the closest
fit will be the checklist
for randomised controlled trials.
The article explores the cognitive effects of regularly
eating dark chocolate in the medium term, in comparison to white chocolate. If
you’re a white chocolate fan, the bad news is that white chocolate contains
fewer of the nutrients that have previously been found to have beneficial
health effects. The nutrients specifically mentioned in this article are flavonoids
– namely epicatechin and catechin - and methylxanthines - namely theobromine
and caffeine - with regard to dark chocolate.
The researchers recruited participants to be divided into two groups: one group eating a set daily amount of dark chocolate and the other
group eating a set daily amount of white chocolate, over 30 days. They were
instructed not to eat any other chocolate during this time and not to exceed
more than 3 cups of caffeinated drink per day. At the start of the trial, at
the end of the trial and 3 weeks after the trial finished, the key measurements
were taken. These included two cognitive tests: the Stroop Colour Word Test
(modified), which uses names of colours printed in different coloured ink to
test reaction to cognitive interference (disruption to usual
thinking/processing), and the Digital Cancellation Test, which involves finding
numbers amongst a sheet of digits, in order to measure attention, processing
speed and executive functioning. The researchers also measured some physical
attributes such as weight, BMI, heart rate and various chemicals in the blood.
CASP Section A: Is the basic study design valid for a randomised controlled trial (RCT)?
This study isn’t claiming to be an RCT, rather a randomised
trial. This is probably because the factors aside from chocolate consumption (like
a participant having extra cake, biscuits etc.!) weren’t absolutely controlled,
they were only given limited instructions to follow at home. The lack of
control over these other factors could allow them to influence the findings –
something to bear in mind.
On the other hand, participants were assigned to white
chocolate or dark chocolate randomly using a computer generator, which is good
practice. Randomisation is important because it minimises bias. For example, if
the researchers were to allocate participants non-randomly, they might assign
healthier-looking participants to dark chocolate for better likelihood of
outcomes.
Another aspect to mention in terms of study design is the
sample size. Twenty students were recruited for the study and two of them
dropped out. This is a very small sample which means the reliability of results
hasn’t been tested on enough people to be really trustworthy. It suggests that
the results might not reflect the wider population. Seeing as the whole sample
was made up of healthy undergraduate students (aged 20-31 years), this is even
more the case. It means we don’t know for sure if the findings could be applied
to other types of people with different ages or health conditions.
CASP Section B: Was the study methodologically sound?
Can you taste the difference between dark chocolate and
white chocolate? Probably very easily yes, because white chocolate is much
sweeter. So, in this study it wouldn’t have been possible to ‘mask’ or ‘blind’
the participants, to prevent them from knowing which chocolate intervention
they received. ‘Blinding’ or ‘masking’ is normally carried out in RCTs when
possible, because it minimises bias caused by participants knowing which type
of treatment they’ve been given. For example, if a person knows that they are
only receiving a placebo (‘dummy drug’) rather than a real drug, their mind and
body could actually respond differently than if they believed they were
receiving the real drug (known as the placebo effect).
That said, the participants weren’t told the exact objective
of the study until after the course of chocolate was complete. In a way, this
acted as an alternative form of masking, limiting the influence of participants’
knowledge on outcomes – unless of course they guessed what the experiment might
be about! The article also says that the researchers were unaware of which
participants were consuming which type of chocolate. This limits the potential
bias from researchers’ knowledge of groupings.
Next, we want to know – were the characteristics of the participants
allocated to dark chocolate similar to the participants allocated to white
chocolate, at the start of the study? If one group was particularly sporty and
fit, but the other wasn’t, that wouldn’t be a fair test! The article tells us that there were no
significant differences, so we’re assuming they distributed genders etc. evenly
between groups. From the supplementary tables we can see that the weight, BMI,
heart rate, blood composition etc. of the 2 groups didn’t vary significantly.
This all means there’s low likelihood of any of these factors affecting the
results. In that case, if the results show differences between groups, it’s
more likely down to the type of chocolate.
Similarly, we want to know if the groups were treated any
differently during the trial, besides being given different types of chocolate.
Unless the participants had some sneaky extra chocolate or coffee, we know that
they followed instructions to limit caffeine intake, refrain from other
chocolate and avoid intense exercise. Again, this would prevent other factors
influencing results. In other aspects, such as other food or drink intake while
at home, we don’t know whether this varied between groups during the trial.
CASP Section C: What are the results?
According to the scores from the cognitive function tests,
there was some improvement in outcomes for the dark chocolate group, but
results were mixed. For example, in the Stroop Colour Word Test, the dark
chocolate group showed improved performance after starting the course of
chocolate; but the white chocolate group also improved on the colour part of
the test. In the third part of the Digital Cancellation Test, the dark
chocolate group showed significant improved total performance, but not in the
first and second parts of the test (neither did the white chocolate group).
Basically, in this study, there are signs of a link between dark chocolate and
improvement in cognitive tests, but the pattern isn’t really consistent or
strong enough to be completely certain.
The measures of theobromine (a nutrient which previous
studies have linked with cognitive performance) from blood plasma show it rose
significantly in the dark chocolate group but not in the white chocolate group.
Caffeine and other chemicals didn’t show such a significant increase. So, if
dark chocolate does improve cognitive function, theobromine is probably at
least one of the reasons behind it. Another contributing element that the
authors describe is Nerve Growth Factor, which was also elevated in the dark
chocolate group compared to the white chocolate group.
The researchers calculated the statistical significance of
their results, to show whether the findings were likely to be due to chance
(p-values). Overall, some of the measurements couldn’t be said to be
statistically significant, but others were, including the increased number of
correct answers for the Stroop test in the dark chocolate group.
CASP Section D: Will the results help locally?
So, overall, what should we make of this study? If you are a
healthy student (or young adult), aged 20-31, eating around 24g of dark
chocolate with at least 70% cocoa every day (cocoa is the key ingredient, so
higher percentages may make a difference!), it’s quite likely that your
cognitive functioning would be higher than a similar person eating white
chocolate (0% cocoa) daily.
However, it’s not so certain whether this applies to the same extent if you are of a different age etc. or if you consume a significantly lower or higher amount of the chocolate per day. There could be some anomalies in cognitive performance even if you mirrored the participants in the study. Meanwhile, you might not be particularly bothered about cognitive performance. You might be more interested in other health factors related to chocolate consumption, such as cardiovascular function, diabetes etc. which aren’t covered in this study. Some other studies have looked at these sort of aspects, both positive and negative (e.g. Morze et al. (2020), Amoah et al. (2022) and Yuan et al. (2017), to name just a few). It really is a case of weighing up the strengths and weaknesses of this study and other studies then coming to a critical judgement depending on your circumstances (and possibly professional advice - health professionals rather than confectionery professionals, that is!).
Whatever your own
conclusions, we hope you can enjoy the spring/Easter season!
(No chocolate was harmed in the making of this blog post…probably.)
We hope you found this article interesting. If you’d like
to find further evidence about the health effects of chocolate, you could search on the NHS Knowledge and Library Hub using keywords such as
Your library team are always happy to support with locating articles.
