Friday, 7 August 2020

Monday, 3 August 2020

Effectiveness of executive function training on mental set shifting: one study reported across two papers or two trials with the same registration number?

Dear Editor
I write to highlight potential issues with the reporting of the randomised controlled trial by Chiu et al., (2018) that was published in the Journal of Advanced Nursing.

Our group have undertaken a series of reviews that have determined nursing trials compliance with ICMJE (De Angelis et al., 2004) requirement for prospective registration (Gray et al., 2017, 2019). Consistently, nursing has done poorly and has one of the lowest rates of trial registration of any clinical discipline (Gray, Brown, et al., 2019; Gray et al., 2017; Gray, Gray, et al., 2019). For example, In a review of the 151 nursing science trials published in 2017, just 17 (11%) were prospectively registered (Gray et al., 2019). The Journal of Advanced Nursing editorial team has taken the issue of prospective registration seriously and we are aware that they have put in place policies and procedures to ensure that the trials they publish are appropriately registered.

One of the practices that trial registration is intended to prevent is selective outcome reporting, where authors only publish a subset of the outcomes measured in order to inflate the magnitude of the treatment effect (Chan et al., 2004). In our previous reviews, we have not reconciled registration entries with published manuscripts to determine if there was any evidence of selective outcome reporting; this has been an acknowledged limitation of our methodology (Gray et al., 2019). We are in the process of completing our second yearly review of trial registration in nursing science (studies published in 2018) and have extended our methodology to address this deficit. Our review has identified a number of discrepancies between what authors have stated in the trial registry entry and reported in the published manuscript. We feel it is incumbent upon us to alert Journal editors about the discrepancies we have observed.

One of the trials included in our review was published in the Journal of Advanced Nursing and authored by Chiu et al., (2018). Several issues relate to this (Chiu et al., 2018) and another – seemingly related – trial by the same group (Kao et al., 2018), published in the International Journal of Nursing Studies (IJNS) and also included in our review. We have written to the IJNS separately about this paper.

Chiu et al., (2018) report a randomised controlled trial of 24 sessions of Executive Function Training on mental set shifting (determined using the Wisconsin Card Sorting Test) in healthy adults aged 65 years or older. The authors concluded that executive function training might be an effective preventative strategy in older adults. When we reconciled the manuscript with the registry entry (Chi-CTR-IOR-14005490), we noted that the authors were vague in their description of the intervention under investigation (described as cognitive training) and primary outcome (cognitive function). In our view, this apparent inconsistency warranted further investigation. We used the trial registration number to search (using Google) for any related publications (such as a trial protocol). A second paper was identified using the same trial registration number and published in BMC Geriatrics (Chan et al., 2019). In this paper, Chan et al., (2019) report the effect of 24 sessions of Interactive Cognitive-Motor Training on eye-hand coordination in older adults. According to the authors, there were no differences between the groups against the primary outcome at the end of the trial.  Our initial thought was that the authors were reporting a single trial across multiple papers (so-called salami-slicing) (Gray & Baker, 2016). This does not seem to be the case: the interventions described in the two papers seem entirely different. In the JAN paper, the authors state that that the Executive Function Training Group “…underwent process-based training using the Chinese version of the Rehacom computer training software (Hasomed GmbH, Magdeburg, Germany). Rehacom is a set of cognitive training software modules that have been proven effective by empirical research for improving the cognitive functions of older adults, including healthy older adults and patients with head injury, stroke and Alzheimer disease…” [sic] (Chiu et al., 2018). The interactive cognitive-motor training (ICMT) intervention in the BMC Geriatrics paper is described as follows “ICMT was performed on the Hot Plus interactive health service system. When using the system, the participants received information visually or acoustically and performed suitable physical responses based on their judgment. Through feedback, coordination between the brain and body can be improved to adjust responses” [sic] (Chan et al., 2019). Outcome measures in the two papers are also different. Mental set-shifting, working memory and inhibition were reported in the JAN paper (Chiu et al., 2018), and visual-motor integration, visual perception, motor coordination and cognitive function in the BMC Geriatrics manuscript (Chan et al., 2019). It seems that, apart from the trial registration number, the papers are reporting different studies. One possible explanation might therefore be that the authors reported the wrong registration number in one of the manuscripts. However, we checked the ethics committee reference number (201312037) which is the same in both papers. And both studies involved 62 participants. The demographic characteristics of the control groups in both papers are also essentially identical; mean age, MMSE score, gender, educational level is precisely the same (to two decimal places). However, the demographic profile of subjects in the experimental group subtly differs between the two studies.

The confusion deepens when a third – seemingly related paper – was reported in the IJNS is considered (Kao et al., 2018). The manuscript also reports a trial of interactive cognitive-motor training in 62 older adults. The primary outcome in this trial was gait performance. Although the Kao et al., (2018) trial has a different trial registration number (ChiCTR-IOR-17013812) to that reported in the Chan et al., (2019) paper (Chi-CTR-IOR-14005490), the demographic profile of participants in both papers match. We have written to the editors of the IJNS about our specific concerns with the paper they published.

In summary, we are unclear if the authors have reported a single trial of the same intervention that is reported across multiple papers. If this is the case the authors need to explain why the interventions are apparently different. Alternatively, the authors may have reported two separate trials and inadvertently used the same registration number. If this is the case, the authors need to explain why the demographic characteristics of control participants are identical in both studies. There may be a justification for the observed discrepancies that we have not considered.  We look forward to a response from the study authors.

Professor Richard Gray PhD
Research Focus Area Director
La Trobe University
Melbourne, VIC 3086

Monday, 13 July 2020

The COVID‐19 epidemic of manuscripts

Roger Watson, Editor-in-Chief
Mark Hayter, Editor

Listen to this as a podcast here.

Monday, 8 June 2020

Nurse staffing levels and workload do matter

Roger Watson, Editor-in-Chief

It seems self-evident that the fewer nurses we have relative to patients and the more that nurses have to to, the more patients must be at risk. And that is exactly what this study, recently published in JAN shows.

The study was conducted in Finland by Jansson et al. (2020) and they published an article titled: The proportion of understaffing and increased nursing workload are associated with multiple organ failure: A cross‐sectional study The study was conducted in one hospital over 10 years. A range of measures of nurse interventions and the nurse to patient ratios were recorded daily as was sequential organ failure. Over 10,000 patient incidents were studied.

Nurses intervention was higher in patients with multiple organ failure and in those who died. According to the authors: '(t)he proportion of understaffing was significantly more common in patients with multiple organ failure than in those without' and '(t)he levels of nursing associated with workload and understaffing were at their worst on weekends.'

The authors concluded: 'The proportion of understaffing and increased nursing workload are associated with multiple organ failure, demonstrating that an adequate level of nurse staffing in relation to patient complexity is a prerequisite for the availability and quality of critical care services.'  Finally, it is worth noting that '(t)he proportion of understaffing did not differ between survivors and non‐survivors.'

You can listen to this as a podcast


Jansson, M, Ohtonen, P, Syrjälä, H, Ala‐Kokko, T. The proportion of understaffing and increased nursing workload are associated with multiple organ failure: A cross‐sectional study. J Adv Nurs. 2020;