Inter-rater reliability of paediatric emergency assessment : physiological and clinical features

Heal, C; Cotterill, S; Rowland, AG; Garratt, N; Long, T; Brown, S; O'Connor, G; Rishton, C; Woby, S; Roland, D

Abstract

Objective
The Paediatric Admission Guidance in the Emergency Department (PAGE) score is an assessment tool currently in development that helps predict hospital admission using components including patient characteristics, vital signs (heart rate, temperature, respiratory rate, oxygen saturation) and clinical features (e.g. breathing, behaviour, nurse judgement). It aims to assist in safe admission and discharge decision making in environments such as emergency departments and urgent care centres. Determining the inter-rater reliability of scoring tools such as PAGE can be difficult. The aim of this study was to determine the inter-rater reliability of seven clinical components of the PAGE Score.
Design
Inter-rater reliability was measured by each patient having their clinical components recorded by two separate raters in succession. The first rater was the assessing nurse, the second rater was a research nurse.
Setting
Two emergency departments and one urgent care centre in the North West of England. Measurements were recorded over one week; data was collected for half a day at each of the three sites.
Patients
A convenience sample of 90 paediatric attendees (aged 0 to 16), 30 from each of the three sites.
Main Outcome Measures
Two independent measures for each child were compared using kappa or prevalence-adjusted bias-adjusted kappa (PABAK). Bland-Altman plots were also constructed for continuous measurements.
Results
Inter-rater reliability ranged from moderate (0.62 [95% CI 0.48 to 0.74] weighted kappa) to very good (0.98 [95% CI 95 to 0.99] weighted kappa) for all measurements except ‘nurse judgement’ for which agreement was fair (0.30 95% CI [0.09 to 0.50] PABAK). Complete information from both raters on all the clinical components of the PAGE score were available for 73 children (81%). These total scores showed good’ inter-rater reliability (0.64 [95% CI 0.53 to 0.74] weighted kappa).
Conclusions
Our findings suggest different nurses would demonstrate good inter-rater reliability when collecting acute assessments needed for the PAGE score, reinforcing the applicability of the tool. The importance of determining reliability in scoring systems is highlighted and a suitable methodology presented.

Citation

Heal, C., Cotterill, S., Rowland, A., Garratt, N., Long, T., Brown, S., …Roland, D. (2021). Inter-rater reliability of paediatric emergency assessment : physiological and clinical features. Archives of Disease in Childhood, 106(2), 149-153. https://doi.org/10.1136/archdischild-2019-318664

• Journal Article Type: Article
• Acceptance Date: Jul 15, 2020
• Online Publication Date: Sep 28, 2020
• Publication Date: Jan 19, 2021
• Deposit Date: Jul 16, 2020
• Publicly Available Date: Sep 30, 2020
• Journal: Archives of Disease in Childhood
• Print ISSN: 0003-9888
• Electronic ISSN: 1468-2044
• Publisher: BMJ Publishing Group
• Volume: 106
• Issue: 2
• Pages: 149-153
• DOI: https://doi.org/10.1136/archdischild-2019-318664
• Publisher URL: http://dx.doi.org/10.1136/archdischild-2019-318664
• Related Public URLs: http://adc.bmj.com/