Skip to main content

Research Repository

Advanced Search

Inter-rater reliability of paediatric emergency assessment : physiological and clinical features

Heal, C; Cotterill, S; Rowland, AG; Garratt, N; Long, T; Brown, S; O'Connor, G; Rishton, C; Woby, S; Roland, D


C Heal

S Cotterill

AG Rowland

N Garratt

S Brown

G O'Connor

C Rishton

S Woby

D Roland


The Paediatric Admission Guidance in the Emergency Department (PAGE) score is an assessment tool currently in development that helps predict hospital admission using components including patient characteristics, vital signs (heart rate, temperature, respiratory rate, oxygen saturation) and clinical features (e.g. breathing, behaviour, nurse judgement). It aims to assist in safe admission and discharge decision making in environments such as emergency departments and urgent care centres. Determining the inter-rater reliability of scoring tools such as PAGE can be difficult. The aim of this study was to determine the inter-rater reliability of seven clinical components of the PAGE Score.
Inter-rater reliability was measured by each patient having their clinical components recorded by two separate raters in succession. The first rater was the assessing nurse, the second rater was a research nurse.
Two emergency departments and one urgent care centre in the North West of England. Measurements were recorded over one week; data was collected for half a day at each of the three sites.
A convenience sample of 90 paediatric attendees (aged 0 to 16), 30 from each of the three sites.
Main Outcome Measures
Two independent measures for each child were compared using kappa or prevalence-adjusted bias-adjusted kappa (PABAK). Bland-Altman plots were also constructed for continuous measurements.
Inter-rater reliability ranged from moderate (0.62 [95% CI 0.48 to 0.74] weighted kappa) to very good (0.98 [95% CI 95 to 0.99] weighted kappa) for all measurements except ‘nurse judgement’ for which agreement was fair (0.30 95% CI [0.09 to 0.50] PABAK). Complete information from both raters on all the clinical components of the PAGE score were available for 73 children (81%). These total scores showed good’ inter-rater reliability (0.64 [95% CI 0.53 to 0.74] weighted kappa).
Our findings suggest different nurses would demonstrate good inter-rater reliability when collecting acute assessments needed for the PAGE score, reinforcing the applicability of the tool. The importance of determining reliability in scoring systems is highlighted and a suitable methodology presented.


Heal, C., Cotterill, S., Rowland, A., Garratt, N., Long, T., Brown, S., …Roland, D. (2021). Inter-rater reliability of paediatric emergency assessment : physiological and clinical features. Archives of Disease in Childhood, 106(2), 149-153.

Journal Article Type Article
Acceptance Date Jul 15, 2020
Online Publication Date Sep 28, 2020
Publication Date Jan 19, 2021
Deposit Date Jul 16, 2020
Publicly Available Date Sep 30, 2020
Journal Archives of Disease in Childhood
Print ISSN 0003-9888
Electronic ISSN 1468-2044
Publisher BMJ Publishing Group
Volume 106
Issue 2
Pages 149-153
Publisher URL
Related Public URLs