Defining significant childhood illness and injury in the Emergency Department: a consensus of UK and Ireland expert opinion

Background Clarifying whether paediatric early warning scores (PEWS) accurately predict significant illness is a research priority for UK and Ireland paediatric emergency medicine (EM). However, a standardised list of significant conditions to benchmark these scores does not exist. Objectives To establish standardised significant illness endpoints for use in determining the performance accuracy of PEWS and safety systems in emergency departments (ED), using a consensus of expert opinion in the UK and Ireland. Design Between July 2017 and February 2018, three online Delphi rounds established a consensus on ‘significant’ clinical conditions, derived from a list of common childhood illness/injury ED presentations. Conditions warranting acute hospital admission in the opinion of the respondent were defined as ‘significant’, using a 5-point Likert scale. The consensus was a priori ≥80% (positive or negative). 258 clinical conditions were tested. Participants and settings Eligible participants were consultants in acute or EM paediatrics, or adult EM, accessed via 53 PERUKI (Paediatric Emergency Research in the UK and Ireland)’s research collaborative sites, and 27 GAPRUKI (General and Adolescent Paediatric Research in the UK and Ireland)’s sites, 17 of which overlap with PERUKI. Main outcome measures To create a list of conditions regarded as ‘significant’with ≥80% expert consensus. Results 43 (68%) of 63 PERUKI and GAPRUKI sites responded; 295 experts were invited to participate. Participants in rounds 1, 2 and 3 were 223 (76%), 177 (60%) and 148 (50%), respectively; 154 conditions reached positive consensus as ‘significant’; 1 condition reached a negative consensus (uncomplicated Henoch-Schönlein purpura); and 37 conditions achieved non-consensus. Conclusions A list of significant childhood conditions has been created using UK and Irish expert consensus, for research purposes, for the first time. This will be used as the benchmark endpoint list for future research into PEWS/safety systems performance in EDs.


INTRODUCTION
The need to verify whether scoring systems are able to accurately predict severe illness or injury in the emergency department (ED) has been highlighted as one of the top research priorities for paediatric emergency medicine (PEM) in the UK and Ireland.For this reason, there is a need to define significant childhood illness and injury in the ED to facilitate quality research in this area.

BACKGROUND
Early recognition of potential severe illness in children and young people is the focus of clinical care, research, policy-making and public interest.Since the 2008 UK Confidential Enquiry into Maternal And Child Health (CEMACH) recommended the introduction of paediatric early warning scores (PEWS) for inpatient paediatric wards to reduce morbidity and mortality, numerous PEWS have been introduced.However, no score has yet demonstrated an impact on reducing mortality for hospitalised children and their effectiveness in detecting significant illness in the ED lacks firm evidence of efficacy.For ED purposes, most scores have been retrospectively modelled, the endpoints chosen for validation purposes are variable and poorly defined, with the studies showing varying accuracy between different scoring system in identifying the sick child or the child who requires hospital admission.In one such study, the authors compiled their own list of significant illness definitions, which they then used as a benchmark to assess PEWS performance.It is unknown however whether their list is reflective of a broader group of expert opinion and there is currently no agreed or standardised list of significant paediatric conditions, illnesses or injuries in existence, against which the efficacy of PEWS and the systems around the use of such scores can be measured.

STUDY OBJECTIVES
To facilitate future PEWS research in the ED, this study set out to create a benchmark list defining the significant acute paediatric conditions that warrant an acute hospital admission from the ED, using a consensus of the UK and Ireland expert opinion.This list could then be used as a robust endpoint dataset against which to assess the performance of PEWS and other safety scoring systems.

Participant recruitment
An online, three-round Delphi survey of PEM, general paediatric and emergency medicine (EM) consultants in the UK and Ireland was conducted using the web-based Smart Survey ( smartsurvey.co.uk) tool.The Delphi period was between July 2017 and February 2018.Each round ran for 4 weeks, separated by 4-6 weeks to allow analysis and interpretation of responses.The study was led by Paediatric Emergency Research in the UK and Ireland (PERUKI) in association with General and Adolescent Paediatric Research in the UK and Ireland (GAPRUKI).PERUKI is a collaborative PEM research network whose membership at the time consisted of 53 EDs, with a mix of secondary and tertiary sites in urban and rural locations.GAPRUKI had 27 sites, 17 of which overlapped with PERUKI.Consultants from these two networks were chosen as participants as they form the most senior level of expertise and clinical decision-making.Participants were identified as follows.
PERUKI: round 1 potential participants were identified by the study site lead at participating PERUKI sites; this number of potential participants was shared with the study team to determine the overall denominator for response rates.The number of suitable consultants varied at each PERUKI site from 1 to 12. Site leads were kept updated on progress, and reminders were sent 2 weeks and 1 week before closing.In subsequent rounds, respondents to the preceding round were invited directly by the study team.Site leads were again kept updated throughout these rounds, with email reminders sent at the same intervals.
GAPRUKI: in round 1, the GAPRUKI chair contacted all consultant members.In subsequent rounds, all those who responded to the preceding round were invited through the same email invitation as PERUKI participants.The GAPRUKI chair sent reminders for round 1, and the PERUKI study lead sent reminders for rounds 2 and 3.

Delphi methodology
Delphi surveys involve a series of sequential rounds interspersed by controlled feedback to gain the most reliable consensus from a group of experts.We adopted a modified Delphi process whereby after each round, the statements that achieved consensus were eliminated, and statements that did not achieve consensus carried through to the next round of questioning.This approach was taken to maximise participation in the study through minimising responder fatigue and is an established mode of conducting a consensus-based Delphi.

Definitions
For the purposes of the study, the term 'acute hospital admission' was used as the focus for defining the significance of the illness or injury, since the decision to admit to hospital reflects a key judgement an ED clinician must make on the balance of risk.The qualifying term 'suspected' was included in some statements for certain conditions as, in the ED environment, clinicians are often presented with undifferentiated illness with the final diagnosis being made at a later stage.Participants were provided with relevant definitions in each round.
'Acute hospital admission' was defined as an acute admission to the hospital including short inpatient stays of under 24 hours (eg, to Short Stay Units or Children's Assessment Units) or the transfer and admission to other offsite inpatient units.It did not include referral to outpatients, ED or other review clinics, or planned future elective admission.'Suspected' illness was defined from the point of view of the respondent being the assessing clinician and covered those clinical situations where sufficient concerns existed to admit for ongoing treatment, diagnostics, or close observation.In some of the clinical situations being evaluated, this would have included those in whom the diagnosis had been confirmed.This definition was dependent on the individual sites' available facilities and access to diagnostic tests, which was expected to vary depending on the respondent's clinical setting.

Survey construction
Previous work was called on to act as a template to classify diagnoses into illness categories (eg, respiratory) and then we as a study group created the list of diagnoses we thought were significant and covered the majority of ED presentations.Round 1 consisted of 161 statements on clinical conditions from the following 17 illness and injury categories: infection, respiratory, cardiac, gastroenterology, neurology, trauma, surgery, allergy, dermatology, endocrine and metabolic, toxicology, musculoskeletal, haematology, renal, safeguarding, mental health and miscellaneous.
For each condition, respondents were asked whether they agreed that the condition was significant enough to warrant acute admission to hospital, with admission being used as a proxy for significant illness or injury.A 5-point Likert scale was used for answers: strongly disagree (1), disagree (2), neutral (3), agree (4) and strongly agree (5).We aimed to use up to three survey rounds, as per accepted Delphi practice.
In round 1, respondents had the opportunity to suggest additional conditions and scenarios they thought would warrant hospital admission from the ED.These suggestions were then tested in round 2. After each round, statements that did not achieve consensus were either repeated or modified in subsequent rounds, according to the following: ► Original round 1 statements reincluded if the phrasing of the statement was deemed to have sufficient clarity by the study team and no feedback to the contrary given by respondents.► Where subgroups of severity/risk were highlighted, original round 1 statements were included with further relevant statements.► Where round 1 feedback highlighted that the original statement was ambiguous and needed more clarity, the statement was rephrased and included.
► Pooling of some statements together into one where appropriate (to reduce statement numbers and so minimise responder fatigue).Round 3 only included statements suggested by round 1 respondents which did not achieve consensus in round 2. Statements that had been carried through from round 1 to round 2 were not included again in round 3 as they had already been through the process of expert opinion retesting.All statements not achieving consensus after a single round of testing were, therefore, given an opportunity for the second round of retesting.
The response 'I do not look after children with this condition' was introduced in rounds 2 and 3, following feedback from respondent's neutral answers in round 1.This intervention sought to reduce the number of neutral responses and, therefore, increase the yield of a positive or negative agreement.When this response was given, it was excluded from the percentage consensus calculation for that condition.A sample of the survey questions is available to view as the online supplementary material.

Analysis
The study team set group consensus as a priori 80% agreement either side of the Likert scale, that is, 80% total of strongly disagree and disagree (negative consensus), or 80% total of strongly agree and agree (positive consensus).Accepted practices of Delphi consensus parameters often quote a threshold of 70% agreement, though this is not a rule.Since a proxy outcome for significant illness (admission) was being used, a higher threshold was chosen to ensure that the level of consensus was more robust.For statements on which consensus was reached, median and IQRs were calculated from the 5-point Likert scale results.For statements which did not reach consensus, medians and IQRs were used to demonstrate the spread of opinion in the responses.The analysis was conducted using MedCalc Statistical Software V.17.9.2, Belgium.Round 2 consisted of 83 statements of which 23 were new statements suggested by round 1 respondents, and 60 were statements originating from round 1 which did not reach consensus; some of the original statements were made into separate statements according to the methodology described above.An exception to round 2 statement inclusion was made for Pneumothorax: tension or non-tension.Though this reached our threshold for consensus, respondent feedback strongly suggested this should be separated into three different statements: (i) tension pneumothorax, (ii) non-tension pneumothorax not related to trauma (defined as any of: >2 cm on chest X-ray, patient is breathless, patient has an oxygen requirement, postneedle aspiration the patient is clinically no better or the pneumothorax is still >2 cm on CXR), and (iii) non-tension pneumothorax secondary to trauma.

Round 3
Round 3 consisted of 14 statements which all originated from the 11 new statements in round 2 that did not achieve consensus.All 14 statements were answered by all respondents, of which 4 (28.5%)reached ≥80% positive consensus.No conditions reached ≥80% negative consensus.
Across all three rounds, 154 conditions reached ≥80% positive consensus and 1 condition (new presentation of uncomplicated Henoch-Schönlein purpura) reached ≥80% negative consensus (online supplementary table 1).Thirty-seven statements did not reach consensus (online supplementary table 2).For statements reaching positive consensus, the median Likert scale responses ranged from 4 to 5, whereas statements not reaching a consensus had median Likert scale responses ranging from 2 to 4. The process of rephrasing round 1 statements not reaching consensus achieved consensus in 33 statements in round 2 originating from round 1 and consensus of 4 statements in round 3 not reaching consensus in round 2. Adding the answer option in round 2 of 'I do not look after children with this condition' resulted in two conditions (status dystonicus and newly presenting infantile spasms) reaching consensus in round 2 that would not have reached the 80% threshold for consensus had the new response option not been available.

DISCUSSION
A list of significant paediatric emergency conditions has been derived by a panel of experts, through this modified Delphi study, to facilitate future research into the performance of PEWS and other safety systems in EDs.This list consists of 154 conditions (with accompanying degree of severity where relevant) across 17 illness and injury categories warranting acute admission from the ED to the hospital in the opinion of these senior clinical decision-makers.
The strength of this study has been the achievement of multicentre expert consensus for the majority of the illness and injury scenarios.The level of expert agreement is high, that is, an 80% consensus threshold which is above most baselines set in the literature.The study was intentionally anonymised to eliminate the introduction of persuasive bias that can arise from dominant individuals.Statements not reaching agreement in round 1 were refined according to participant feedback.Participants were given the opportunity to offer further suggestions on significant presentations, which were then tested in round 2. Statements not reaching consensus after a single round of testing all underwent a second round of testing, ensuring that the same rigour of evaluation by expert opinion was applied to each suggested condition.
A consensus was not achieved on 37 statements, demonstrating that for many conditions management will vary widely depending on the child's clinical presentation, clinician experience, individual practice and the available resources.Variation in opinion for these statements was reflected in the wider range of median statement responses ranging from 2 (disagree) to 4 (agree); in the qualitative feedback, this was evident in frequently encountered statements such as 'medicine is not black and white.'This statement was also seen for some conditions that achieved consensus, especially (i) if conditions could present in ways differing from those described, or (ii) if there were exceptional circumstances preventing the normal practice of admitting a child if circumstances had been otherwise.An example of the latter would be the use of ambulatory antibiotics when inpatient bed availability was insufficient, resulting in the child returning to the department for each drug dose administration.Such considerations only served to highlight to the study team that the need for hospital admission is not a fixed categorical decision and will vary depending on the unique circumstances surrounding each case.For ED purposes, we used the term 'suspected' to define many of the statements as the job of an ED clinician is decision-making on the balance of suspicion and risk assessment, with often the final diagnosis being made later in time.The number of children staying in the hospital for <24 hours is increasing.Therefore, we included short inpatient stays of <24 hours (including in Short Stay Units or Children's Assessment Units) in our definition of 'acute admission' to capture the respondents concern that the presenting condition was significant enough to delay discharge home from the ED, such as to allow a period of observation or acute treatment.
Round 2 statements were refined and rephrased when needed, based on the data and written feedback from round 1 respondents.
After round 1, it became apparent that some respondents were giving neutral answers for statements involving conditions they did not typically encounter.As a result, the option of 'I do not look after children with this condition' was introduced for subsequent rounds, to reduce neutral answers.Both of these interventions boosted the yield of statements reaching consensus.
This study improves on the methodology employed in previous preliminary work where the authors retrospectively assigned a classification of 'minor' or 'significant' along with a physiological system to diagnoses of children presenting to that ED during the study period.It was recognised that if prospective work on PEWS was to be undertaken, there was a need for a more standardised list of conditions, created using a systematic methodology to reflect a broader consensus of expert opinion, hence the purpose of this current work.

Limitations
The majority of responders were based in emergency care but most participants (65.2%) came from a paediatric background either in PEM (27.6%) or general paediatrics (37.6%), meaning that the bulk of opinion was formed by professionals specifically trained in paediatrics.Most respondents were from tertiary centres and so our results may not be representative of opinion from non-tertiary centres where resources and management pathways may differ.Separate specialty subgroups analysis was not performed.The survey was confined to the UK and Ireland, and therefore these results may not be applicable in different healthcare settings where the types of disease, disease prevalence and presentations may differ along with management practices, cultural attitudes and available resources.Since 68% of all the PERUKI and GAPRUKI sites that were invited to participate responded, this study does not reflect the opinion of the 32% of sites that failed to respond.At study sites, staff who were not PERUKI or GAPRUKI affiliated were not prevented from taking part in the survey if they so wished.This allowed us to be as representative as possible, though we appreciate that the survey was not distributed to every ED in the UK and Ireland, and not every single view will be captured due to the unavailability of some staff.Each round saw a predictable reduction in the number of sites responding, that is, responder fatigue.'Acute admission' was taken as the study's defining criteria, being readily measurable, clearly definable and less open to interpretation.Clearly, this does not imply that illnesses not requiring acute hospital admission are of no significance-many such conditions are now managed in ambulatory settings, in outpatient departments, in review clinic or admitted electively at a later opportunity.However, for ED purposes where a key part of a clinician's role is establishing whether a patient can be safely discharged (or not), defining illness/injury as 'significant' using the measure of whether the condition warrants 'acute' hospital admission was deemed an appropriate measure.
We acknowledge that this list of conditions is not backed up by data to support the need for admission based on the risk of morbidity or mortality; rather this list reflects current opinion of experts in this field.It could be argued that the study team (unintentionally, or otherwise) created a bias towards 'agree' responses from the outset by creating a list of statements around conditions that they considered to be significant, covering the majority of significant paediatric ED presentations.Testing every possible paediatric ED presentation would have overcome this potential for bias, but this would have resulted in an unwieldy number of scenarios making such a survey undeliverable.This pragmatic approach represents the key common clinical conditions which present to UK and Ireland EDs.
This study's achievement is the creation of a standardised list of statements that have been agreed by a consensus of expert opinion.
It is important to note, however, that the primary aim of this research was to develop a set of measures to act as a tool for future research purposes, such as the validation of PEWS and child safety systems in the ED.This list was not designed or validated to provide clinical guidance or be used to judge the quality of care between hospitals.Given that respondents were informed that the list was to be established for research purposes, it is possible that different answers with a different list would have arisen, should a specifically defined clinical emphasis have been placed in the expert's thoughts, rather than our defined research intent.

CONCLUSION
In conclusion, through consensus opinion, a list of 154 paediatric illnesses and injuries warranting acute admission to hospital from the ED has been established.This robust list of conditions can now be used to investigate the performance of PEWS and other child patient safety initiatives in the UK and Ireland systems, and potentially other countries with similar healthcare settings.

295 consultants identified from PERUKI and GAPRUKI networks. All invited to Round One 223 participated in Round One 177 participated in Round Two 148 participated in Round Three Ethics
No formal research ethical approval was required for this study as it was a survey of health professionals identified via established research networks.Participation was deemed as consent.

Table 1
Specialty background of survey respondents and category of centres respondingRESULTSA total of 295 consultants, identified as being available from PERUKI and GAPRUKI site leads, were invited to participate.Two hundred and twenty-three (76%) participated in round 1, 177 (60%) participated in round 2 and 148 (50%) participated in round 3. The greatest proportion of respondents (61.1%) came from an EM background (either adult EM, EM with PEM, or paediatrics with PEM) (table1).Most participants (61%) were based in a tertiary centre (table1).Forty-three (68%) of 63 sites responded.A summary of the percentage level of positive consensus for each illness category by Delphi round is given (table2).Categories with less overall acuity such as musculoskeletal, safeguarding and miscellaneous conditions seemed to score lower levels of agreement between the rounds.The flow of statements through the Delphi are summarised.All 161 round 1 illness/injury statements were answered by all respondents; 106 statements (65.8%) reached ≥80% positive consensus; none reached ≥80% negative consensus.The mean percentage of neutral responses was significantly greater for statements not reaching consensus versus those meeting consensus; 28.0% (95% CI 26.1% to 29.9%) and 4.7% (95% CI 3.8% to 5.5%), respectively (p=0.01).

Table 2
Summary of percentages (with numbers) of statements reaching ≥80% positive consensus in each round, for each illness/injury category