Oxygen systems to improve clinical care and outcomes for children and neonates: A stepped-wedge cluster-randomised trial in Nigeria

Summary

Background

Improving oxygen systems may improve clinical outcomes for hospitalised children with acute lower respiratory infection (ALRI). This paper reports the effects of an improved oxygen system on mortality and clinical practices in 12 general, paediatric, and maternity hospitals in southwest Nigeria.

Methods and findings

We conducted an unblinded stepped-wedge cluster-randomised trial comparing three study periods: baseline (usual care), pulse oximetry introduction, and stepped introduction of a multifaceted oxygen system. We collected data from clinical records of all admitted neonates (<28 days old) and children (28 days to 14 years old). Primary analysis compared the full oxygen system period to the pulse oximetry period and evaluated odds of death for children, children with ALRI, neonates, and preterm neonates using mixed-effects logistic regression. Secondary analyses included the baseline period (enabling evaluation of pulse oximetry introduction) and evaluated mortality and practice outcomes on additional subgroups. Three hospitals received the oxygen system intervention at 4-month intervals. Primary analysis included 7,716 neonates and 17,143 children admitted during the 2-year stepped crossover period (November 2015 to October 2017). Compared to the pulse oximetry period, the full oxygen system had no association with death for children (adjusted odds ratio [aOR] 1.06; 95% confidence interval [CI] 0.77–1.46; p = 0.721) or children with ALRI (aOR 1.09; 95% CI 0.50–2.41; p = 0.824) and was associated with an increased risk of death for neonates overall (aOR 1.45; 95% CI 1.04–2.00; p = 0.026) but not preterm/low-birth-weight neonates (aOR 1.30; 95% CI 0.76–2.23; p = 0.366). Secondary analyses suggested that the introduction of pulse oximetry improved oxygen practices prior to implementation of the full oxygen system and was associated with lower odds of death for children with ALRI (aOR 0.33; 95% CI 0.12–0.92; p = 0.035) but not for children, preterm neonates, or neonates overall (aOR 0.97, 95% CI 0.60–1.58, p = 0.913; aOR 1.12, 95% CI 0.56–2.26, p = 0.762; aOR 0.90, 95% CI 0.57–1.43, p = 0.651). Limitations of our study are a lower-than-anticipated power to detect change in mortality outcomes (low event rates, low participant numbers, high intracluster correlation) and major contextual changes related to the 2016–2017 Nigerian economic recession that influenced care-seeking and hospital function during the study period, potentially confounding mortality outcomes.

Conclusions

We observed no mortality benefit for children and a possible higher risk of neonatal death following the introduction of a multifaceted oxygen system compared to introducing pulse oximetry alone. Where some oxygen is available, pulse oximetry may improve oxygen usage and clinical outcomes for children with ALRI.

Trial registration

Australian New Zealand Clinical Trials Registry: ACTRN12617000341325.

Keywords:

Death rates – Neonates – Hospitals – Malaria – Nigeria – pneumonia – Oxygen – Neonatal sepsis

Introduction

Oxygen is an essential medical therapy that is important in resuscitation, medical, surgical, and obstetric care [1]. Oxygen therapy is particularly important for acutely unwell children and neonates, as hypoxaemia affects approximately one in five neonates and one in 10 children who are admitted to hospital [2] and increases their risk of death approximately 6-fold [3–6].

Currently, many seriously ill children in low- and middle-income countries (LMICs) lack access to oxygen therapy because of inadequate equipment, weak maintenance systems, high oxygen costs, and poor clinical practices [7–13]. Effective improvement of oxygen systems is complex, requiring context-specific technical, clinical, and managerial solutions [7].

Previous studies in low-resource settings have shown that improved oxygen systems can reduce inpatient deaths from pneumonia by up to 35% when implemented as part of a comprehensive quality-improvement program [14, 15]. If similar results were achieved from the scale-up of pulse oximetry and oxygen in the 12 countries with the largest number of child deaths, 148,000 pneumonia-related child deaths could be prevented annually [16]. Although hypoxaemia complicates many other common conditions, there are limited data on the impact of improved oxygen systems for children with non-pneumonia conditions [7, 15].

As a populous middle-income country in sub-Saharan Africa, Nigeria continues to struggle with high neonatal and child mortality rates (2017: under-5 mortality 100, neonatal mortality 33, per 1,000 live births) [17]. Recent global estimates show that Nigeria contributed one-third of under-5 malaria deaths and one-sixth of under-5 pneumonia deaths globally [18]. The conditions responsible for the largest number of child deaths in Nigeria are pneumonia (18%), malaria (14%), complications of prematurity (12%), neonatal encephalopathy and trauma (11%), diarrhoeal diseases (10%), and neonatal sepsis (5%) [18]. Data from sub-Saharan Africa suggest that hypoxaemia affects approximately 14% of children and neonates admitted to hospital, including 28%–49% of children with pneumonia, 8%–30% of children with malaria, and 22%–41% of neonates [2, 3, 6, 19–26].

The Oxygen Implementation Project was established in 2015 to evaluate an improved oxygen system in secondary health facilities in southwest Nigeria and provide evidence for future scale-up activities. The project was implemented with the support of the Gates Foundation, World Health Organization (WHO), and federal and state health agencies. Earlier findings have been published elsewhere [6, 27–29].

This paper reports the effects of an improved oxygen system on mortality and clinical practices for neonates and children under 15 years of age in 12 Nigerian hospitals.

Methods

This study obtained ethics approval from the University of Melbourne (1543797.1) and University of Ibadan/University College Hospital Ethics Committee, Ibadan, Nigeria (UI/EC/16/0413).

Study design

We conducted an unblinded stepped-wedge cluster-randomised trial in 12 hospitals. We introduced pulse oximetry to all hospitals in November 2015, followed by stepped introduction of the full oxygen system intervention over four sequences (three hospitals every 4 months from March 2016 to March 2017) following a prespecified timetable (Fig 1, S1 Fig). We completed postintervention data collection at the end of October 2017. We collected 2 years of preintervention data retrospectively from all hospitals for extended analyses (November 2013 to October 2015).

We chose a stepped-wedge design as the most efficient and pragmatic trial design, enabling all hospitals to receive the full oxygen system intervention over a period that was commensurate with the logistics of implementation. Clusters were at the hospital level, the natural level at which the intervention would be implemented. Geographic separation between hospitals ensured minimal risk of contamination. Additional detail on study methods has been published elsewhere [27] (protocol DOI: https://doi.org/10.1186/s13063-017-2241-8).

Participants

We selected hospitals in collaboration with state health authorities, purposefully selecting hospitals that were representative of secondary-level health facilities and that provided inpatient care to children. Hospitals were eligible if they were registered as secondary-level health facilities and admitted at least 150 children per year. We included government and mission-run hospitals but not other private hospitals. We included hospitals in four states of southwest Nigeria (Oyo, Ondo, Ogun, and Osun states). We included nine general hospitals and three hospitals that focused on paediatric and/or maternity care (see S1 Text for additional detail on participating hospitals). AGF and AIA enrolled hospitals following an informational meeting with hospital representatives, with consent provided by hospital representatives prior to randomisation. We did not require individual patient/carer consent.

We collected data on all children (aged <15 years), including neonates (aged <28 days), admitted to participating hospitals during the study period. We had no additional exclusion criteria. We assumed participants admitted in different periods were independent. For effectiveness analysis, we excluded participants admitted during a 2-week ‘wash-in’ period at the beginning of each crossover period to implement the intervention and avoid contamination.

Randomisation

We conducted randomisation and allocation at a team meeting in Ibadan, Nigeria, using a computer-based random sequence generator to allocate hospitals to receive the full oxygen system at prespecified dates. We stratified hospitals by size (large > 1,500 paediatric admissions per year; small/medium < 1,500) prior to randomisation, ensuring one of the four largest hospitals was represented in each step. Concealment of the intervention or its start date was not possible, as the intervention required active participation from hospital staff in planning and implementation activities.

Intervention

The multifaceted intervention (full oxygen system) was delivered at the cluster (hospital) level and involved (1) a standardised oxygen equipment package, (2) clinical education and support, (3) technical training and support, and (4) infrastructure and systems support (Box 1, S2 Table). We designed the intervention based on experience from previous oxygen projects [7, 30, 31] and WHO guidelines [32–34] and were informed by educational, behaviour change, and implementation theories and strategies [35–39]. We adapted the intervention to local needs using a participatory planning process including a joint stakeholder meeting of representatives from all hospitals, site visits and needs assessment [13], and regular meetings between the project team and hospital staff. AGF, AIA, AAB, and HG conducted randomisation, invited hospital representatives to the major planning meeting, and then obtained consent to participate.

Procedures

We commenced prospective data collection and introduced pulse oximetry to all hospitals in November 2015. We introduced pulse oximetry prior to implementation of the full oxygen system to enable quantification of hypoxaemia prevalence and evaluation of the effect of the introduction of pulse oximetry alone (extended analysis). Hospitals received the full oxygen system in March 2016, July 2016, November 2016, or March 2017, in the randomised order. We conducted retrospective data collection in late 2015, covering the preintervention period (November 2013 to October 2015) for extended analysis.

During the preintervention period, hospitals provided care as usual, including oxygen therapy using their existing supplies. During the pulse oximetry period, hospitals continued usual care with the additional availability of project-supplied pulse oximeters. Following the full oxygen system intervention, hospitals provided clinical care using the new oxygen systems, clinical guidelines, and maintenance protocols.

Program theory

We hypothesised that to reduce mortality, effective oxygen systems must achieve (1) reliable oxygen access and (2) good clinical use of oxygen targeting hypoxaemic patients and that this would require technical, clinical, and administrative components [7]. We expected implementation of the full oxygen system to have greatest benefit for children with acute lower respiratory infection (ALRI) and preterm neonates, some effect on other conditions commonly associated with hypoxaemia (e.g., sepsis, cerebral malaria, meningitis, neonatal encephalopathy), and negligible effect on conditions rarely associated with hypoxaemia (e.g., diarrhoea, neonatal jaundice).

Outcomes

To evaluate effectiveness, our primary outcome was death (defined as death during hospitalisation or discharged expected to die), comparing the full oxygen system period to the pulse oximetry period. We report the primary outcome for four main subgroups: (1) children (excluding neonates), (2) children with ALRI, (3) neonates, and (4) preterm and/or low birth weight (LBW) neonates.

Secondary outcomes included (1) clinical outcome of death for additional subgroups (malaria, acute febrile encephalopathy [AFE], diarrhoea, neonatal encephalopathy, and neonatal sepsis) and (2) practice outcomes relating to oxygen-related quality of care (proportion of children/neonates with documented oxygen saturation, proportion of hypoxaemic children/neonates who received oxygen therapy, proportion of children/neonates who received oxygen therapy and who had a documented indication). We analysed outcomes (1) comparing the full oxygen system period to the pulse oximetry period and (2) comparing the full oxygen system period and the pulse oximetry period to the preintervention period (see ‘extended analysis’ description below).

We used standard WHO case definitions to classify diagnoses (ALRI = cough or difficult breathing and any of the following: fast breathing, lower chest wall indrawing; AFE = fever and any of the following: seizures, altered conscious state; diarrhoea = >3 loose stools per day, not >14 days’ duration; malaria = fever [or history of fever] and positive malaria test [rapid test or microscopy]; preterm/LBW = <37 weeks’ gestational age or <2,500-g birth weight) [34]. We based diagnostic classification on documentation in clinical records (i.e., if a particular sign was not recorded, we assumed that it was not present). Additional neonatal diagnostic classifications (neonatal sepsis, jaundice, neonatal encephalopathy) were classified according to the admission diagnosis.

All outcomes were determined using data extracted from case notes by trained data collection nurses who were based within each hospital but employed by the project. Standardised case report forms (CRFs) were completed by data collection nurses, returned to the project manager (AAB) for quality checking, and then submitted to the data manager (OBO) for double entry by trained data entry clerks using EpiData 3.1 (EpiData Association, Odense, Denmark). HG, AAB, and OBO reviewed datasets periodically in accordance with quality assurance and data management protocols [27].

Statistical analysis

We calculated power using the Hussey and Hughes method [40, 41], basing assumptions on admission, prevalence, and mortality data from our needs assessment [13], and informed by intracluster correlation coefficient (ICC) estimates from previous studies [42]. We estimated that we would enrol 16,770 children and 7,670 neonates (1,290 preterm/LBW) during the periods of pulse oximetry and full oxygen system and based our power calculations on this time period only (without the extended preintervention period). Our original estimation contained a mathematical error, and after correction, we calculated approximately 50% power to detect a 20% reduction in overall child mortality (4% to 3.2%, ICC 0.018), 35% power to detect a 35% reduction in child pneumonia mortality (6.1% to 4.0%, ICC 0.032), 50% power to detect a 20% reduction in neonatal mortality (8.2% to 6.6%, ICC 0.063), and 55% power to detect a 25% reduction in preterm/LBW mortality (18% to 13.5%, ICC 0.123). We calculated 99% power to detect a 10-percentage-point change in proportion of hypoxaemic children who received oxygen therapy (ICC 0.05).

We described patient characteristics by study period (preintervention, pulse oximetry, and full oxygen system periods). We summarised binary variables using frequencies and percentages. We summarised continuous variables using means and standard deviations or medians and interquartile ranges (IQRs) for non-normally distributed data.

We analysed outcomes using intention-to-treat principles. We excluded participants with missing outcome data from analysis. We report intervention effects as odds ratios (OR) with 95% confidence intervals (CIs) and report the crude proportions in tabular form and visually in heat-map tables (by hospital and step).

Analytical models

To evaluate effectiveness, we built mixed-effects logistic regression models fitted to individual-level binary outcome data [43, 44], following a prespecified analysis plan (S2 Text). Primary analysis compared the stepped introduction of the full oxygen system to the pulse oximetry period (steps 7–12).

Our primary analysis model included fixed effects for intervention and time (4-month steps), with random effects for cluster (hospital). To decide whether extensions to the basic model were warranted, we looked for evidence of hospital–time interaction (using a variance component for random hospital-by-time effects in an extended analysis model and comparing this with the simpler model that assumed random hospital effects only, using a likelihood ratio test) [45]. We found no evidence of hospital–time interaction for the primary clinical outcome (death). However, there was evidence of interaction for the practice outcomes, and we adjusted models accordingly.

We did ‘severity-adjusted’ analysis including individual signs of illness severity, age, sex, and type of hospital (mission versus government) as additional fixed effects, as we believed clinical outcomes may be confounded by changes in admission patterns (e.g., hospitals that referred sick patients elsewhere prior to having oxygen available would keep these sicker patients, some of whom would die) (details in S2 Text).

We fitted these models to an extended time period (‘extended analysis’) including the preintervention data (steps 1–6), to evaluate the effect of pulse oximetry, and the full oxygen system against the previous standard of care. We estimated the ICC and corresponding 95% CI using Stata’s ‘estat icc’ program after fitting the mixed-effects models. We performed all analyses using Stata 15.1 (Statacorp, College Station, TX, USA).

This trial was registered with the Australian New Zealand Clinical Trials Registry: ACTRN12617000341325 (http://www.ANZCTR.org.au/ACTRN12617000341325.aspx).

Results

During the 2-year stepped crossover period (November 2015 and October 2017), 7,716 neonates and 17,143 children were admitted to participating hospitals and contributed to primary analysis. An additional 24,117 children (8,813 neonates and 15,304 older children) were admitted during the baseline period (from November 2013) and contributed to extended analysis. Fig 1 shows the distribution of admissions by hospital and step.

Admission numbers dropped in early 2016 in the context of economic recession, healthcare workforce industrial action, and hospital closures (Fig 1, Table 1). The distribution of participants between government and mission hospitals is unbalanced during periods of industrial action and closure of government hospitals (e.g., June/July 2016).

The distribution of patient diagnoses changed little over time, although patients tended to be slightly older during the intervention period (Table 1). Children tended to present with more severe illness during the pulse oximetry and full oxygen system periods (e.g., higher proportion with altered conscious state or WHO emergency signs).

We had complete clinical outcome data for 24,778/24,859 (99.67%) of participants in the pulse oximetry and full oxygen system periods (prospective data collection) and 23,812/24,117 (98.74%) of participants in the preintervention period (retrospective data collection). We had complete practice outcome data for 24,534/24,859 (98.69%) of participants in the pulse oximetry and full oxygen system periods and 20,677/24,117 (85.74%) of participants in the preintervention period.

Fidelity and variation from protocol

The intervention was deployed as planned, with equipment installation and training occurring over 2–4 days at each hospital, usually in the first 2 weeks of the designated installation month (S2 Table). Solar-power installations were delayed because of technical problems with the initial solar installations (in hospitals H1, H2, and H3) requiring rectification, system redesign, and change of provider. This delayed subsequent solar installations and required temporary use of alternative power sources (mains power and/or generators). Despite this setback, all intervention hospitals had continuous access to oxygen therapy from the newly installed systems throughout the intervention period.

The project team visited each hospital at least once every 3 months and had regular communication by phone and email. Hospital staff generally reported equipment problems promptly, and our project team provided prompt assistance (including repair or replacement), ensuring patients had continuous access to oxygen therapy.

Hospitals took up some, but not all, aspects of the project’s support for systems. Most hospitals started multidisciplinary ‘oxygen team’ meetings. Few hospitals improved their medical record keeping. All hospitals continued to charge patients for oxygen, and most did not adopt measures to reduce costs to patients until late 2017. During the study period, Nigeria experienced an economic recession, resulting in reduced income for hospitals, unpaid healthcare worker wages, and associated industrial action [46]. Although hospital staff and administrators insisted that these financial pressures did not impede patient access to oxygen, we received anecdotal reports of more patients becoming indebted to the hospital and leaving against medical advice.

Intervention effects

Primary clinical outcomes

Table 2 shows the crude rates and intervention effect sizes for the primary analysis of clinical outcomes.

Primary analysis suggests that, compared to the pulse oximetry period, the full oxygen system was not associated with death for children overall (aOR 1.06; 95% CI 0.77–1.46; p = 0.721) or children with ALRI (aOR 1.09; 95% CI 0.50–2.41; p = 0.824) and was associated with increased risk of death for neonates overall (aOR 1.45; 95% CI 1.04–2.00; p = 0.026) but not preterm/LBW neonates (aOR 1.30; 95% CI 0.76–2.23; p = 0.366). Severity-adjusted analysis was qualitatively similar to the basic model, with stronger evidence of harm for neonates (Table 2).

At reviewer request, we conducted post hoc analysis on subgroups with hypoxaemia and children under 5 years of age, with qualitatively similar results (S3 Table).

Secondary clinical outcomes and extended analysis of preintervention period

Table 3 shows the crude rates and intervention effect sizes for the primary and secondary clinical outcomes, including the extended analysis comparing pulse oximetry and full oxygen periods against the preintervention period.

Compared to the preintervention period, the pulse oximetry and full oxygen system intervention were associated with decreased risk of death for children with ALRI (aOR 0.45, 95% CI 0.17–1.18, p = 0.104; aOR 0.57, 95% CI 0.31–1.04, p = 0.066), with qualitatively similar results from severity-adjusted analysis (aOR 0.33, 95% CI 0.12–0.92, p = 0.035; aOR 0.50, 95% CI 0.26–0.98, p = 0.044). Neither pulse oximetry nor the full oxygen system was associated with death for children overall, neonates overall, or any of the other subgroups, compared to the preexisting standard of care.

Practice outcomes

Table 4 shows the crude rates and intervention effect sizes for the practice outcomes, including the extended analysis across all three study periods. Use of pulse oximetry in the clinical care of hospitalised children improved dramatically across the study periods (from <5% of admissions to >90%), with lesser change evident for oxygen practices.

For neonates, there was a statistically nonsignificant increase in overall oxygen use (aOR 1.34; 95% CI 0.96–1.87; p = 0.085) following introduction of the full oxygen system, with no change in oxygen administration for neonates with hypoxaemia (aOR 0.77; 95% CI 0.34–1.75; p = 0.531) compared to the pulse oximetry period. Pulse oximetry had minimal impact on overall oxygen use (aOR 1.29; 95% CI 0.82–2.03; p = 0.264) or the use of oxygen for neonates with signs of hypoxaemia (aOR 1.06; 95% CI 0.54–20.9; p = 0.857) compared to the preintervention period.

For children outside the neonatal period, results suggest that the full oxygen system increased overall oxygen use (aOR 1.89; 95% CI 1.34–2.66; p < 0.001) and improved oxygen administration for those with hypoxaemia (aOR 2.71; 95% CI 1.12–6.56; p = 0.027) compared to the pulse oximetry period. Pulse oximetry had minimal impact on overall oxygen use (aOR 0.72; 95% CI 0.43–1.19; p = 0.200) or the use of oxygen for those with signs of hypoxaemia (aOR 1.00; 95% CI 0.51–1.95; p = 0.995).

Heat-map graphs suggest that pulse oximetry and oxygen practice changes were achieved incrementally over a number of months, with most of this change occurring early in the pulse oximetry period (Fig 2). Of note, the proportion of hypoxaemic (SpO₂ < 90%) children receiving oxygen therapy increased to >75% coverage within 4–8 months of receiving pulse oximeters. Three hospitals are outliers (H9, H12, and H5’s children’s ward), being the only hospitals that genuinely had no oxygen access for children prior to the intervention and showed no improvement in oxygen use until after the full oxygen system was implemented.

Heat-map graphs suggest oxygen use was better targeted at neonates and children with hypoxaemia in the pulse oximetry period, but this is more weakly reflected in the adjusted effect estimates (aOR 2.00, 95% CI 0.80–4.95, p = 0.136 and aOR 1.21, 95% CI 0.55–2.68, p = 0.634 for neonates and children, respectively) (Fig 2, Table 4). The ‘indication for oxygen’ outcome uses a composite measure of hypoxaemia (SpO₂ < 90% on admission, or signs of hypoxaemia if no SpO₂ documented), which may be biased because of the dramatic change in SpO₂ documentation over the study periods (i.e., reflecting patients with signs of hypoxaemia in the preintervention period and patients with SpO₂ < 90% in the latter periods).

Discussion

This study reports results from the first cluster-randomised trial of improved oxygen systems. We found that the improved oxygen system had no effect on outcomes for children when compared against the introduction of pulse oximetry. However, pulse oximetry may have reduced the risk of death from pneumonia by approximately 50% compared to baseline. We found that the improved oxygen system was associated with increased risk of neonatal death when compared against the introduction of pulse oximetry. However, neither pulse oximetry nor the full oxygen system had any effect on neonatal death when compared against baseline.

As with all stepped-wedge trials, this study is complex in design and analysis, and interpretation of the estimated treatment effects must be made in the context of the study and with consideration to potential additional confounders. With that in mind, we offer two brief comments before discussing the results.

Firstly, this study was not an individual trial of oxygen versus no oxygen. Rather, the introduction of an improved oxygen system was compared to a previous standard of care augmented with pulse oximetry. At baseline, this previous standard of care included a functional oxygen cylinder or concentrator in 11/12 hospitals and in the paediatric area of 6/12 hospitals [29].

Secondly, during the study period, Nigeria experienced a major economic recession, which impacted individuals, households, health facilities, and the broader health system [46]. Participating government hospitals had extended periods of unpaid salaries, resulting in industrial action, hospital closure, increases in user fees, and deterioration in morale [28]. In a user-pay system, economic hardship for staff and patients likely contributed to the lower admission numbers and increased severity of presenting illness during the intervention period and possibly impacted quality of care and clinical outcomes.

The far-reaching impact of the economic recession may have introduced confounding that could not be accounted for in analyses. Our primary analysis model for effectiveness included adjustment for clustering (accounting for hospital-specific influences on outcomes) and time (accounting for step-specific influences on outcomes). Our severity-adjusted analysis model adjusted for additional patient-specific factors, including age, signs of severe illness, and diagnosis. In the context of the recession, the severity-adjusted model may represent a better estimate of intervention effect.

Interpreting results

We will now explore the clinical and practice outcomes in light of our program theory and the broader study context to provide a balanced and informed interpretation of results.

Clinical impact for children

We expected the improved oxygen system to have the most benefit for children with ALRI. We found no mortality benefit for children with ALRI or for other subgroups of interest in primary analysis (comparing the full oxygen system to the pulse oximetry period), although the study was underpowered to do so. However, extended analysis suggested that the improved oxygen system realised a mortality benefit for children with ALRI compared to the baseline period and that it was fully realised through the introduction of pulse oximetry.

These findings are consistent with our practice outcome results. Our baseline assessment suggested that oxygen supplies were limited in all participating hospitals, with only half of hospitals having oxygen supplies available for children or neonates at the time of assessment [13]. However, the larger hospitals typically had a better supply and, since they contributed more observations to our current study overall results, may be more reflective of this scenario. Furthermore, practice data suggest that most hospitals significantly improved their oxygen practices during the pulse oximetry period, reaching high levels of oxygen therapy administration for hypoxaemic children prior to the full oxygen system period.

This leads us to two likely interpretations. Firstly, when oxygen is available, pulse oximetry enables healthcare workers to better target it to those who are hypoxaemic. Indeed, even though oxygen supplies are insufficient to meet the needs of all hypoxaemic children, pulse oximetry may improve oxygen practices sufficiently to reach those who would benefit the most (i.e., with more severe hypoxaemia) with less opportunity for further mortality reduction with enhanced oxygen supplies. Secondly, when oxygen supplies are limited, objective evidence of high hypoxaemia prevalence, through the use of pulse oximetry, enables hospitals to mobilise additional oxygen supplies and prioritise paediatric patients. Anecdotal feedback from hospitals staff supports this theory, with only two small hospitals (H9 and H12) unable to mobilise additional oxygen from other wards and remaining without oxygen for the entire pulse oximetry period.

These theories are supported by previous before–after studies in two Papua New Guinean hospitals that reported ALRI mortality reductions with the introduction of pulse oximetry (4.2% down to 2.7% and risk ratio 0.65; 95% CI 0.41–1.02) [12, 47] in the context of reasonable access to oxygen cylinders at baseline (e.g., oxygen available for 87% of children with SpO₂ < 90% [12]). These effect sizes are similar to our findings (aOR 0.57; 95% CI 0.31–1.04; p = 0.066) and to previous before–after studies involving the introduction of pulse oximetry and improved oxygen systems [14, 15, 48]. Of these, the highest-quality data come from the five-hospital study from Papua New Guinea, which found that an improved oxygen system reduced inpatient child pneumonia case fatality rates (CFRs) by 35% (risk ratio 0.65; 95% CI 0.52–0.78) [14].

We did not find evidence of a mortality benefit for children overall, in either the primary analysis or the extended analysis. Previous studies have reported reductions in non-pneumonia CFRs following introduction of an improved oxygen system, including the study in Papua New Guinea, which found non-pneumonia CFRs were reduced by 21% (risk ratio 0.79; 95% CI 0.70–0.89) [31, 49].

Clinical impact for neonates

We expected the improved oxygen system to have the most benefit for preterm neonates. Surprisingly, our primary analysis suggested that the full oxygen system may have increased the odds of death for neonates overall (aOR 1.45; 95% CI 1.04–2.00), although there was no significant increase in the preterm/LBW neonate subgroup (aOR 1.30; 95% CI 0.76–2.23). In contrast, extended analysis suggested no effect of either pulse oximetry or the full oxygen system on odds of neonatal death when compared to the baseline period. The latter would fit with practice outcome findings of high oxygen use for neonates in the larger hospitals at baseline (and little room for improvement).

It is possible that the finding of harm to neonates may be due to chance, given the multiple analyses and discordance in results between the primary analysis and the extended and secondary analyses. However, there are other potential explanations that warrant consideration.

First, we know that excessive oxygen administration can cause harm. Oxygen therapy can increase the production of reactive oxygen species, which can overwhelm the body’s antioxidant defences and damage the structure and function of cells [50]. In practical terms, this has the greatest implications for preterm neonates, particularly for their developing eyes (retinopathy of prematurity) and lungs (pulmonary dysplasia leading to chronic lung disease) [50]. For this reason, neonatal guidelines recommend targeting oxygen saturations in preterm neonates receiving oxygen [32].

However, giving more oxygen to preterm neonates is unlikely to result in higher mortality. Five large studies have compared the use of higher (91%–95%) to lower (85%–89%) oxygen saturation targeting for extremely preterm neonates, demonstrating increased mortality in the lower-saturation group (risk ratio 1.17; 95% CI 1.04–1.31) [51]. In our study, harm from excessive oxygen is even less likely because oxygen usage for neonates at the larger hospitals, where the vast majority of neonates were admitted, was already quite high at baseline, with negligible change across study periods.

Second, it is possible that implementation of our full oxygen system resulted in broader unintended consequences that may have contributed to excess mortality (e.g., distracting staff from other duties). User feedback suggests that, although healthcare workers initially found pulse oximetry to be burdensome, they felt that the full oxygen system made their work easier and reduced stress [28].

Third, it is possible that the mortality outcome was confounded by externally driven factors that we could not account for in analysis (and that were perhaps stronger for neonates than older children). In the context of the recession, these confounders may have included changed admission criteria (e.g., admitting more very sick neonates that would have otherwise been palliated or, conversely, fewer mildly unwell neonates that would have survived), care-seeking behaviours (e.g., delayed presentation, poorer birth care), care practices (e.g., not giving antibiotics promptly, increased cost of care), or general quality of care (e.g., staff levels, motivation). If there was general confounding, then we might expect to see similar effect across all neonates (or potentially less negative effect among particular neonates who actually benefited from oxygen).

Fourth, analysis of outcomes on a hospital-by-hospital basis shows significant heterogeneity (S4 Table), and it is possible that the multifaceted intervention worked differently in different contexts. For example, H1 was a large mission hospital that maintained services relatively unchanged throughout the economic recession and realised mortality reductions for both children and neonates. In contrast, H7 was a large government hospital that provided free care to patients and recorded few deaths preintervention but was strongly impacted by the economic recession (including introducing patient fees and cutting services). H7 realised mortality increases for both children and neonates. Small study numbers in individual hospitals make definitive subgroup analysis challenging. However, preliminary exploration of this heterogeneity has identified a number of possible contextual and mechanistic factors that will be evaluated more fully in a separate paper.

We are not aware of any previous oxygen improvement studies that included neonates, and our findings raise some important questions. However, we urge caution in interpreting these data, as confounding and chance may account for some of the mortality differences we observed.

Limitations

Using a stepped-wedge design enabled us to gain the benefits of randomisation and learn valuable lessons about implementation along the way, but it did make adjusting for temporal effects harder than if it were a parallel cluster trial. Stepped-wedge designs work best when potential confounders remain relatively stable during the study period, and this was potentially compromised by the major economic and social changes in Nigeria that likely introduced additional confounding.

We used a 2-week ‘wash-in’ period; however, actual practice change took longer, and oxygen use changed less than we had anticipated. A heat map helped visualise this incremental change for practice outcomes, but low death rates make heat-map figures less useful for clinical outcomes.

We included over 24,000 participants in primary analysis; however, the small number of clusters and the unbalanced cluster size limited the power of our study. This was particularly challenging for neonatal data, with the four largest hospitals contributing 84% of the data. The power to detect significant change in mortality was lower than expected because of (1) lower-than-expected baseline CFRs, (2) lower overall admission numbers, and (3) higher-than-expected ICCs. We reported the primary clinical outcome for multiple subgroups and evaluated multiple secondary practice outcomes. We used prespecified analysis methods but recognise the risk of chance findings. Both positive and negative findings should be interpreted with caution.

We used data from clinical documents using an approach similar to others conducting implementation research through clinical research networks [52]. Our use of dedicated research nurses minimised the amount of missing data, and our audit of documentation practices prior to starting the study reassured us that documentation practices overall were excellent [13]. We used standardised case definitions but did not impose additional restrictions on healthcare workers regarding their documentation. This could result in diagnostic misclassification or missed signs of severity but is unlikely to have altered our findings substantively.

Practical implications

Our study was part of a pragmatic field trial to better inform efforts to improve oxygen access to children and newborns in Nigeria and other LMICs. We have reported results from our needs assessment and hypoxaemia prevalence estimates previously, showing that hypoxaemia is common in both respiratory and nonrespiratory childhood illnesses and that many hospitals have limited access to pulse oximetry and oxygen therapy [6, 29].

In recognition of this challenge, the Nigerian federal government recently released a national oxygen policy and strategic road map for the scale-up of oxygen services in Nigerian health facilities [53–55]. We hope that emerging findings from our project will help others seeking to improve oxygen access, and we offer the following humble suggestions.

• Pulse oximetry is key to improving oxygen use and likely improves other aspects of inpatient care (e.g., assessment and monitoring, confidence and motivation among healthcare workers, communication and trust with caregivers). As a relatively affordable and potentially life-saving practice, pulse oximetry should be central to all efforts to improve oxygen systems and paediatric hospital care—including as a core vital sign in assessing all acutely unwell children and newborns.

• Oxygen therapy is part of a system of care, and outcomes are influenced by many other factors. The challenges to oxygen access include many factors that also compromise other aspects of care, including weak equipment maintenance systems, poor power supplies, staff shortages, lack of clinical guidelines, and challenges of interdisciplinary cooperation. These challenges provide opportunities to use oxygen access as a means to reveal systemic weaknesses and incrementally improve the broader hospital system for improved patient outcomes.

• Oxygen, administered as low-flow oxygen or with continuous positive airway pressure (CPAP), is an important part of neonatal care. We advise readers against making changes to neonatal oxygen policies or practices on the basis of this study and to adhere to WHO guidelines [32]. Oxygen, like any medication, should be given judiciously, accompanied by treatment of the specific medical condition and good supportive care. Future research and quality-improvement activities should seek to better understand how oxygen can be used most safely and effectively in neonatal care—particularly for the most at-risk preterm neonates.

Conclusion

For children, there is consistent evidence across primary and secondary analyses that the introduction of pulse oximetry improved oxygen practices in Nigerian hospitals and may have reduced ALRI mortality, with no additional clinical benefit from the full oxygen system. This finding is plausible and consistent with previous studies and may indicate that pulse oximetry improves the use of existing oxygen supplies and encourages hospitals to mobilise additional oxygen supplies for paediatric use (and may improve other aspects of care).

For neonates, the interpretation is more challenging. On one hand, there was suggestion of harm for neonates when comparing the full oxygen system period to the pulse oximetry period, without a clear explanation. On the other hand, we found no effect of the improved oxygen system on neonatal outcomes compared to baseline and negligible change to oxygen use for neonates.

Supporting information

S1 CONSORT Checklist [docx]

S1 Fig [tif]
Participant flow diagram.

S1 Text [docx]
Description of participating hospitals.

S2 Text [docx]
Analysis plan and severity-adjusted analysis model.

S1 Table [docx]
Description of the improved oxygen system intervention.

S2 Table [docx]
Timetable of project implementation.

S3 Table [docx]
Primary outcomes subgroup analyses.

S4 Table [docx]
Primary clinical outcomes by individual hospital.