Assessing emergency obstetric and newborn care: can performance indicators capture health system weaknesses?

Abstract

Background

Regular monitoring and assessment of performance indicators for emergency obstetric and newborn care can help to identify priorities to improve health services for women and newborns. The aim of this study was to perform a district wide assessment of emergency obstetric and newborn care performance and identify ways for improvement.

Methods

Facility assessment of 13 dispensaries, four health centers and one district hospital in a rural district in Tanzania was performed in two data collection periods in 2014. Assessment included a facility walk-through to observe facility infrastructure and interviews with facility in-charges to assess available services, staff and supplies. In addition facility statistics were collected for the year 2013. Results were discussed with district representatives.

Results

Approximately 65% of expected births took place in health facilities and 22% of women with complications were treated in facilities expected to provide emergency care. None of the facilities was, however, able to perform at the expected level for emergency obstetric and newborn care since not all required signal functions could be provided. Inadequate availability of essential drugs such as uterotonics, antibiotics and anticonvulsants as well as lack of ability to perform vacuum extraction and blood transfusion limited performance.

Conclusions

Performance of emergency obstetric and newborn care in Magu District was not in accordance with expected guidelines and highly influenced by lack of available resources and an insufficiently functioning health care system. Improving assessment approaches, to look beyond the signal functions, can capture weaknesses in the system and will help to understand poor performance and identify locally applicable ways for improvement.

Background

Emergency Obstetric and Newborn Care (EmONC) must be available and accessible to all women in order to avert maternal deaths and morbidities from the most frequent causes as haemorrhage, sepsis, eclampsia, obstructed labour and abortion complications. EmONC can also prevent fresh stillbirths and early neonatal deaths and morbidities [1].

The past decades Tanzania has given Reproductive Maternal Neonatal and Child Health (RMNCH) high priority which at policy level resulted in development of the ‘One Plan – to accelerate reduction of maternal, newborn and child deaths between 2008 and 2015’ [2, 3]. Implementation of policies, primarily the responsibility of regional and district councils, has received less attention, especially with regards to specific maternal and neonatal interventions [2, 4]. As a consequence insufficient progress has been made to reduce maternal and neonatal mortality, in particular in the Western and Lake Zone regions [2, 3].

There is no single intervention that works to achieve the desired reduction in maternal and neonatal mortality [5], as direct and underlying causes for poor maternal health and mortality are complex [6]. Along the continuum of care from pre-conception to post-birth as well as from individual-, family-, community level up to facility and policy level, packages of interventions can have great impact if implemented well [7]. Skilled care at birth and access to EmONC is of vital importance especially for those 15% of pregnant women who are expected to experience life-threatening maternal and/or neonatal complications [8].

Over the years useful process indicators have been developed to evaluate availability, accessibility and quality of EmONC [9, 10]. Regular assessment and monitoring of EmONC indicators can help programme planners in the field of RMNCH to identify priorities and needs to strengthen health services for women and their newborns [10]. The aim of this study was to perform a district-wide assessment of EmONC performance, to identify gaps and propose ways for improvement.

Methods

Setting

Magu District is one of eight districts of Mwanza Region of Tanzania and is bordered to the north by Lake Victoria and to the west by the city of Mwanza, which is the second largest city in Tanzania. The estimated population in 2012 was 299,759 [11]. Magu district had 46 health facilities: one government district hospital, four government health centres, 30 government dispensaries, nine private dispensaries, one faith based dispensary and one parastatal dispensary. Out of all government health facilities 31 provided antenatal, delivery and postnatal care. In terms of ownership, facilities are funded and supported by the district council of which the Council Health Management Team (CHMT) is responsible for decision-making. Main actors in the provision of maternal health care are (assistant) medical officers, clinical officers, enrolled and registered nurses/midwives, as well as medical attendants.

Since 2012 the Woman Centered Care Project (WCCP), a project of the African Woman Foundation [12–14], has been performing baseline research activities in the district and formed a partnership with the Magu District Council with the aim to improve accessibility and availability of quality maternal health care. Although focus has mainly been on antenatal care services and community involvement, the project together with members of the CHMT discussed the need for an EmONC assessment as improvements in the health situation of women in the district can only be reached when the complete system functions well along the continuum of care.

Data collection

This study is based on two periods of data collection. A facility survey was performed at 13 dispensaries, one health centre and the district hospital in April-May 2014, including investigation of the availability of supplies and medicines to provide basic EmONC. Facilities were selected purposefully based on participation in the WCCP project locations. In December 2014 an EmONC assessment was performed including assessments at the district hospital and all four health centres in the district regarding their EmONC performance within the past three months. The research team interviewed (assistant) clinical officers or nurses in charge of the facilities. Facility visits included a walk-through to observe facility infrastructure, available services, staff and supplies. Maternal death review forms of 2013 were also reviewed. During the field visits and during meetings with different members of the CHMT and other stakeholders relevant observations were documented and discussed.

Data collection tools

For the facility survey in April-May a tool was developed based on the Johns Hopkins Program for International Education in Gynecology and Obstetrics (JHPIEGO) manual for monitoring birth preparedness and complication readiness [15]. The tool was digitalized using Magpi Data Collection Software [16]. Health facilities were visited to collect information on facility statistics for the year 2013, facility infrastructure, available services, equipment and supplies. The survey was conducted while facilities were awaiting their new batch of quarterly supplies. If supplies were told to be present but not able to be located or accessible during our visit, they were documented as absent. Non-functioning equipment was also recorded as not available.

For the EmONC assessment, the handbook ‘Monitoring Emergency Obstetric Care’ was used [10]. EmONC signal functions are the key medical interventions that are used to treat direct obstetric complications that cause the vast majority of maternal deaths around the globe. The seven functions of Basic EmONC are expected to be provided at health centre and dispensary level [3]. In referral district hospitals the nine functions of Comprehensive EmONC should be provided to save lives (see Table 1). For assessment of signal functions at dispensary level, focus was on availability of materials and supplies, rather than on performance in the last three months. Therefore signal function 4, manual removal of the placenta, was not included at dispensary level.

Table 1 Signal functions for basic and comprehensive EmONC services

Data collection team

The data collection team consisted of members and volunteers of the WCCP, in collaboration with their partners. For the facility survey, a volunteer nurse-midwife from Australia performed facility visits together with ASM. The EmONC assessment was performed by co-authors JvR and JS, experts of the Working Party on International Safe Motherhood and Reproductive Health in the Netherlands, both consultant obstetricians with several years of working experience in Tanzania and other sub-Sahara African countries. Facilities were notified of the visit prior to arrival. For all facility visits representatives of the District Medical Officer accompanied the team and joined in the assessment. Interviews during the facility survey were conducted with the help of a translator, for the EmONC assessments interviews were done in Kiswahili, with occasional translation.

Data analysis

All data was entered in Microsoft Excel Version 15.0 to allow for calculation of frequency. To analyse the indicators, data collected from both the EmONC assessment and the facility survey at lower level facilities were used. Birth data was collected from registers (delivery register, admission register, operating theatre register, monthly reports) of the year 2013. If data was not accessible for the whole year, data was extrapolated from known parts of the year. To estimate the expected number of births in the district a Crude Birth Rate of 39 was used based on figures of the Demographic and Health Survey 2010, for mainland rural areas [17]. Research notes based on field visits were compared and discussed within the research team. A report of the facility survey was presented to the District Medical Officer and discussed with a Steering Committee on June 18^th, 2014. The steering committee is overseeing project activities of the WCCP and comes together on a quarterly basis. Members included district and community representatives. After the EmONC assessment results were presented and discussed at a meeting of the CHMT during their annual planning meeting on December 6^th, 2014.

Results

A total of 13 dispensaries, four health centres and one district hospital were included in the facility survey. In 2013, 2504 (21%) births took place in the district hospital and 2531 (22%) births in the four health centres. In dispensaries births ranged from 50 to 296 births per year, with a mean of 100 births. Assuming similar patterns in the facilities, which were not surveyed, births in dispensaries were estimated to cover a total of 2600 (22%) births. In total these facilities covered 7635 facility births, approximately 65% of the expected births.

None of the dispensaries complied with the guidelines to provide basic EmONC and were either not able to administer one or all of the essential drugs: anti-convulsants, uterotonics or antibiotics. None of the dispensaries had complete birth kits. Five facilities were able to locate all essential neonatal resuscitation equipment. An overview of availability of essential equipment and supplies is presented in Table 2.

Table 2 Facility survey: availability of supplies^a for EmOC in 2014

Results with regards to the EmONC indicators are presented in Table 3. Three health centres did provide all signal functions for basic EmONC except administration of MgSO4 to prevent or treat eclampsia (supply problems) and performing assisted vaginal deliveries. One health centre appeared to have more problems; no manual vacuum aspirations were performed, parenteral antibiotics were not administered and newborn resuscitation was not performed. The district hospital was able to provide comprehensive EmONC but did not fully qualify due to non-availability of blood transfusions and no use of the available vacuum extractor in the past three months. Lack of blood was a contributing factor in at least three out of 22 maternal deaths and a frequent indication for referral to the regional hospital in Mwanza. Although a vacuum extractor was available and functioning, observations during the facility visit and lack of documentation on performance of vacuum extraction indicated it unlikely to be used.

Table 3 Indicators, results and interpretations of EmONC indicators

Discussion

None of the facilities in Magu district was able to perform at the expected EmONC-level covering all signal functions. These findings are not surprising considering that the nation wide availability of basic and comprehensive EmONC facilities reduced between 2012 and 2015 [3, 18]. All facilities scored particularly poor on performance of assisted vaginal delivery. Poor clarity on who is qualified, lack of exposure and training and beliefs that higher cadres of health workers should perform such procedures contribute to almost district wide absence of performance of this signal function [1, 19]. Unavailability of facilities offering full basic and/or comprehensive EmONC is worrisome and contributes to poor outcomes, even after women have reached institutions. Previous studies in Tanzania reached similar conclusions [4, 19–22]. Negative in-facility experiences influence women’s perceptions of care and might influence future care seeking, leading to an increase in delay to seek care and preference for home birth [6].

The average facility distribution of 31 facilities per 300.000 population shows there is adequate coverage and plotting the institutions on a map shows good geographical distribution of facilities. Our estimation that 65% of births took place in health facilities is encouraging, considering the national average of skilled birth attendance of 50% [17]. However, health facilities appear underutilized as only 22% of women with expected major obstetric complications were treated in facilities and 1.6% of births were caesarean sections, similar to findings in other sub-Saharan countries [23].

Although our assessment did not include the full range of Essential Newborn Care it is important to include the ‘N’ in EmONC when conducting such assessments. Newborns have been left out of the global agenda for decades but are now increasingly being acknowledged through initiatives such as the ‘Every Newborn Action Plan’ [24]. Since fresh stillbirth and early neonatal death are usually caused by complications during childbirth [25], the ‘N’ is a vital part of EmONC assessments. Our assessment found low numbers of fresh stillbirths and early neonatal deaths assuming severe underreporting. To our surprise we did not find any documentation of neonatal deaths outside the district hospital. Monthly reporting books do not request this information, other than number of stillbirths and number of children born with Apgar score below 7 after 5 min. Difficulties in distinguishing between fresh stillbirths versus early neonatal deaths and overall poor reporting of perinatal deaths were challenges found in other studies as well [26].

Maternal deaths have been relatively stable in the district over the past years ranging from 17 to 22 registered maternal deaths [27]. Of the registered deaths in 2013, 81% occurred in the district hospital. Review of notification forms raised concerns about the quality of care [28]. Of all maternal deaths, 33% were classified due to indirect causes and could have been prevented with adequate attendance and quality of antenatal care. These findings highlight the importance of ensuring quality of service delivery across the continuum of care.

Although the EmONC indicators are widely used to assess and monitor EmONC performance, their usefulness to understand and improve the level of performance is debated [4, 26]. Additional assessment methods, beyond reliance on routinely available data, are suggested to be necessary to get a complete picture of the challenges in providing EmONC [23]. Our study shows that assessing supply levels of facilities at a given moment in time can help in gaining a sense of their ability to perform at least 6 of the 7 signal functions. This is in particular useful for facilities with low level of activity (e.g dispensaries) where assessing performance of signal functions over the past three months would not give a realistic idea of their ability to tackle emergencies due to low numbers of births.

Also for the higher-level facilities, inadequate availability of essential medicines and supplies for EmONC affects their ability to perform all signal functions and is a common problem in Tanzania [4, 29, 30]. It is important to note that this does not mean facilities and their staff are not capable or skilled to provide EmONC services. On the contrary, highly skilled health workers can be demoralized due to continuous shortages affecting work motivation and consequently quality of care [31, 32]. Assessments of EmONC should therefore increase attention to supply chain problems, including available procurement and distribution systems and accountability mechanisms.

The assessment captured other weaknesses in the system contributing to poor availability, accessibility and quality of EmONC, which are not captured in the assessment of indicators alone. Poor utilization of the partograph to monitor birth process is a contributor to poor quality of monitoring labour progress and poor outcomes. Although monitoring and management of labour using partograph is part of the signal functions for routine care [9] we argue its use is significant for quality basic EmONC and should therefore not be left aside in EmONC assessments. Insufficient training and limited health worker knowledge contribute to poor use of the partograph [33], however health facility staff reported that unavailability of stationary or funds to print partograph was a major contributing factor for their lack of use.

Similarly, availability of an ambulance for emergency transport depends on families contributing for fuel and cesarean sections could not always be performed at the district hospital due to lack of functioning generator or lack of available anesthesia equipment or blood for transfusion. Such poor organization of services and human resources indicate weaknesses in the system which show how relatively small problems, such as lack of access to stationaries, can prevent access to life saving services [34]. Preventing maternal and perinatal mortality requires a functioning health system, including adequate quality of care in its EmONC facilities. EmONC assessments should therefore be supplemented by mixed methods designs, including qualitative studies, which allow for contextual understanding of findings [26, 35].

Limitations

Our findings should be interpreted with caution. Data collection in dispensaries was restricted to half of the available dispensaries and do, therefore, not necessarily give a complete picture of the district. However, our findings showing inadequate provision of basic EmONC services at all levels, make it very unlikely other lower level facilities are fully functioning EmONC facilities. Facility assessment in dispensaries was done prior to receiving new supplies, which should be taken into account. However re-assessment one year later after new supplies were provided did not show much improvement

Although our findings are typical for (maternal) health systems in poor rural districts in sub-Sahara Africa, our assessment was restricted to institutions in Magu District and the findings cannot be extrapolated to other districts. Neighbouring, Busega district, has no District Hospital and therefore women often rely on services of Magu District Hospital. This may have influenced the data explaining the relatively high percentage of facility births. Recall bias could have influenced data quality negatively, since we asked in-charges of health centres and district hospital about functioning of their facility in the past three months.

Data quality was limited at all levels of care. The research team observed many problems with documentation and reporting. All delivery books had missing information regarding outcome of birth and whether mother and child went home alive. Also many irregularities and inconsistencies were found between registries within facilities. At dispensary level there were no records of referral or adverse outcomes, despite staff informing us that these do occur. Poor documenting and reporting has been observed in other settings, and despite our knowledge of poor data quality, much of the decision-making is based on these poor numbers [36]. It is easy to assume there is underreporting of complications and poor outcomes, but at the same time many instances where staff do provide high quality of care also go undocumented.

Conclusions

Although availability of health facilities expected to deliver EmONC are adequate in Magu District, none of the facilities perform all signal functions of BEmONC or CEmONC. Performance of signal functions is dependent on sufficient availability of supplies and resources as well as adequately trained and qualified staff, which can be assessed in line with EmONC indicators. In addition robust data collection systems are required for reliable assessments. Improving assessment approaches to account for local realities and contextual constraints will help to strive for improvements over time. Based on the reported results recommendations were formulated and discussed together with the Council Health Management Team. Importance of implementation of the recommendations and follow up according to the quality cycle was emphasized.