Skip to main content

Development of quality indicators for low-risk labor care provided by midwives using a RAND-modified Delphi method



In childbirth, most deliveries are low-risk, defined as spontaneous labor at full term without special high-risk facts or complications, especially in high-resource countries where maternal and perinatal mortality rates are very low. Indeed, the majority of mothers and infants have no serious conditions during labor. However, the quality of care provided is not assured, and performance may vary by birthing facility and provider. The overuse of technology in childbirth in some parts of the world is almost certainly based on assumptions like, “something can go wrong at any minute.” There is a need to assess the quality of care provided for mothers and infants in low-risk labor. We aimed to develop specific quality indicators for low-risk labor care provided primarily by midwives in Japan.


We used a RAND-modified Delphi method, which integrates evidence review with expert consensus development. The procedure comprises five steps: (1) literature review, including clinical practice guidelines, to extract and develop quality indicator candidates; (2) formation of a multidisciplinary panel; (3) independent panel ratings (Round 1); (4) panel meeting and independent panel ratings (Round 2); and (5) independent panel ratings (Round 3). The three independent panel ratings (Rounds 1–3) were held between July and December 2012.


The assembled multidisciplinary panel comprised eight clinicians (two pediatricians, three obstetricians, and three midwives) and three mothers who were nonclinicians. Evidentiary review extracted 166 key recommendations from 32 clinical practice guidelines, and 31 existing quality indicators were added. After excluding duplicate recommendations and quality indicators, the panel discussed 25 candidate indicators. Of these, 18 were adopted, one was modified, six were not adopted, and four were added during the meeting, respectively.


We established 23 quality indicators for low-risk labor care provided by midwives in labor units in Japan.

Peer Review reports


Almost all infants and mothers have high levels of well-being and no serious conditions during term labor. In Japan, care staff sometimes assume that “something can go wrong at any minute during childbirth”, which leads to overprotection against rare adverse events. Midwife-led continuity of care for low-risk labor offers important benefits for mothers and babies, and no adverse effects have been identified [1, 2]. Furthermore, there are regional variations in the numbers of obstetricians, midwives, nurses, and birthing facilities. As such, the quality of labor care is not assured, and performance might vary by facility and provider. However, few quality indicators (QIs) have been developed to measure the quality of care formally. The aim of this study was to develop indicators to measure the quality of low-risk labor care.

The United States Institute of Medicine defines quality as “the degree to which healthcare services for individuals and populations increase the likelihood of desired outcomes and are consistent with current professional knowledge” [3]. The measurement of healthcare quality has gradually diffused into many areas of practice [4]. However, quality assessment is rarely conducted in routine labor and birth, perhaps because most childbirths are normal, without any serious adverse events. Although some indicators exist, their validity is uncertain [5, 6]. Furthermore, there are few such indicators for low-risk labor [7,8,9]. To measure the quality of care for low-risk labor, the process of care should be examined [10,11,12].

The majority of deliveries occur following low-risk labor [13]. There is currently no universal definition of low-risk labor; however, it generally includes labor following pregnancies without specific high-risk factors or complications, as detailed in Table 1. We defined low-risk labor by considering three aspects: 1) pregnancy risk self-assessment score; 2) administrative criteria for the payment of medical service fees for high-risk pregnancy and labor administration in Japan; and 3) standards for subjects of low-risk labor care provided by midwives in the clinical practice guidelines (CPGs). When a pregnancy involved any of the items listed in these three categories, we defined it as “not low-risk labor.” Generally, in Japan, an obstetrician decides whether mothers at low risk are capable of giving birth in a midwifery ward. A pregnant female can then choose whether to give birth in an obstetrics ward or a midwifery ward [14,15,16].

Table 1 Definition of low-risk labora

In Japan, the numbers of obstetricians, midwives, and maternity facilities vary regionally, with lower numbers generally found in rural areas. In 2008, the Japanese Nursing Association established to promote an inpatient, midwife-led care system as part of the perinatal medical system. This system combines an outpatient department and an inpatient ward, both overseen primarily by midwives, in contrast to regular delivery wards, where obstetricians provide care [17]. Under this system, care is generally provided to mothers undergoing low-risk labor and birth from the beginning of labor until 1 week after birth. If necessary, emergency care is provided by obstetricians in the same hospital [18]. Although QIs of care are needed to evaluate performance and improve care or midwife competency, objective QIs have neither been proposed nor validated within this Japanese system.

In this study, we identified QIs that can be extracted from medical records and applied on site, allowing for visualization and quantification of care quality.

QIs should incorporate the unique aspects of each country’s healthcare system and sociocultural preferences. Accordingly, some countries have established their own QIs [7, 8, 19]. This study aimed to develop QIs for low-risk labor care provided for mothers and infants primarily by midwives in Japan.


Study design

We used a RAND-modified Delphi method (RAND/UCLA Appropriateness Method) [20, 21]. This consensus method has been widely used to develop QIs [22,23,24,25]. It comprises two steps: a systematic literature review followed by a face-to-face meeting with a multidisciplinary panel (Fig. 1). Thereby, this method enables the integration of scientific evidence with expert opinions [26]. Because de novo development of evidence-based QIs is very costly and time-consuming, methods using existing CPGs have gained interest as viable alternatives [25]. Thus, rather than searching for primary research articles, we retrieved existing and relevant CPGs and QIs from the literature.

Fig. 1

Overview of the literature review: review of the literature, guidelines, and quality indicators (QIs) extracted for generating QI candidates. N, number of extracted clinical guidelines or quality indicators; n, number of extracted recommendations. To avoid duplication, 25 QI candidates were assembled from 32 guidelines and 31 existing QIs

This study was approved by the Ethics Committee of Kyoto University Graduate School and Faculty of Medicine.

Step 1: Review evidence and generate indicator candidates

To generate QI candidates, we extracted existing recommendations from CPGs related to obstetric care for low-risk labor. In June–August 2012, we searched for data sources using the terms “pregnant women,” “mothers,” “infant,” “perinatal care,” “prenatal care,” “postnatal care,” “delivery,” “obstetric,” and “surgical procedure.” We searched for CPGs in eight databases provided by the following organizations: United States Agency for Healthcare Research and Quality (AHRQ) National Guideline Clearinghouse, Australian National Health and Medical Research Council, Canadian Medical Association, Guidelines International Network, Minds with the Japan Council for Quality Health Care, United Kingdom National Institute for Health and Clinical Excellence (NICE), United Kingdom Scottish Intercollegiate Guidelines Network, and the New Zealand Guidelines Group. Two QI databases (AHRQ National Quality Measures Clearinghouse and National Quality Forum) were used. We also searched medical literature databases, including MEDLINE, CINAHL, and ICHUSHI of the Japan Medical Abstract Society. Finally, we performed a manual search to identify literature that might be relevant to this study.

We searched literature published in English and Japanese between July 2007 and June 2012. We included sets of QI and CPGs for which the title or abstract included the keywords “guideline,” “practice guideline,” “clinical guideline,” “quality indicator,” “clinical indicator,” “performance measurement,” or “quality standard.” We excluded those including the keywords “16 years old and younger,” “40 years old and older,” “premature delivery,” “multiple births,” “breech presentation,” “pre-pregnancy obesity,” “pregnancy complication,” “obstetric history,” “abnormal pregnancy progress,” “infant congenital disease,” “diagnosis and treatment for infant disease,” “birth weight under 2000 g or over 4000 g,” “anesthesia,” “operation and examination procedure,” “28 or more weeks gestation,” “one week post-partum,” or “normal medical care not provided in in-hospital midwifery.” For CPG recommendations, we included graded recommendations expressed as “recommend for,” “recommend against,” and “suggest for.” We did not include recommendations of “weakly suggest,” as indicated by the Grading Recommendations, Assessment, Development, and Evaluations (GRADE) method [27]. We also considered the feasibility of data collection and measurability before inclusion.

Two researchers (KU and SO) extracted data independently. KU assembled the candidates, supervised by two experts: TN, an epidemiologist experienced in developing a variety of Japanese CPGs and QIs, and MK, a nursing instructor and midwife.

Step 2: Forming a multidisciplinary panel

We assembled a multidisciplinary panel comprising healthcare clinicians (obstetricians, pediatricians, and midwives), public health specialists, and mothers who were nonclinicians. We selected healthcare clinicians with 5 or more years of clinical experience in childbirth and newborn care. We included at least two members from each specialization to prevent disproportionate weight from being placed on particular perspectives. All panel members were required to have either worked or be interested in inpatient midwifery.

Several recently developed QIs in areas other than labor and childbirth have incorporated patient perspectives [28, 29]. We believe that inclusion of mothers’ viewpoints is important for low-risk labor issues. Therefore, we added mothers who were not health professionals to the panel. Potential panel candidates were recruited primarily from staff, students, and graduates of the Kyoto University School of Public Health. After explaining the study context and confirming panel members’ participation, we obtained written informed consent from all panel members. Consequently, the 11 members constituted the multidisciplinary panel.

Step 3: Independent panel ratings (round 1)

Consensus building involved three rounds of independent rating. During each round, panel members rated the appropriateness of each QI candidate on a 9-point scale, where 1 and 9 were “least suitable” and “most suitable,” respectively. In addition, panel members were given an opportunity to provide comments or suggest additional candidates.

For Round 1, a list of candidate QIs and a description of the rating method were mailed to the panel members. To facilitate decision making, the sources and relevant literature citations for each candidate QI were provided. These ratings were made independently, without interaction among panel members. Based on criteria from the U.S. National Quality Forum Measure Evaluation Criteria [30] and the American College of Cardiology/American Heart Association [31], we rated the appropriateness of each QI according to: 1) usefulness in improving outcomes for mothers and infants; 2) whether the measure is clinically relevant; 3) validity; 4) reliability; 5) feasibility of measure implementation; and 6) overall assessment of the candidate QI (Table 2).

Table 2 Sample rating questionnaire form

Step 4: Panel meeting and independent panel rating (round 2)

In Round 2, a face-to-face meeting moderated by KU and TN was held. Each member received a document showing the distribution of Round 1 ratings by all members. In this meeting, candidate QIs were discussed, and decisions regarding adoption were made. During the discussions about the QI candidates, each panel member rated them on a questionnaire. Additional candidate QIs were also proposed at this meeting.

Step 5: Independent panel rating (round 3)

In Round 3, the additional candidate QIs were evaluated using a second postal survey (conducted in the same manner as the first) to determine which QIs would be adopted (see below).

Statistical methods

QI candidates were adopted if the median individual “overall assessment” during Round 2 or 3 was greater than 7 and if three or fewer panel members rated it less than 3 [20].


From the literature review, we extracted 32 CPGs (166 recommendations) and 31 existing QIs (Fig. 1). We selected 16 guidelines from the AHRQ National Guideline Clearinghouse, 8 from NICE Guidance, 3 from the Canadian Medical Association, 2 from Minds with the Japan Council for Quality Health Care, 1 from the Australian National Health and Medical Research Council, 1 from the Guidelines International Network, and 1 obtained from our manual search. Twenty-five QI candidates pertaining to the Japanese healthcare system were developed from the CPGs and QIs.

All panel members agreed to participate in the study. All responded to the postal surveys, attended the meeting, and participated in the entire process. The consensus-development process was completed in December 2012.

Figure 2 shows the process of QI development. The Round 2 ratings resulted in 18 QIs selected from among the 25 QI candidates; 6 indicator candidates were not adopted. One indicator was modified and included as an additional candidate following the panel’s suggestion that it was needed for induced labor and to promote delivery (Indicator 9, Table 3). Four new QI candidates were introduced in the panel meeting (Indicators 2, 12, 14, and 23, Table 3). Five indicator candidates proposed at the meeting were ultimately adopted at Round 3. Consequently, 23 QIs were established (Table 3).

Fig. 2

Development process of quality indicators using a RAND-modified Delphi method for low-risk labor care. This flow diagram illustrates each stage within the process of quality indicator development. The top white box indicates the identification of initial quality indicator candidates: Steps 1–2. The middle gray square indicates the first rating of quality indicator candidates: Steps 3–4. The lower dark gray square indicates the final stage, in which the panel rated additional quality indicator candidates: Step 5

Table 3 Quality indicators for low-risk labor care provided primarily by midwives in Japan


We established 23 QIs for low-risk labor care provided primarily by midwives using a RAND-modified Delphi method based on CPGs and existing QIs. The main purpose of using QIs for low-risk labor care is to ensure effective quality improvement of providers’ performance.

Our study differs distinctly from previous studies, which have reported QIs for low-risk labor or normal birth [7,8,9] but did not report transparent procedures for how they created QI candidates following their literature searches. Only one study reported evidence sources, including guidelines. Furthermore, these studies considered both home and hospital birth. We focused on low-risk labor care provided primarily by midwives in hospital.

Panel composition may affect the final consensus on QIs. While incorporating patient views is considered important, patient participation in QI development has been limited [24, 29, 32]. Our panel included three mothers with birth experience, so the QIs we developed may more readily reflect insight and ideas from mothers concerning the quality of care in inpatient midwifery wards.

Prior attempts at QI development have widely used the RAND-modified Delphi method (i.e., the RAND/UCLA appropriateness method) [22, 25, 33], whereby a literature review is conducted and a consensus is developed. Methods to extract guidelines during the literature review have attracted much attention, because the traditional review method using primary research articles is time-consuming. However, such methods have only recently been developed, and thus, they have yet to be standardized [24, 25]. Processes to develop new QIs based on existing CPGs have been described in various forms and therefore may include a degree of arbitrariness. Thus, to increase the reproducibility of the present study, two of the authors (KU and SO) independently extracted evidence under the supervision of another author (TN). When developing guideline-based QIs, objective and transparent processes must be executed, even if some of their aspects may remain arbitrary. As in traditional systematic reviews, QI developers should describe their methods and reasons for selecting recommendations from existing CPGs and clarify which databases, selection criteria, and search keywords were used. One challenge we faced was determining the grades for CPG recommendations. The present study used all CPG recommendations except “suggest not to do” or “weak recommendation against.” These expressions are sometimes difficult to use when interpreting whether a recommendation should be implemented [27].

Strengths and limitations

Our results should be considered in light of some limitations. First, panel composition may influence consensus development and outcomes at several points. If a member has expertise in a certain area and makes strong assertions, the consensus may become biased. To address this concern, we had two moderators (KY and TN) and two moderator assistants (SO and MK) to prevent one-sided discussions. At the beginning of the meeting, the moderators explained the rules for discussion (e.g., “respect the end time of the meeting”). In addition, one assistant managed the meeting time, while the other confirmed each panel member’s degree of participation, reporting observations to the moderators. Neither the moderators nor the assistants voted in the consensus development.

Second, our electronic search did not identify some CPGs, e.g., the WHO guidelines “Making Pregnancy Safer WHO 2009” and “Care in Normal Birth WHO 1996”. The former mainly included indicators to assess structure of facilities, however, we focused on the process and outcome indicators for each individual practice. The latter was not relevant considering our planned publication period and the main contents are covered by the following CPGs that we included. Although including these CPGs would not seriously change the present results, search strategy and selection criteria of existing CPGs may need refinement when updating the current QIs.

Third, the validity of our consensus method may be influenced by the representativeness of the panel members. Complete representativeness is probably not possible, but transparency in how the panel is assembled and of the whole consensus process is critical for readers to assess the validity of the outputs. We have detailed the selection criteria, recruitment process, and panel characteristics. The panel included only one member with experience in CPG development, whereas guideline developers and representatives of academic societies might be better qualified for QI development than others, because they are expected to be well informed in these areas. When relevant academic societies develop QIs based on their own clinical practice guidelines, our proposed QIs might guide their activities. The present QI sets need practical validation to confirm their clinical relevance [26]. A study to test their validity is currently in progress.

Fourth, patients’ perspectives may not have been adequately included. The 23 QIs developed in this study did not include items assessing such factors as patient experience, patient–provider relationship, and its empowerment of mothers. This may be a general limitation of QI development based on guidelines. Although patient representatives were included in the panel, patient experience and the patient–provider relationship were not reflected in the QI items.

Ideally, QI development should include the patient’s viewpoint, but this is rare in reality. Therefore, a strength of our study was the inclusion of individuals who had experienced labor but were not healthcare professionals [24, 34]. There is a danger that nonclinicians may have difficulty rating indicator candidates and joining the discussion. To address this, we provided additional information to explain specialized terms pertaining to the list of QI candidates. We also carefully addressed their questions by mail or telephone throughout the entire process. As the three panel members who were not healthcare professionals were healthcare researchers, there may have been fewer barriers to their participation in the consensus process than that of a general layperson. It was found that they contributed sufficiently to the discussion.


Using a RAND-modified Delphi method incorporating CPGs and existing QIs, we established 23 QIs for low-risk labor care provided primarily by midwives. These QIs can be used to assess and improve practices for low-risk labor managed by midwives. They should also initiate discussion of QIs among relevant health professionals and societies.



Agency for Healthcare Research and Quality


Clinical practice guideline


National Institute for Health and Clinical Excellence


Quality indicator


  1. 1.

    Sandall J, Soltani H, Gates S, Shennan A, Devane D. Midwife-led continuity models versus other models of care for childbearing women. Cochrane Database Syst Rev. 2016;4:Cd004667.

  2. 2.

    Sydsjo G, Blomberg M, Palmquist S, Angerbjorn L, Bladh M, Josefsson A. Effects of continuous midwifery labour support for women with severe fear of childbirth. BMC Pregnancy Childbirth. 2015;15:115. 10.1186/s12884-12015-10548-12886.

    Article  PubMed  PubMed Central  Google Scholar 

  3. 3.

    Institute of Medicine Committee to Design a Strategy for Quality, Review Assurance in, Medicare. Washington (DC): National Academies Press (US); 1990. 

  4. 4.

    Pettker CM. Quality in obstetrics: the search continues. Am J Obstet Gynecol. 2014;211(2):85–6. 10.1016/j.ajog.2014.1004.1019.

    Article  PubMed  Google Scholar 

  5. 5.

    Howell EA, Zeitlin J, Hebert PL, Balbierz A, Egorova N. Association between hospital-level obstetric quality indicators and maternal and neonatal morbidity. JAMA. 2014;312(15):1531.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  6. 6.

    Grobman WA, Bailit JL, Rice MM, Wapner RJ, Varner MW, Thorp JM, Jr., Leveno KJ, Caritis SN, Iams JD, Tita AT et al. Can differences in obstetric outcomes be explained by differences in the care provided? The MFMU Network APEX study. Am J Obstet Gynecol. 2014;211(2):147.e141-147.e116.

  7. 7.

    Sandin Bojo AK, Hall-Lord ML, Axelsson O, Uden G, Wilde Larsson B. Midwifery care: development of an instrument to measure quality based on the World Health Organization's classification of care in normal birth. J Clin Nurs. 2004;13(1):75–83.

    Article  PubMed  Google Scholar 

  8. 8.

    Murphy PA, Fullerton JT. Development of the optimality index as a new approach to evaluating outcomes of maternity care. J Obstet Gynecol Neonatal Nurs. 2006;35(6):770–8.

    Article  PubMed  Google Scholar 

  9. 9.

    de Bruin-Kooistra M, Amelink-Verburg MP, Buitendijk SE, Westert GP. Finding the right indicators for assessing quality midwifery care. Int J Qual Health Care. 2012;24(3):301–310.

    Article  PubMed  Google Scholar 

  10. 10.

    Wei S, Wo BL, Qi HP, Xu H, Luo ZC, Roy C, Fraser WD. Early amniotomy and early oxytocin for prevention of, or therapy for, delay in first stage spontaneous labour compared with routine care. Cochrane Database Syst Rev. 2013;8:CD006794.

    Google Scholar 

  11. 11.

    Alfirevic Z, Devane D, Gyte GM, Cuthbert A. Continuous cardiotocography (CTG) as a form of electronic fetal monitoring (EFM) for fetal assessment during labour. Cochrane Database Syst Rev. 2017;2:Cd006066.

  12. 12.

    Smyth RM, Markham C, Dowswell T. Amniotomy for shortening spontaneous labour. Cochrane Database Syst Rev. 2013;6:CD006167. 10.1002/14651858.CD14006167.pub14651854.

    Google Scholar 

  13. 13.

    Vital statistics of Japan 2015. National Statistics Center: Stactics Japan: Ministry of Health, Labour and Welfare. 2016. []. Accsessed 20 Aug 2017. Japanese.

  14. 14.

    Nakabayashi M. Study on safety steps in obstetrics department. Health Labour Science Research Grant Research on Health Technology Assessment. The summary and sharing documentation in 2004. 2005: p1-16.

  15. 15.

    Social insurance ABC for obstetricians: Japan Society of Obstetrics and Gynecology; 2011. p81-87. Japanese.

  16. 16.

    Practice Guideline for Midwives in Japan Japanese Midwives Association 2014. []. Accsessed 7 July 2017. Japanese.

  17. 17.

    Japanese Nursing Association In-Hospital Midwife-Led Care System. Japanese Nursing Association.  []. Accsessed 4 July 2017.

  18. 18.

    Hisanori M. Explanation of guideline for obstetrical practice in Japan: a revised edition CQ414 what are low risk pregnancy and labor provided primarily by midwives ? Acta Obstet Gynaecol Jpn. 2012;64(1):33–4.

    Google Scholar 

  19. 19.

    Kesmodel US, Jolving LR. Measuring and improving quality in obstetrics--the implementation of national indicators in Denmark. Acta obstetricia et gynecologica Scandinavica. 2011;90(4):295–304.

    Article  PubMed  Google Scholar 

  20. 20.

    Kathryn Fitch SJB, Maria Dolores Aguilar, Bernard Burnand, Juan Ramon LaCalle, Pablo Lazaro, Mirjam van het Loo, Joseph McDonnell, Janneke Vader, James P. Kahan: The RAND/UCLA appropriateness method user's manual. 2001:126.

  21. 21.

    Grol R, Baker R, Moss F: Quality improvement research: understanding the science of change in health care. Qual Saf Health Care 2002, 11(2):p6-28.

  22. 22.

    Campbell SM, Hann M, Hacker J, Durie A, Thapar A, Roland MO. Quality assessment for three common conditions in primary care: validity and reliability of review criteria developed by expert panels for angina, asthma and type 2 diabetes. Qual Saf Health Care. 2002;11(2):125–30.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  23. 23.

    Higashi T. Lessons learned in the development of process quality indicators for cancer care in Japan. Biopsychosoc Med. 2010;4:14.

    Article  PubMed  PubMed Central  Google Scholar 

  24. 24.

    Kotter T, Blozik E, Scherer M. Methods for the guideline-based development of quality indicators--a systematic review. Implement Sci. 2012;7:21.

    Article  PubMed  PubMed Central  Google Scholar 

  25. 25.

    Blozik E, Nothacker M, Bunk T, Szecsenyi J, Ollenschlager G, Scherer M. Simultaneous development of guidelines and quality indicators -- how do guideline groups act? A worldwide survey. Int J Health Care Qual Assur. 2012;25(8):712–29.

    Article  PubMed  Google Scholar 

  26. 26.

    Campbell SM, Braspenning J, Hutchinson A, Marshall MN. Research methods used in developing and applying quality indicators in primary care. BMJ. 2003;326(7393):816–9.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  27. 27.

    Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schunemann HJ. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ (Clinical research ed). 2008;336(7650):924–926.

    Article  Google Scholar 

  28. 28.

    Kerr EA, Hayward RA. Patient-centered performance management: enhancing value for patients and health care systems. JAMA. 2013;310(2):137–8. 10.1001/jama.2013.6828.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  29. 29.

    den Breejen EME, Nelen WL, Schol SF, Kremer JA, Hermens RP. Development of guideline-based indicators for patient-centredness in fertility care: what patients add. Human reproduction (Oxford, England). 2013;28(4):987-996.

  30. 30.

    National Quality Forum Measure Evaluation Criteria and Guidance for Evaluating Measures for Endorsement National Quality Forum. 2016. []. Accsessed 20 Aug 2017. 

  31. 31.

    Spertus JA, Eagle KA, Krumholz HM, Mitchell KR, Normand SL. American College of Cardiology and American Heart Association methodology for the selection and creation of performance measures for quantifying the quality of cardiovascular care. Circulation. 2005;111(13):1703–12.

    Article  PubMed  Google Scholar 

  32. 32.

    Stelfox HT, Boyd JM, Straus SE, Gagliardi AR. Developing a patient and family-centred approach for measuring the quality of injury care: a study protocol. BMC Health Serv Res. 2013;13:31.

    Article  PubMed  PubMed Central  Google Scholar 

  33. 33.

    Boulkedid R, Sibony O, Goffinet F, Fauconnier A, Branger B, Alberti C. Quality indicators for continuous monitoring to improve maternal and infant health in maternity departments: a modified Delphi survey of an international multidisciplinary panel. PLoS One. 2013;8(4):e60663.

  34. 34.

    Kotter T, Schaefer FA, Scherer M, Blozik E. Involving patients in quality indicator development - a systematic review. Patient Prefer Adherence. 2013;7:259-268.

Download references


We thank the Research Group collaborators and the panel members for their work in developing the quality indicators. We would also like to acknowledge Hiroyuki Kitajima, MD, PhD, Department of Neonatology, Osaka Medical Center and Research Institute for Maternal and Child Health; Isao Kuwahara, MD, Kuwahara Children Clinic; Miho Egawa, MD, Department of Gynecology and Obstetrics, Graduate School of Medicine, Kyoto.

University; Fumitomo Nishimura, MD, Department of Gynecology and Obstetrics, Graduate School of Medicine, Kyoto University; Yoshihito Goto, MD, MPH, the School of Public Health Kyoto University; Yoshimi Miyaki, Vice chief of Midwife, Perinatal Center Medical Birth Center Hamamatsu Medical Center; Kazue Watanabe, MW, Saiseikai Yokohamashi Tobu Hospital; Makiko Suga, Faculty of Business Administration, Tezukayama University; Miyamoto Keiko, Department of Medical Communication Kyoto University School of Public Health; Satoko Zenitani, Center for Cancer Control and Information Services The National Cancer Center; and Yoko Chiba, School of Public Health Kyoto University, School of Human Health Science, for their valuable insight and comments.


This study was conducted with the support of grants-in-aid from the Ministry of Health, Labour and Welfare, Japan, as follows: (i) Research on development of clinical practice guidelines based on systematic reviews and promotion of EBM in clinical practice (2012–2013), project number 201325053A; and (ii) Research on the subjects of clinical practice guidelines to realize socially accountable health care (2014–2015), project number 201424051A. We also received (iii) a grant-in-aid from the Ministry of Education, Culture, Sports, Science and Technology, Japan, as follows: Research on the applicability of quality indicators for low-risk labor care provided by midwives (2013–2015), project number 26870745. The funding agency had no role in study design, collection, analysis, or interpretation of data; writing of the manuscript; or the decision to submit the manuscript for publication.

Availability of data and materials

The datasets used or analyzed during the current study are available from the corresponding author on reasonable request.

Author information




KU, SO, and TN planned the study. KU, SO, MK, and TN prepared and carried out the RAND-modified Delphi procedure. KU, SO, MK, and TN chaired the panel meeting. KU and SO conducted the data analyses. KU and SO prepared the manuscript. MK and TN critically commented on the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Kayo Ueda.

Ethics declarations

Ethics approval and consent to participate

We obtained informed consent from all panel members.

This study was approved by the Ethics Committee of Kyoto University Graduate School and Faculty of Medicine (No 994).

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ueda, K., Ohtera, S., Kaso, M. et al. Development of quality indicators for low-risk labor care provided by midwives using a RAND-modified Delphi method. BMC Pregnancy Childbirth 17, 315 (2017).

Download citation


  • Low-risk labor
  • Quality indicator
  • Clinical practice guidelines
  • RAND-modified Delphi method