Identification and prioritization of population health indices in Iranian population, a qualitative study

Abstract

Introduction

A population health index is a combination of two or more health indicators, used to evaluate different dimensions of health in a population. While various population health indices have been developed globally, there remains a gap in systematically identifying and prioritizing such indices tailored to the Iranian context. This study aims to identify and prioritize population health indices relevant to the Iranian context using expert consensus through the Delphi method.

Methodology

A multi-methodology with two steps was employed in this study. Firstly, a comprehensive list of existing composite health indicators was identified by reviewing the literature. In the second stage, the review results were used as input for a web-based Delphi survey involving 22 experts across two rounds. After two rounds, a list of indices that gained at least 70% of consensus and a mean score of 2.5 out of 5 was prepared.

Results

In the initial phase of the study, a refined set of 21 indices emerged. Participation rates for the first and second rounds were 72% and 68%, respectively. A total of 21 indices were assessed, and after refining them based on expert feedback, the following five indices were determined to have the highest priority after two rounds: Disability-Adjusted Life Year (4.23 ± 0.32), UHC Service Coverage index (3.96 ± 0.25), Sustainable Development Goals index (3.78 ± 0.31), Human Development Index (3.58 ± 0.36), Quality-Adjusted Life Year (3.58 ± 0.37). The Inter-rater reliability test found significant absolute agreement among experts in the second round (ICC: 0.970, 95% CI: 0.931–0.989).

Conclusion

This study’s contribution lies in offering a compilation of composite health indicators, which can guide forthcoming research on the health measurement of the Iranian population. Furthermore, the study underscores the value of involving expert professionals and soliciting diverse perspectives in selecting health indices.

Introduction

Achieving health for all is a global goal, requiring valid tools to assess and monitor health status and inequalities within populations [1,2,3]. Health indicators serve this purpose by providing measurable constructs that summarize complex health phenomena and inform public health decisions [4]. A population health indicator provides a population-level summary of a particular health characteristic [5].

Considering health as a complex and multidimensional phenomenon, as defined by the World Health Organization (WHO) [6], a large number of indicators from various aspects are needed to evaluate health [7]. Interpreting this amount of information requires a lot of effort. Combining indicators into a composite health indicator or a health index is one way to overcome this complexity [7, 8]. According to the definition of the Organization for Economic Co-operation and Development (OECD), individual indicators are combined into a single index to create a composite indicator based on a multidimensional concept to be measured [9].

The Iranian context presents unique health challenges that are not adequately captured by existing global health indices. For instance, Iran faces a high prevalence of non-communicable diseases (NCDs), and the mortality from NCDs in Iran has steadily increased from 50% in 1990 to 82% in 2017 [10]. Additionally, Iran’s healthcare system operates under the constraints of international sanctions, which have led to shortages of essential medicines and medical supplies [11]. Therefore, there is a need to develop health indices tailored to Iran’s specific health context to effectively monitor and address the nation’s health priorities. Although previous studies have explored population health indicators in various contexts, research on their systematic selection and prioritization in Iran remains limited. Given Iran’s unique epidemiological landscape and health system constraints, such as the rising burden of non-communicable diseases and the impact of international sanctions, a tailored framework for selecting relevant indices is essential [12, 13].

Methods for choosing indices include systematic (participatory) and nonsystematic approaches [14]. Although nonsystematic methods rely on data availability, in systematic approaches the selection is directly based on expert opinions. In a systematic approach, the different stakeholders’ involvement is crucial [14, 15]. These individuals should be familiar with the monitoring processes at the national, regional, and local levels [16]. Systematic methods facilitate decision-making in cases the evidence is just insufficient or controversial [17]. Among these methods, the Delphi technique has been widely used for selecting healthcare indicators [14, 17].

This study conducted with two following objectives: (a) to identify and describe health indices used to monitor population health in Iranian population, and (b) to prioritize the identified population indices through the Delphi method based on expert opinion.

Research methodology

To select relevant health indices for evaluating the Iranian population’s health from the perspective of experts and stakeholders, this study employed a two-stage methodology combining a scoping literature review and a Delphi consensus process.

Stage 1: initial identification of indices

Similar studies identified

During the literature review, we identified similar studies worldwide to inform our own methodology and scope. For example, in 2019, Ashraf et al. conducted a scoping review to identify and evaluate population health indices globally, examining the methods used in their development [18]. Their study identified 27 indices, covering overall health outcomes, disease outcomes, and health resource allocation for priority subgroups. The review also highlighted the lack of comprehensive frameworks in the development of some indices, emphasizing the need for systematic and rigorous methods.

Additionally, Kaltenthaler et al. conducted a systematic review of population-based health indexes in Europe and North America [4]. Their study found considerable variation in the indicators used, the geographical levels of aggregation, and the methods of combining the indicators. The study also pointed out that many indices lacked proper validation, which is crucial for ensuring their utility in health monitoring.

Based on this foundation, we conducted a scoping literature review to systematically identify composite health indicators used globally to measure population health.

Databases searched

A scoping literature review for articles published until 2022 was conducted to identify existing composite health indicators used worldwide to measure population health. The searches were performed in the following databases: PubMed, Web of Science, Scopus, and Cochrane. The search strategy used included terms like “health index*” OR “health indices” OR “composite health indicator*” in combination with “population*.” Additionally, we investigated national health datasets relevant to population health assessments such as the Iranian Maternal and Neonatal Network (IMaN Net), the National Survey of Risk Factors of Non-Communicable Diseases (STEPS), and Sib, as well as international websites such as the World Health Organization, World Bank, and OECD to identify indices that are currently being measured.

Inclusion and exclusion criteria

Given the limited number of available population health indices, we aimed to include as many relevant indices as possible to allow expert opinions to determine which should remain in the final list. The inclusion criteria were based on the relevance of the indices. Indices that had not been reported in the last ten years or those that were duplicates were excluded. In cases where two composite indices measured almost similar data, we included the more comprehensive one approved by international health organizations.

Stage 2: Web-based delphi

In the second stage, the results of the literature review were used as an input for a web-based survey. To engage a wide range of experts with different backgrounds and geographical locations, we decided to use the Delphi method. The Delphi method is a structured process that involves a panel of experts providing their opinions on a particular topic [19].

Panel selection

A purposive sample of experts and stakeholders was selected from those working in Medical Science Universities in Tehran. Our inclusion criteria were being over 35 years, having more than five years of professional experience, being familiar with the health indices, and being willing to participate in the study.

Experts for the Delphi panel were initially suggested by our research team, which includes professors from the fields of epidemiology, public health, and community medicine. We employed a snowball sampling method, where initial experts recommended additional qualified individuals. The selected experts were drawn from various Medical Science Universities in Tehran, including Iran University of Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran University of Medical Sciences, Baghiyatollah University of Medical Sciences, and AJA University of Medical Sciences (Artesh), all of which offer health-related disciplines. Additionally, experts from the Ministry of Health and Education were included. To ensure a diverse and representative sample, we selected individuals from a range of healthcare and academic fields. After confirming that each expert met the inclusion criteria, 22 experts were contacted by phone, informed about the study, and invited to participate.

Number of rounds

A two-round Delphi process was initially planned; however, additional rounds were considered if inter-rater reliability fell below 0.7.

Consensus definition

The concept of consensus varied during the rounds. In the initial round, consensus was defined as receiving at least 70% agreement and a mean score of at least 2.5 out of 5. However, in subsequent rounds, we only used the mean score criteria.

Delphi process

We conducted an online survey due to the COVID-19 restrictions.

Round 1

A table with all the listed indices, the definition of each index, and the index components were placed in an online questionnaire. The link was emailed to all participants with a request to respond within two weeks. The questionnaire included demographic information (name, email address, age, academic degree, job title and professional experience), and then a brief outline of the research process and purposes explained. The main part of the questionnaire consisted of 22 questions, including 21 questions for all 21 indices and one at the end for the experts’ suggestions. All questions were designed using the same pattern to facilitate the completion of the survey (The translated version of the questionnaire is available as supplementary material).

After providing required information about each index, they were asked to answer the following questions: (1) Do you agree that this index should be measured in Iran? (2) If yes, please rate this index on a 5-point Likert scale based on the following statements: Importance, explicitness, measurability, comprehensiveness and ease of monitoring. (1 = lowest score, 5 = highest score). At the end of the questionnaire, they were requested to name health indices that they thought should be measured but are not listed on the table.

Round 2

Responders who took part in the first round received an overview of the results (Table 2), which were the percentage of agreement with each index, as well as the score each index received at the prioritization stage. In this round, the same criteria were used to grade the indices that had more than 70% agreement in the previous round and a mean score of at least 2.5 out of 5. The participants received a link to the questionnaire, with a request to respond within two weeks. In both rounds, a reminder was sent two weeks after the first mailing. In this round, the only measure that demonstrated consensus was receiving a mean score of at least 2.5 out of 5 based on the Delphi methodology [20]. was selected based on established practice in Delphi studies, where a mean score ≥ 2.5 on a 5-point Likert scale is considered to reflect moderate agreement and practical relevance [21]. In another example, a study on the key items of the neurological examination used a threshold of at least an average 2.5 rating out of 4 for both feasibility and acceptability to determine inclusion in the consensus process [22]. This approach balances inclusiveness with rigor and has been used in similar health-related Delphi consensus processes.

Table 1 Population health indices extracted in first step

Statistical analysis

The data collected in each round was analyzed using descriptive statistics. To facilitate the interpretation of the results, the overall score for each index was considered as the average of its five points. Accordingly, an index that scored five based on all five criteria would get an overall score of five. The mean and standard deviation, as well as median, interquartile range, and range were calculated for each index. The Intraclass Correlation Coefficient (ICC) test in SPSS was used to measure the inter-rater reliability of the second round’s results. We used the Absolute Agreement test to determine the level of consensus required to determine whether further rounds were necessary. Our subjects were the means of the overall scores of indices. Finally, ICC was provided as average measures with a 95% confidence interval.

Ethical considerations

This study was approved by the Research Ethics Committee of the Iran University of Medical Sciences. (Ethical code: IR.IUMS.FMD.REC.1399.454). Informed consent was obtained from all participants, and their anonymity was maintained throughout the study. Participants were also informed that their opinions would be confidential.

Results

In the initial phase of the study, an extensive review of national and international sources was conducted to identify a comprehensive set of health indices. Following consultation with experts, redundant and less relevant indices were excluded. Consequently, a refined set of 21 indices emerged, which have been presented in Table 1, along with their corresponding explanations.

The electronic forum employed in this study consisted of a set of indices that were distributed to 22 experts who met the predetermined inclusion criteria. Following three reminders, a total of 16 experts responded, yielding a commendable response rate of 72% for the first round. However, five out of the 16 experts did not take part in the next round, decreasing the participation rate in the second round to 11 experts (68%). These five experts dropped out of the study and did not respond to follow-up communication, probably due to their professional commitments and time constraints.

With the expert panel established and participation finalized, the following section outlines the key findings from the Delphi process, including the selection and prioritization of population health indices. Among the participating experts, 62.5% were male. Half of the experts were affiliated with Iran University of Medical Sciences, an academic institution located in Tehran. Notably, a significant proportion (75%) of the participants had over 15 years of experience in various managerial positions within the public health system. These positions included Chief of Health Network, Health Centers and Hospitals, and Vice President of Medical University.

After the specified deadlines, the results were collected and tabulated. The outcomes of the first and second Delphi rounds are presented in Tables 2 and 3, respectively. The first column in Table 2 denotes the percentage of agreement associated with each index in the first round, while other columns showcase the descriptive of overall scores obtained during the Delphi process.

Table 2 Rating results from the first round of the Delphi

Table 3 Rating results from the second round of the Delphi

As outlined in the methods section, only indices that garnered agreement from more than 70% of the experts and achieved a mean score of at least 2.5 out of 5, were considered for inclusion in second-round rating. Consequently, seven indices - Multimorbidity Index, Euro-Healthy Population Index, Multiple Deprivation Index, Health Utility Index, Handicap -Free Life Expectancy index, Hologic Global Women’s Health Index, and Disability-Free Life Expectancy (DFLE) index- were excluded because they did not meet these criteria.

In the second round of the Delphi, the 14 remaining indices underwent reevaluation to be rated again. During this round, two further indices, namely Life Expectancy Free of Avoidable Mortality (LEFAM) and Health and Activity Limitation Index (HALex), were excluded as they did not receive a mean score of at least 2.5 out of five.

On the other hand, six indices were unanimously agreed upon by all participants in the first round. These indices include Disability-Adjusted Life Year (DALY), UHC Service Coverage Index, Sustainable Development Goals Index, Human Development Index (HDI), Quality-adjusted life year (QALY), and Healthy Lifestyle Index (HLI).

Based on the expert opinions presented in Table 3, the following five indices were determined to have the highest priority after two rounds: Disability-Adjusted Life Year (4.23 ± 0.32), UHC Service Coverage index (3.96 ± 0.25), Sustainable Development Goals index (3.78 ± 0.31), Human Development Index (3.58 ± 0.36), Quality-Adjusted Life Year (3.58 ± 0.37).

The Inter-rater reliability test found significant absolute agreement among experts in the second round (ICC: 0.970, 95% CI: 0.931–0.989). Since this test yielded a substantial agreement surpassing our predefined threshold of 0.7, further rounds were deemed unnecessary, and the study concluded after these two rounds. Although at the end of the questionnaire of each round, the experts were given the opportunity to suggest any additional health indices they believed should be included, no new indices were proposed. Figure 1 details the iterative Delphi process, highlighting the inclusion criteria and expert consensus formation over two rounds.

Fig. 1

Delphi process flowchart

Table 4 presents the final list of suggested indices for evaluating Iranian population health. These indices have been categorized based on previous studies, including the recommended list by the World Health Organization (WHO). In 2018, the WHO published a comprehensive list of 100 primary health indicators, grouped into four categories. The first category consists of indicators related to health status, such as mortality rates by age and sex, as well as life expectancy. The second category includes indicators associated with risk factors, such as nutrition, environment, and behavior. The third category encompasses indicators related to service coverage, such as vaccination rates and the prevention of communicable diseases. Finally, the fourth category focuses on indicators associated with health systems, such as the density and distribution of healthcare facilities, as well as information systems [23]. Koohpayehzadeh’s study also followed a similar classification approach [12].

Table 4 Final list of the suggested population health indices in Iranian population

Drawing on these established frameworks, the composite indicators in our study have been classified accordingly. The sustainable development goals index has been placed within the “other indices” group, as it spans multiple categories. All participating experts endorsed the final classification of indices.

Discussion

This study presents an overview of composite health indicators used in Iran and prioritizes them based on expert opinions through the Delphi method. This method enabled us to engage a diverse group of experts with extensive knowledge and experience in the field of public health and healthcare. In our study, the consensus seemed to be reached after a two-round Delphi process. Previous studies suggest that a two-round Delphi process balances response accuracy with expert participation rates [24, 25]. Also, evidence suggests that more rounds may result in lower response rates [24].

Our review did not consider all the available health assessment indices in Iran, as these indices did not meet our inclusion criteria. Nevertheless, the composite indicators we presented still do not offer a comprehensive view of the overall health of the population. In fact, our analysis revealed that these composite indicators often overlook numerous factors influencing health. There are exceptions, such as the Sustainable Development Goals Index and Universal Health Coverage Index, which encompass a broad spectrum of health determinants.

The indicators cover various aspects, including population health, mortality, women’s and children’s health, quality of life, and lifestyle. The study identifies five indices with the highest priority: disability-adjusted life years (DALY), UHC service coverage index, sustainable development goals index, human development index, and QALY. While similar studies regarding health indicators exist, this research stands out as one of the first to prioritize composite health indicators in Iran, focusing on the overall health status of the population.

Comparing our findings with other studies, we observe some similarities. For example, a study by Gonzalez-Bautista et al., which focused on health system responsiveness for older adults, also emphasized distal outcomes such as mortality and life expectancy, which align with our prioritization of Disability-Adjusted Life Years (DALY) and Quality-Adjusted Life Years (QALY). Their study also identified functional assessments and poverty as high-priority indicators, which differs from our broader focus on indices like Universal Health Coverage (UHC) and the Sustainable Development Goals (SDGs) Index [26]. In another study, Ashraf et al. (2019) conducted a comprehensive review of 27 population health indices. Unlike our study, which focuses on prioritizing a set of indices based on expert opinion, Ashraf et al.‘s research provides a broader view of the development and application of health indices globally. Their review identifies a lack of indices that address specific health topics or subpopulations, an area where our study aims to provide more tailored insights, such as indices related to maternal health [18].

The results from other studies focusing on individual health indicators align well with our findings. For instance, in a qualitative study conducted by Koohpayezadeh et al., experts’ opinions were utilized to prioritize health indicators for achieving universal health coverage [12]. This study highlighted maternal mortality, vaccination of children under five years of age, and life expectancy at birth as the most important indicators, reflecting a similar emphasis on maternal and child health aspects. Likewise, a study carried out by Haghdoost et al. aimed to assess major health indicators for monitoring the transformation of the healthcare system [27]. This study identified life expectancy at birth, neonatal mortality, and maternal mortality as the most critical indicators, mirroring our findings in emphasizing maternal and child health. The convergence of these results across studies can be attributed to the common thread of the DALY concept.

Disability-Adjusted life years (DALY): The findings of our study revealed that the highest-priority composite health indicator for collection is the DALY. DALY, a comprehensive measure that combines years of life lost due to premature death and years lived with disability, provides a holistic perspective on health burden [28,29,30]. This definition inherently includes indicators that capture maternal and child health aspects, as well as mortality rates. This index can serve as a crucial tool for identifying high-burden diseases that require urgent intervention. By integrating DALY into health policy planning, Iran could prioritize interventions that reduce the overall burden of disease, such as focusing on non-communicable diseases (NCDs) and improving preventive care to reduce mortality and disability rates.

UHC service coverage index indicates that all people should have access to health services in all aspects of health including treatment, rehabilitation, and prevention, without pressure and financial burden [31]. UHC is a cornerstone of the Sustainable Development Goals (SDG 3), aiming to ensure that all individuals receive necessary healthcare without financial hardship. WHO’s UHC Cube Model emphasizes three key dimensions: service coverage, financial protection, and population coverage, all of which must progress simultaneously to achieve equitable health outcomes [32]. The UHC cube, as introduced by the WHO, comprises three key dimensions: the extent of direct cost coverage, the range of services included, and the encompassed population. It is essential for all three axes to advance in harmony. Any imbalanced growth among these dimensions could result in adverse outcomes and inequity [33, 34]. Currently, studies have been conducted in Iran on the extent and universality of the use of this index [35]. In the study of Haqdoost et al., indicators of infant mortality, maternal mortality, as well as economic factors such as the percentage of out-of-pocket payments and universal insurance coverage are mentioned as health priorities [27]. The service coverage dimension of UHC is intricately linked to factors like immunization coverage, which has been identified as one of the most crucial indices in the research conducted by Kohpayehzadeh et al. [12]. Policymakers could use this index to monitor and improve service coverage, especially in underserved areas. By targeting regions with lower UHC scores, health authorities can allocate resources more effectively, ensuring that marginalized populations receive the necessary healthcare services, including preventive care, treatment, and rehabilitation.

Sustainable Development Goals (SDGs) Index is a comprehensive and evaluative framework that measures a country’s progress toward achieving the global Sustainable Development Goals which was set forth by the United Nations [36]. These 17 goals encompass a broad spectrum of economic, social, and environmental objectives aimed at promoting a more prosperous world by 2030 [37]. By aggregating these indicators into a composite index, the SDGs Index provides policymakers, researchers, and civil society with a holistic understanding of a country’s advancement towards these critical global objectives [36]. A study conducted by Selmani et al., identified 18 composite indices, each of which offered insights into different facets of health [38]. These indices encompassed a range of health determinants, including anthropometric variables, specific aspects of infants, oral health, healthy lifestyle, and functional ability. variables, aligning closely with SDG 2 (Zero Hunger) and SDG 3 (Good Health and Well-being). Policymakers can use this index to track progress toward global health goals and ensure that national health policies align with the global sustainability agenda. This holistic perspective can inform policies that address underlying issues such as poverty, education, and environmental health, which have significant implications for public health outcomes.

Strengths and limitations

In terms of the chosen method for prioritizing health indices, the use of an electronic questionnaire replaced the traditional face-to-face interview approach. This shift was necessitated by the constraints posed by COVID-19 restrictions and the busy schedules of the participating experts. Moreover, employing a web platform to administer the survey and monitor its progress enhanced efficiency and streamlined data entry, responses, analysis, and information collection from individuals. However, potential limitations include bias in expert selection, as the pool may not fully represent all relevant perspectives, and the online survey method may have excluded experts less familiar with digital tools. Additionally, response bias could arise from individual interpretations of the indices. Future research could address these biases by expanding the expert pool and using a combination of online and in-person methods.

Conclusion

This study provides a comprehensive set of population health indices, derived through expert consensus, which can guide future health measurement efforts in Iran. The identified indices, including Disability-Adjusted Life Year (DALY), UHC Service Coverage Index, Sustainable Development Goals Index, Human Development Index (HDI), and Quality-Adjusted Life Year (QALY), are essential for evaluating population health and addressing current health priorities. These indices are recommended for incorporation into health policy and strategic decision-making.

Future research could focus on refining these indices through pilot testing and evaluating their applicability in various contexts within Iran. Further studies could also explore the integration of these indices into national health monitoring systems to assess their utility in shaping health interventions and tracking progress toward health-related goals.