Acceptability, feasibility, fidelity and quality implementation of the culturally adapted version of the Social Competence Promotion Program among Young Adolescents (“Mi Mejor Plan”) to prevent substance use among adolescents in Chile: a pilot randomized control study

Abstract

Introduction

Substance use among adolescents is a public health problem. We culturally adapted The Social Competence Promotion Program for Young Adolescents (SCPP-YA) program to the school context in Chile (henceforth “Mi Mejor Plan or MMP”) and assessed the acceptability, feasibility, fidelity, and quality of the implementation among 6^th graders. We also explored the efficacy of the program in improving individual protective factors and reducing risk factors and substance use.

Methods

Cluster randomized controlled trial conducted in Chile. The schools were randomly assigned to one of two conditions in a 1:1 ratio: 1) the "MMP" intervention group, and 2) the Control group. The program consisted of a 16-h class-based curriculum promoting social problem-solving skills delivered by a trained facilitator. Primary outcomes were acceptability, feasibility, fidelity, and quality of the implementation using detailed reports of facilitators and from observers of the performance of the facilitators in vivo. Additionally, we explored the efficacy of the intervention on secondary outcomes: 30-day prevalence of tobacco, alcohol, and cannabis use and individual risk and protective factors promoted by MMP. We performed an intention-to-treat analysis using mixed models, taking into account the hierarchical nature of the data.

Results

Seven hundred sixty-five 6th graders from 11 schools were enrolled (one school dropped out after the randomization); 608 were analyzed at baseline, and 538 were analyzed post-intervention. 52.5% were male, and the average age was 11.3 in both groups. All 16 sessions were implemented, and students’ attendance at each session ranged from 83.8% to 92.4%. The program was generally well-received, with up to 91.3% of students rating acceptability positively. Facilitators and observers reported high adherence to the contents of the program in most sessions. Protective factors, such as negative beliefs about tobacco and alcohol, increased significantly in the IG. Still, there were no significant changes in substance use, risk factors, emotion regulation, or school membership.

Conclusions

The MMP program was well accepted, and we achieved high levels of implementation and fidelity. The program improved some individual protective factors, such as negative beliefs about tobacco and alcohol, with no changes in substance use among adolescents.

Trial registration

ClinicalTrials.gov, number NCT04236947; registration date: 17/01/2020.

Introduction

Globally, mental and substance use disorders are topics of particular concern regarding the global burden of disease among children and youth, accounting for 25% of all years lived with a disability (YLD) and 5.7% of disability-adjusted life years (DALYs) [1]. In addition, the use of alcohol, cannabis, and nicotine affects brain development during adolescence [2], increasing the risk of depression [3], anxiety [4], cognitive problems [5], psychosis [6], and suicidality [7]. These acute and chronic alterations have several consequences for future development, such as work opportunities and health and legal problems [8].

In Chile, the last available national study conducted in 2021 showed that 11.1% of adolescents smoked tobacco in the last month, 24.0% drank alcohol, and 11.2% used cannabis [9], making Chile one of the top countries in the prevalence of substance abuse among secondary students in the Americas [10]. Although this prevalence has been somewhat reduced recently, they are still alarming, especially considering the increase in crimes related to drug trafficking in Chile [11]. The Chilean government has implemented different initiatives in the last decades, such as creating the Explicit Health Guarantee for the Harmful Use and Dependence of Alcohol and Other Drugs for people under 20 years old, which grants benefits for the treatment and post-intervention of alcohol and drug abuse. It has covered 26,571 cases up to March 2023 [12]. Second, the implementation of a manualized school-based universal prevention program called “Continuo Preventivo” which includes four to six lessons per academic year between preschool and 12 th grade, since 2009 [13]. The effectiveness of this intervention has not been tested using a randomized controlled study [14]. Schools are a significant social context for adolescents. Here, students interact with peers and teachers, and it is a space for learning academic subjects and life skills [15]; and several studies have proved the importance and usefulness of implementing preventive measures in schools [15]. However, it is equally important to provide evidence of the acceptance, feasibility, and preventive impact of these interventions to avoid potential negative results [16, 17], especially if they are not well-informed by theory [18] or studied using rigorous methodology [19].

Several international substance use prevention programs have been successfully implemented in the school context with positive results. First, The Life Skills Training Program is a multi-component preventive intervention for students to develop substance resistance skills by learning self-management strategies, establishing healthy relationships, and engaging in responsible decision-making [20]. Another such program is The Lion’s Quest, Skills for Adolescence Program, which aims to develop socio-emotional competencies in students from preschool to 12 th grade and includes a Substance Prevention Module [21]. Finally, the Unplugged program, based on the theory of cognitive social influence, promotes intrapersonal skills (critical thinking versus normative beliefs) and interpersonal skills (assertiveness and skills of how to resist peer pressure) while also providing information on the different substances [22, 23].

The programs mentioned above have a complex theory of change, including several theoretical approaches, from Social Learning Theory to Normative beliefs theory and promoting the development of different intrapersonal and interpersonal skills [20,21,22], which sometimes makes it difficult to determine the actual mediators of the effectiveness of the intervention. Additionally, the payment of a usage license and the need for a costly certified training program reduces the possibility of scaling up the program if the results show it to be effective. Most of these programs can be implemented for students of different ages, making it also difficult for schools to choose the most convenient time for the intervention. Finally, all school-based interventions tested in other cultural contexts require an adaptation and a close collaboration with original developers. Roger Weissberg’s New Haven Adolescent Socio-Emotional Development Program [24], also known as the Social Competence Promotion Program for Young Adolescents (SCPP-YA) [25], has been evaluated and found effective in substance use prevention among adolescents. It uses a clear theory of change, no license costs are involved, there is a clear target age group, which makes it ideal for exploring the potential benefits and mediating variables of effectiveness; and finally, our research team had a strong relationship with the developer which helped to adapt the intervention appropriately.

The Social Competence Promotion Program for Young Adolescents (SCPP-YA) [25], was developed in 1990 and successfully implemented in numerous schools in the USA, especially in areas with greater economic vulnerability. Its first results showed that 6 th graders receiving the program, relative to controls, improved their strategies for resisting peer pressure and coping with stress and developed more negative attitudes about substances [26]. Later, a randomized controlled trial showed that the program had several beneficial effects for young teens: it allowed them to acquire problem-solving skills, improved their social relationships with peers, reduced their behavioral problems and alcohol consumption, and decreased the development of criminal behaviors [27]. Regarding problem-solving capacity, students who participated in this program improved the quantity and quality of their problem-solving choices compared to the control group [27]. The program also increased positive engagement with peers [25].

The program SCPP-YA is a manualized universal prevention intervention for 6 th graders. Its curriculum has a core module called Problem-solving Skills and other problem-oriented modules such as the Substance Use Prevention module. The first module has 27 sessions of 45 min, each focused on practicing several problem-solving skills based on six steps: (1) Stop, calm down, and think before you act; (2) Say the problem and how you feel; (3) Think of a positive goal; (4) Think of lots of solutions; (5) Think ahead to the consequences; and (6) Go ahead and try the best plan using interactive methodologies. The Substance Use Prevention module has nine sessions of 45 min, each centered on learning how to prevent substance use and how the skills learned in the problem-solving module can be applied to prevent substance use.

The SCPP-YA program aims to reduce substance use by targeting a range of cognitive, emotional, and social mediators, which has been evaluated in other preventive interventions among adolescents, and they are important. The program includes activities to address these mediators. At its core, the program strengthens problem-solving skills [28] and self-regulation [29] through a structured six-step model, which enhances emotional regulation, cognitive planning, and goal setting. Additionally, the Substance Use Prevention module seeks to modify both positive and negative beliefs about substance use—including beliefs related to tobacco, alcohol, and marijuana—by informed decision-making. By reducing positive beliefs [30] and reinforcing negative beliefs [31] about substance use, the program aims to shift students’ attitudes in a protective direction. The intervention further seeks to increase risk perception [31] related to substance use and strengthen protective factors such as self-efficacy and future orientation. Moreover, the program trains refusal skills, by improving students’ ability to manage peer pressure and make autonomous decisions in high-risk situations [32]. Together, these mediators are theorized to reduce intentions for future substance use, serving as the mechanisms through which the program ultimately promotes healthier decision-making and prevents the initiation of risky behaviors.

This program was the starting point of a significant educational movement: implementing interventions to promote social and emotional learning (SEL) skills. Roger Weissberg was one of the founders of Collaborative for Academic, Social, and Emotional Learning (CASEL), a Chicago-based nonprofit organization supporting SEL implementation since 1994 [33].

This program has never been studied in Latin American countries. Our research team foresaw the potential benefits of the cultural adaptation of this program. It started to work with Roger Weissberg to make this program more suitable for the Chilean educational reality while maintaining the core components of the intervention. The result was a newly adapted program called “Mi Mejor Plan” (MMP), which included a core problem-solving module of 11 sessions and a substance use prevention module of 5 lessons for 6 th graders, which can be implemented in one academic year, and three booster sessions in the following year.

Cultural adaptation of school-based substance use prevention programs is a critical component in enhancing their relevance, acceptability, and overall effectiveness among adolescent populations. Adolescents’ attitudes, beliefs, and behaviors related to substance use are deeply embedded within their sociocultural environments, which shape normative expectations, communication styles, and perceptions of risk. Implementing prevention programs without adequate cultural adaptation may limit their impact, as they may fail to resonate with students’ lived experiences or address context-specific risk and protective factors. By aligning the program’s content, language, and delivery methods with the cultural norms and values of the target population, cultural adaptation can improve student engagement, facilitate comprehension of key messages, and strengthen the intervention’s behavioral impact. Furthermore, culturally responsive interventions are more likely to promote equitable outcomes across diverse groups, thereby supporting broader public health goals in adolescent substance use prevention.

Conducting a pilot study is a critical step in the implementation of evidence-based interventions, particularly when adapting programs to new cultural or educational contexts. A pilot allows for the systematic assessment of key implementation outcomes—namely acceptability, feasibility, and fidelity—which are essential for determining whether a program can be effectively delivered in real-world settings [34]. Assessing acceptability helps identify how well the intervention is received by students, educators, and other stakeholders, informing potential refinements to increase engagement and relevance. Evaluating feasibility ensures that the program’s structure, duration, and resource demands align with the constraints and capacities of the school environment. Monitoring fidelity verifies that the program is being implemented as designed, which is crucial for maintaining the integrity of its theoretical foundations and expected outcomes. Together, these indicators provide valuable information that can guide necessary adaptations, support scale-up efforts, and increase the likelihood of achieving meaningful and sustained impacts on adolescent substance use prevention [34].

Given the exploratory nature of this study and its relatively small sample size, the pilot study serves a crucial role in generating preliminary evidence regarding the potential impact of the intervention [35]. While statistical power may be limited, pilot studies are not primarily designed to detect statistically significant effects but rather to provide effect size estimates that inform the practical or clinical relevance of the outcomes. Reporting these effect sizes allows readers and future researchers to evaluate the magnitude and direction of the observed effects, even in the absence of statistical significance. Moreover, these estimates offer valuable guidance for the planning of larger, adequately powered trials by informing sample size calculations and refining the selection of outcome measures. Emphasizing effect sizes in this context aligns with best practices in prevention science, where early-stage studies contribute to the evidence base by identifying promising intervention components and informing subsequent phases of research.

Therefore, the aims of this pilot study were: 1) to assess the acceptability and feasibility of “Mi Mejor Plan” in the school context in Chile among 6^th graders; 2) to assess the fidelity and quality of the implementation; and 3) to explore the efficacy of the program comparing self-reported substance use and risk and protective factors of 6^th graders participating in the MMP program and control schools immediately after the end of the intervention, controlling by baseline assessment. The impact and acceptability of the booster sessions and follow-up assessments are not part of this article.

Materials and methods

Study design, participants, procedure, and ethical considerations

This is a cluster randomized controlled trial, parallel-group type, with two arms: (1) MMP group and (2) control group, with the standard school prevention curricula. Schools were the unit of allocation, and individual participants were the unit of analysis. The trial registration identifier is NCT04236947 in Clinical Trials [ClinicalTrials.gov].

Inclusion criteria of schools:

• 1) Schools located in Region Metropolitana (RM), Chile.

• 2) Schools of Primary Education (1 st grade to 8 th grade). The MMP curriculum was designed to be implemented in 6 th grade.

• 3) Mixed-sex schools.

• 4) Schools with at least two classes in the 6 th grade.

• 5) Schools with at least 30 students per class

• 6) Schools with vulnerability (≥ 50%). We used the School Vulnerability Index – National System of Equality Allocation (IVE-SINAE) to state the degree of vulnerability. This index is the proportion of students with high vulnerability in a given school. This index considers the following socioeconomic variables to group the schools: mother’s educational level, father’s educational level, and total monthly household income, among others [36].

Exclusion criteria of schools:

• 1) Four or more classes in 6 th grade. This criterion was considered for economic and practical reasons because the larger the school, the more expensive the implementation and supervision.

• 2) Implementing other manualized substance use prevention programs. However, schools may implement preventive lessons in the Orientation curriculum (see below).

The recruitment for this study started in November 2019; however, in December 2019, the SARS-CoV-2 pandemic started, and the Chilean Minister of Health and the Minister of Education implemented sanitary measures that kept schools closed until October 2021. We stopped the recruitment and restarted at the end of 2021 to be ready to start the implementation in March 2022. To recruit schools for the present study, we implemented the strategy of First Come, First Served (FCFS), which implies that schools were invited till the sample size was completed because of the exploratory nature of the project and limited resources. Out of 575 Schools in RM, 480 schools were eligible for the study. We sent invitations to schools until we reached the sample size planned. Finally, we completed 165 invitations, and school authorities were informed about the study aims, methods, and assessments. Twelve schools agreed to participate and signed a consent. Randomization took place before the baseline assessment by an independent statistician (1:1) to assign schools to either MMP or control groups using a computer-generated randomization sequence. After randomization, all schools were informed of their allocation; at this time, one school from the intervention arm withdrew, saying that they did not have time for the implementation; this resulted in having five schools in the intervention arm and six schools in the control arm. Then, an informed and written consent form was sent to the parents or main caregivers for signature. Finally, students also provided informed and written assent. The total number of students enrolled in the schools was 765, and after signed consent by caregivers and assented by students, 608 (79.5%) adolescents answered the baseline assessment questionnaire (April 2022). In the post-intervention assessment (October 2022), 538 students answered the questionnaire. See the flowchart in Fig. 1.

Fig. 1

Recruitment; baseline; and post-intervention of schools and students

Regarding ethical considerations, data was collected following the Declaration of Helsinki [37], with the approval of the ethics committee of the Universidad de los Andes (CEC201983, November 5 th, 2019). The students and their parents or main caregivers received no reimbursement for participating in this study. Students or parents/main caregivers could request the withdrawal from the study at any time throughout the project. Confidentiality was granted for all assessments, and an independent statistician handled and coded the information.

Sample size

Since this is a pilot study, it is not appropriate to calculate a sample size for establishing the effectiveness of the intervention [38]. However, we have calculated a suitable number of adolescents for this study. According to recommendations for feasibility studies, a minimum of 30 participants per arm is helpful to estimate the parameters for future sample size calculations [39]. In addition, we wanted to ensure some variability, so we proposed to include three schools per arm, with three grades (Year 6, 7, and 8), with at least 60 students per grade in each arm. However, to test the efficacy or the effectiveness of school-based preventive interventions, it is necessary to work with a large sample size [40], so no definitive conclusions can be drawn from exploring the efficacy presented in this study.

Intervention and control group

Intervention group

Adaptation

The original SCPP-YA was developed by Roger Weissberg and collaborators in 1990 and first implemented in New Haven (USA) [25]; they gave authorization to the research team to adapt the program to the Chilean context and rename it “Mi Mejor Plan” (MMP). First, a translation was done by a bilingual professional translator and revised by two research assistants; afterward, the research team performed a linguistic adaptation. We interviewed potential stakeholders and schoolteachers to understand the potential obstacles to implementing the whole program in Chilean educational settings. Those interviews provided valuable information about the current preventive activities in the school and the time arranged for those activities during the academic year. For instance, all schools in Chile have a class for preventive lessons every week of the academic year. This class is called “Orientation”. The number of potential “Orientation” classes is 38 for the whole academic year [41]; however, schools usually use at least half of the classes to organize students for several other non-academic activities such as school trips, sports events, and conflict resolution meetings. Therefore, the availability for implementing preventive programs is less than 20 classes. Additionally, each session should last under 45 min to use the time provided for preventive lessons within the school curriculum. Therefore, a content adaptation was needed to have a shortened program version to fit schools’ calendars during one academic year.

The program curriculum has two modules: the first one is called the Problem-solving Skills module, and the second one is called the Substance Use Prevention Module. The first module was shortened to 11 sessions from the original 27 sessions of 45 min. Each session focused on practicing several problem-solving skills based on six steps of problem-solving strategy [25]: (1) Stop, calm down, and think before you act; (2) Say the problem and how you feel; (3) Think of a positive goal; (4) Think of lots of solutions; (5) Think ahead to the consequences; and (6) Go ahead and try the best plan using interactive methodologies. The second module was shortened to five sessions from the original nine sessions of 45 min, each centered on learning how to prevent substance use, how to use the skills learned in problem-solving to improve performance on social problems, and how to deal with peer influence. The research team analyzed the superficial and core components of the original program [42] and the theory of change presented in the Introduction. We carefully culturally adapted all superficial aspects of the program (e.g., pictures, posters, characters) and kept the sessions'core components. However, we faced a difficult challenge in the adaptation process. Adapting the program requires conducting culturally relevant changes while keeping the core elements [42, 43]. The original program had 27 sessions (problem-solving module) plus nine sessions for the substance use module. In this challenging decision, most of the sessions that we reduced were those dedicated to practicing the skills, trying to condense the core components of the program (the 6-step problem-solving method) into 11 sessions, always providing interactive activities to practice all steps of the problem-solving method. The same was true for the sessions for the substance use model. In addition, we carefully kept the sequential approach of the problem-solving structure presented in the original manual. In both modules, even though we intended to keep the core components, we may have compromised the effect of the intervention through at least two mechanisms. On the one hand, reducing the exposure time needed to adopt a skill students (by reducing the practice sessions) may have negatively impacted the skill acquisition. On the other hand, altering the whole structure of the program, the mediators of the intervention, even though they may act independently of each other, the combined indirect effects separately for each mediator may differ from the overall natural indirect effect if there are additive interactions between the mediators'effects on the outcome [44].

The new 16-session program was culturally appropriate and increased the feasibility of implementation in Chilean settings. Each session was organized into three sections, each with specific activities: opening, central, and closure. Details of the session selection process can be found in Supplementary Material 1.

Implementation

The intervention group implemented the adapted version of the SCPP-YA program called “Mi Mejor Plan (MMP)” during the academic year within school hours, usually in the “Orientation” weekly classes. The program’s curriculum was delivered by a trained facilitator (member of the research team) and the school teacher (who was also trained). The program includes the facilitator´s manual and the student´s workbook.

Control group

The control group received standard school drug prevention curricula (“Treatment-As-Usual,” TAU), which is usually a non-manualized intervention. The standard curriculum in the “Orientation” class may include teaching on health promotion and preventive messages for substance use.

Measures

Primary outcomes

Acceptability, feasibility, and fidelity

Acceptability

Students provided information regarding acceptability using a self-reported survey that included four sections related to the perception and their experience in the sessions: (1)“General acceptability” (4 items); (2) “Implementation & material acceptability"(5 items); (3) “Helpfulness of MMP program” (7 items); and (4) “Satisfaction with MMP program” (4 items). All items were responded with a Likert scale (1 = “Strongly disagree/Strongly dissatisfied” to 5 = “Strongly agree/Strongly satisfied”).

Feasibility

The feasibility of the recruitment process was measured using a registry of consent, assent, absence, and questionnaire response rates. We also registered the degree of attendance of students by session.

Fidelity and quality of implementation

We used a questionnaire that included three dimensions of fidelity [45]: (1) adherence, (2) pedagogy, and (3) logistics, answered by the facilitator after the delivered sessions. Additionally, an observer attended sessions in the schools when they were delivered, registered, and rated in written form questions regarding two dimensions of fidelity (adherence and pedagogy). It was expected that observers rated at least 25% of all delivered sessions.

Adherence refers to how much the activities described in the manual were fully implemented. In the case of facilitators, they reported if they implemented the activities in their entirety, and if they could not implement all activities, and provided the reasons for doing so. For the latter, we used the following options: Lack of time and difficulty in group management. In the case of observers, adherence was assessed in more detail, providing a rating of all sections of each session: opening, central, and closure activities. Observers provided the assessment score in six possibilities. Briefly, 0% (Not Started), 20% (Barely Started), 40% (Partially Complete), 60% (Moderately Complete), 80% (Almost Complete), and 100% (Fully Complete). We also assess the degree of knowledge of the manual demonstrated by the facilitator during sessions. It followed the following scoring: 0% (No Knowledge); 20% (Basic Initial Knowledge); 40% (Partial Knowledge); 60% (Moderate Knowledge); 80% (Advanced Knowledge); and 100% (Expert Knowledge).

Pedagogy refers to strategies to facilitate delivery quality, such as physical space preparation, improving participation and group dynamics, and the perception of students´ responsiveness (e.g., attention and participation during the sessions). These strategies were assessed by facilitators after each session, using a 5-point scale from 1 (Never) to 5 (Always). Additionally, we evaluated group management by observers using a scale based on the following scoring in a 6-level: 0% (No Control); 20% (Minimal Control); 40% (Partial Control); 60% (Moderate Control); 80% (Strong Control); and 100% (Mastery of Group Management).

Logistics refers to the duration of each session and the total number of sessions implemented.

Other detailed aspects of fidelity and quality of implementation, such as adherence by each facilitator and school, are presented in the Supplementary Material 3.

Secondary outcomes

• 1. The European Drug Addiction Prevention Trial Questionnaire (EU-Dap) [46]. EU-Dap is a self-reported questionnaire that aims to screen substance use and assess risk factors (positive beliefs about tobacco, positive beliefs about alcohol, positive beliefs about cannabis, normative beliefs, and future substance use) and protective factors (negative beliefs about tobacco, negative beliefs about alcohol, negative beliefs about cannabis, risk perception, and refusal skills) among adolescents. It has 45 multi-item questions and focuses especially on tobacco, alcohol, and cannabis use. The EU-Dap instrument has been validated in Chile; all subscales had acceptable internal reliability (omega ≥ 0.65) [47].

• 2. The Social Problem-Solving Inventory-Revised (SPSI-RS) [48]. SPSI-RS consists of 25 items measuring five subscales that evaluate two dimensions of positive (Positive Problem Orientation and Rational Problem-Solving) and negative (Negative Problem Orientation, Impulsive/Careless Style, and Avoidance Style) problem-solving strategies. Each subscale has five items, answered on a 5-choice Likert-like scale, in which the subjects indicate to what extent the proposed coping strategy applies to their characteristic way of dealing with the problems of daily life (0 = Very untrue, 1 = Somewhat untrue, 2 = Neutral, 3 = Somewhat true, and 4 = Very true). There is also a total problem-solving score, which includes the 25 items, where the higher the score, the better the performance in the problem-solving strategies. In this study, we used the total score in the analysis, which demanded that the score of all negative scales be reversed. It has acceptable internal reliability (Cronbach alpha > 0.75) [49].

• 3. The Emotion Regulation Questionnaire for Children and Adolescents (ERQ-CA) [50]. Emotional regulation is the ability to control and manage one’s emotions and appears to be an essential factor in social adjustment. This instrument has ten items organized into two scales: Cognitive Reappraisal (six items), which consists of redefining a potential emotion-eliciting situation in a way that its emotional impact is changed, and Expressive Suppression (four items), encompassing a style that consists of the inhibition of ongoing emotion expressive behavior. For each statement, the answers are on a five-point Likert scale quantifying the agreement with the behavior described (1 = strongly disagree, 2 = disagree, 3 = half and half, 4 = agree, 5 = strongly agree). It has been validated in Chile and showed acceptable internal reliability (Cronbach alpha ≥ 0.75) [51].

• 4. The Psychological Sense of School Membership Scale (PSSM) [52]. The sense of school membership refers to the student’s perceptions of the respect and acceptance that teachers and other students show them and their sense of belonging. It has 13 items (Chilean version) and is a single dimension. Our research team validated the scale in Chile in early adolescents, showing a high internal reliability (Cronbach’s alpha 0.92) [53].

Statistical analyses

The Consolidated Standards of Reporting Trials (CONSORT) statement was followed for reporting the results of this study [54].

For primary outcomes, students'acceptability of the program was described using frequencies of the degree of agreement with several statements. The feasibility of the recruitment, assessment, and intervention implementation was described using frequencies. For the fidelity and quality of the implementation (adherence, pedagogy, and logistics), we present the frequencies, proportions, and means of scores for different items of surveys completed by facilitators and observers.

For secondary outcomes, all schools and participants were included in the analysis using an intention-to-treat approach [54]. First, we used descriptive statistics to assess balance across arms at baseline. We analyzed each outcome measure using two different 2-level mixed models; the first model (Model 1) accounts for fixed effects for baseline scores, group effect (intervention vs control), time (baseline, endpoint). The second model (Model 2) accounts for the same fixed effects as Model 1 but adds sex and age as covariates. Every model included a random effect for school and participants (allowing correlations within schools and repeated measures within participants). For binary outcomes, we used logistic mixed-effects model analysis to compare the intervention and control groups from baseline to post-intervention, and then we calculated odds ratios. For continuous outcomes, we used linear mixed-effects model analysis to compare the intervention and control groups from baseline to post-intervention. We calculated adjusted mean differences with 95% Cis and Cohen´s d effect size. All analyses were performed using STATA 17.

Results

Descriptive features at baseline

A total of 608 students were included in the baseline of the study. A 52.5% of students were male in both intervention and control groups, and their mean age was 11.3 in both groups. Substance use was slightly higher in the intervention group, especially in the category of cannabis use in the last month. Regarding the risk and protective factors for substance use assessed with the EU-Dap questionnaire, all risk factors, such as “positive beliefs about tobacco”, were slightly higher in the intervention group, and all protective factors, such as “negative beliefs about tobacco”, were slightly lower in the intervention group. Emotion regulation strategies were similar in both groups. Social problem-solving skills were poorer in the intervention group. School membership score was also lower in the intervention group. See Table 1.

Table 1 Baseline characteristics

Primary outcomes

Acceptability

In general, the program had good acceptability by the students. All aspects assessed in the “General acceptability” and “Implementation and material acceptability” sections were considered highly positive, up to 91.3% (“Agree/Strongly agree that the Teacher and the Facilitator get along”). Regarding the “Helpfulness” section, students reported a high sense of helpfulness, especially that “The program is useful to learn about the dangers of drugs” (84.6% reported Agree/Strongly agree). Finally, 77.5% of students reported being satisfied/strongly satisfied with the program. See Table 2.

Table 2 Students´ acceptability of the program

Feasibility

Out of 575 Schools in RM, 480 schools were eligible for the study. 165 were invited to participate in the project, and school authorities were informed about the study aims, methods, and assessments. 123 schools never answered our invitation, and 31 refused to participate, mainly for not accepting the possibility of being allocated to the control group. Finally, 12 agreed to participate and signed a consent. After randomization, but before the baseline assessment, one school in the intervention group withdrew from the study, saying they no longer had the time to implement the sessions. See Flowchart in Fig. 1.

Regarding the students, the questionnaire response rate at baseline concerning school enrollment was 76.5% and 81.9%, and 78.9% and 84.5% concerning the consent in the intervention and control groups, respectively. These figures at post-intervention assessment were 68.6% and 71.7%, and 70.8% and 74.0%, respectively. See Table 3.

Table 3 Students´ response rate at baseline and post-intervention

The level of attendance at the sessions can be seen in Table 4. By session, the lowest attendance was in session 2 (84.4%), and the highest was in session 13 (92.4%). By school, attendance rate ranged from 83.8% to 91.8%. See Table 4.

Table 4 Students´ attendance

Fidelity and quality of implementation

The following information regarding facilitators’ reports can be seen in 5a-5 d, and the observer´s reports can be seen in Tables 5, 6, 7 and 8.

Table 5 Program adherence reported by facilitators and observers

Table 6 Program pedagogy reported by facilitators and observers

Table 7 Program logistics reported by facilitators

Table 8 Program logistics, number of sessions implemented by the facilitator

Adherence

Six facilitators implemented sessions of the program. Facilitators´ self-reports were obtained for 160 sessions (all sessions implemented). Facilitators reported that all activities were conducted in most of them (n = 139, 61.9%). However, 61 sessions were performed partially because of a lack of time (30.0%) or difficulties in group management due to the students´ behavior (8.1%).

Regarding observer´s reports, 52 sessions were assessed in the field (32%). The assessment showed that adherence was high for all sections of sessions: opening (94.6%), central activities (77.2%), and closure activity (80.9%). The degree of knowledge demonstrated by facilitators was close to “expert knowledge” (98.5%). For further information see Supplementary Material 2.

Regarding the performance of each facilitator and school, see Supplementary Material 3.

Pedagogy

Most of the facilitators reported that they often or always prepared the space for the session (90.7%) and used strategies to improve the student’s participation (88.8%) and group dynamics (89.3%). Participants had high levels of attention (71.9%) and participation (82.5%).

Group management was assessed by the observer close to “Mastery of group management” (90.4%).

Logistics

Regarding the duration of each session, 88.1% of the facilitators used the designated time, 6.3% outrun the designated time, and 5.6% used less of the designated time.

All 16 sessions (100%) were implemented. Most sessions were implemented by one facilitator (53.8%).

Secondary outcomes

Even though substance use variables had higher odds in the intervention group than the control group, the results are not statistically significant. See Table 9.

Table 9 Substance use

None of the risk factors were significantly reduced with the program. Regarding the protective factors, negative beliefs about tobacco (Cohen´s d = 0.30) and alcohol (Cohen´s d = 0.30) were both significantly increased in the intervention group compared with the control group.

The intervention did not improve emotion regulation and problem-solving strategies. Finally, school membership was also not improved in the IG. See Table 10

Table 10 Risk and protective factors

Discussion

This is the first study exploring the acceptability, feasibility, and fidelity of the adapted version of The Social Competence Promotion Program among Young Adolescents (SCPP-YA), under the name of “Mi Mejor Plan” (MMP) in Latin America. This study also explored the efficacy of the intervention in reducing substance use and risk and promoting protective factors. All sessions were implemented, and the facilitators and observers reported high fidelity and quality of the intervention. The intervention was well received by students, and most of them said that they learned about the risk of using substances of abuse and thought that the program gave them skills to reduce using drugs in the future. Although there were no differences in the 30-day prevalence of substance use, it seems that the program increased critical protective factors such as negative beliefs about tobacco and alcohol.

Although there are some preventive programs showing effectiveness in high-income countries, such as School Health and Alcohol Harm Reduction Project (SHAHRP) in Australia [55], Life Skills Training (LST) program in the USA [56,57,58], and Unplugged in seven European countries [59], fewer interventions have been rigorously tested in Latin America. For instance, Unplugged has been studied in Brazil [60] and Chile [23] with promising results. Our research team already adapted Unplugged when we decided to adapt SCPP-YA to Chile, and this work guided the adaption process in this study. Although Unplugged includes some content promoting social problem-solving skills [58, 61], SCPP-YA uses this strategy as the main drive to generate behavioral changes among early adolescents. For our team, it was important to test another intervention on substance use prevention and give adolescents, schools, and policymakers a wider range of options when deciding which intervention best suits the school culture.

Regarding feasibility, our team implemented all 16 sessions in the intervention group. Some experiences have found that trained school teachers do not perform well on fidelity and implementation [47]. In contrast, other studies have found better implementation quality when teachers delivered the program compared with external facilitators [62]. While others have found no differences between teachers and external specialists in student outcomes or implementation quality [63]. We decided to hire and train external facilitators, either free-lance teachers or psychologists, to reduce the risk of failure in the intervention delivery. This consideration was based on Chilean teachers'well-known high workload in day-to-day activities [64], making it challenging to include the training, delivery, and supervision during the program's implementation in their routine. However, we decided to train external facilitators in the same training sessions along with school teachers to provide good knowledge of the program and get to know each other to become close collaborators during classroom sessions. In this regard, the highest scores reported by students in the acceptability survey were the items “Teacher and Facilitator get along” (4.4, SD = 0.7) and “How satisfied are you with the facilitator” (4.4, SD = 0.8). Therefore, they perceived the collaboration and relationship between teachers and facilitators to be good. We considered this good relationship a potential factor explaining the high degree of implementation.

The program was well accepted by adolescents with a high degree of satisfaction, and they liked the activities. The program included interactive activities, role-playing, and games to increase participation. Some authors have recommended that school-based preventive interventions should include these kinds of practices to promote active student learning [65, 66]. We may be confident that the changes appealed to the adolescents because the program was well accepted. Adolescents also could identify the aims of the program because they reported that the program helped them to know about drugs and to learn the skills needed to reduce drugs in the future. These results are consistent with the findings of increasing some of the protective factors promoted by the program.

Adapting the program was challenging, requiring culturally relevant changes while preserving core elements [42, 43]. As we have explained in the Methods regarding the adaptation process, reducing the practice sessions may have negatively affected skill development, knowing that practice is important. Additionally, affecting the whole structure of the original program may have cut out some indirect effects of the mediators'interactions [44]. Knowing that practice is essential, we decided to provide more opportunities to practice the skills and designed three booster sessions, which were implemented in the following year. The whole 2-year MMP program results will be presented in future publications.

Finally, the exploratory results of the efficacy showed that the MMP program increased negative beliefs about tobacco and alcohol in the intervention group compared to the control group. A recent study of the effectiveness of Unplugged in Nigeria showed that the intervention significantly reduced the prevalence of recent alcohol use in the intervention compared to control pupils. Negative beliefs about substance use were one of the mediators of Unplugged's effect on alcohol use. Therefore, there is some evidence that these protective factors could benefit students in the future. On the other hand, there was no evidence of efficacy in reducing any of the measured substance use. They may be disappointing results, but these are exploratory. This was a pilot study with the primary objective of describing the acceptability and feasibility of the implementation of MMP without the statistical power to test either the efficacy or the effectiveness of this intervention. Additionally, the students in the intervention group had poorer results in substance use and on the risk and protective variables measured at baseline. This may reflect that these students were coming to the study with more difficulties than the control group. A larger randomized controlled study may help to reduce the imbalance between groups. Finally, the post-intervention assessment happened right after the end of the 16 th session. Some of the promoted skills may unfold over time, underscoring the necessity for ongoing practice and reinforcement [67], so follow-up assessments are valuable for exploring this hypothesis and seeing if the expected preventive effect appears with time.

Limitations

One key limitation was the reliance on self-reported data, which may be subject to social desirability bias. Additionally, the intervention took place after the pandemic when schools were re-installing normal academic activities with many disruptions in functioning. Chile was the Organization for Economic Co-operation and Development (OECD) member country that kept schools closed for the longest time during the pandemic. The country closed schools for 147 school days in 2020 and 112 in 2021, totaling 259. Chile was the only member nation of the OECD that exceeded 250 days of school closures [68]. Even though the after-pandemic was a difficult time for schools, the implementation of the program reached high levels of feasibility. As we already mentioned, some imbalances regarding substance use and risk and protective factors and differences in school characteristics, such as size and administrative structure, also arose by chance between groups.

Additionally, when comparing the groups, the control group was considered as Treatment-As-Usual because we did not interfere with the content of the “Orientation” class implemented in all schools in Chile. A limitation in this regard can be related to the lack of information the research team had on the content, material, and pedagogy used in these classes and whether they were or were not related to substance use prevention. Another limitation is related to the representative of our results. The schools included in this study were located in highly deprived areas and served low-income families. This is the case for 89% of schools in Chile [69], so they represent a large population in Chile. However, these results cannot be extrapolated to middle- to high-income families. An additional limitation for a larger expansion of the MMP program is related to the fact that we used external trained facilitators to deliver the intervention. This decision helped us ensure high fidelity; however, it limited cost-effectiveness and widespread scale-up potential due to the added costs and efforts. Despite these limitations, the study’s primary focus on acceptability and feasibility yielded valuable insights for planning larger-scale evaluations of the intervention. Finally, even though we recognized the importance of assessing acceptability among external facilitators and schoolteachers, we did not plan to systematically assess it because of the workload in their teaching activities, making it difficult for them to answer an acceptability survey. We plan to include their perspectives in future phases of the study.

This is a limitation for future implementation, especially if we want to make this kind of intervention sustainable in time. Future research should include easy options to assess this information and make this process more participatory.

Conclusions

This is the first study of the SCPP-YA (MMP) adaptation to prevent substance use among early adolescents in Latin America. The program was well accepted, and we achieved high levels of implementation and fidelity. The program improved some individual protective factors, such as negative beliefs about tobacco and alcohol. However, preliminary results on the efficacy of reduction of substance use showed no effect of the intervention. Future studies should explore the long-term effect of the intervention and the impact of the three booster sessions implemented in the following year. Moreover, these findings offer important contributions to the field of prevention science in Chile by highlighting key areas for intervention within school settings. The results underscore the need for culturally relevant, evidence-based approaches that address both individual and contextual factors influencing substance use. From a policy perspective, there is still a challenge in how school-based prevention strategies can be integrated into preventive education into the national curriculum, alongside systematic schoolteacher training in these matters. Solving these issues could enhance the sustainability and effectiveness of prevention efforts, contributing to a more comprehensive and proactive public health response.