The effectiveness of providing training and ongoing support to foster cultural humility in volunteers serving as mentors to youth of color: a mixed-methods study protocol

Abstract

Background

The aim of this randomized control trial is to test the impact of providing additional training and support to volunteers who are paired with youth of color in the Big Brothers Big Sisters (BBBS) community-based mentoring program. The aim of the intervention activities is to enhance the capacity of mentors to have more culturally responsive and informed interactions with their mentees of color, thereby strengthening the youth’s ethnic/racial identity and abilities to both cope with experiences of racism and contribute to causes that advance social justice.

Methods

Recruitment started in June 2022, with a goal of enrolling 240 dyads (i.e., “matches”), each consisting of a volunteer mentor and a youth of color aged 9- to 17-years old with whom they were paired through BBBS. Each match is assigned randomly to receive either standard BBBS services or to services that incorporate the intervention activities (i.e., approximately 3 h of initial training that is then supplemented with booster emails and in the context of the contacts that case managers have with mentors routinely in the program). The BBBS staff who are responsible for delivering the enhancements receive preparatory training as well as ongoing support with implementation. The study has a mixed-methods design. Survey data, on outcomes (e.g., ethnic/racial identity, sense of mattering, efficacy) aligned with the theory of change, are collected at multiple time points within 12 months from mentors, youth and their parent/guardian, and BBBS staff. Multiple qualitative interviews are conducted with a subset of youth, mentors, parents and BBBS staff to examine how the intervention works and how it impacts relationship development and youth outcomes over time. Integration of quantitative and qualitative data will aim to better understand whether and how the intervention works with respect to its potential influence on mentor attitudes (e.g., cultural humility), mentor-youth interactions, and emergent identities and capacities that have well-documented importance for the resilience and well-being of youth of color.

Discussion

This culturally tailored training and support intervention for volunteer mentors may be one way to increase the effectiveness of mentoring programs for youth of color. Study findings will have implications for youth mentoring programs and for other settings (e.g., schools, after school programs) in which children and adolescents form relationships with adults.

Trial registration

www.clinicaltrials.gov—Clinical Trial #NCT05391711; original 05.21.2022; Amendment 07.01.2022: study status was updated and more details were provided on outcome measures; Amendment 11/13/2022: sample size was modified, a few mentor outcome measures were added in the 12-months survey, the timing of a mentor measure was updated.

To reduce the likelihood that adolescents engage in behaviors linked to chronic health problems later in life, an objective of Healthy People 2023 [1] is to increase the proportion of adolescents who have an adult in their lives with whom they can talk about serious problems. Research supports mentoring program participation as an evidence-based strategy for enhancing the availability of adult support for this age group and an effective way to improve behavioral, social, emotional, and academic outcomes in adolescents (for meta-analyses, see OHDP, DuBois et al., Raposa et al., and Tolan et al.) [1,2,3,4]. Yet, these effects are typically small in magnitude, suggesting potential to strengthen the effectiveness of these programs. Given that many of these programs serve children and adolescents of color [5], one way to improve their effectiveness is to better train and support the volunteer mentors who serve these populations. We will investigate the effectiveness of an innovative approach to cultivating cultural humility among mentors and staff in a youth mentoring context.

Many low-income youth of color spend much of their lives outside their homes interacting with adults whose backgrounds (e.g., race, ethnicity, socioeconomic status) differ from their own. A national study [5] of 1,451 youth mentoring programs serving over 400,000 youth nationwide revealed that most volunteer mentors were White, whereas the mentees they served were mostly youth of color. Another study [6] of 1,310 low-income children and adolescents (over half were youth of color) served through primarily Big Brothers Big Sisters (BBBS) mentoring programs found that most mentors were dissimilar to their mentees in their socioeconomic, cultural or family background. For example, only 15% of mentors reported having experienced poverty, while most mentees were from low-income homes. Given these differences, mentors may begin their interactions with youth with preconceived notions about youth and their background. For instance, a qualitative study [7] of the reflections of 32 predominantly White college student mentors serving mostly low-income youth of color, revealed that some had negative assumptions and stereotypes about their mentees when they entered the relationship. An online survey [8] of 1,022 White adults who worked or volunteered with children revealed that at least 27% endorsed one or more negative stereotypes (i.e., lazy, violence prone, unhealthy habits) about Black teens and other teens of color, whereas fewer endorsed negative stereotypes about White teens. Further, a recent qualitative study [9] of 36 mentoring relationships which included mostly youth of color noted that many mentors held deficit views about their low-income mentee’s familial economic situation, which in turn, contributed to negative attitudes and interactions with the family. Negative views such as these could affect the initial stages of relationship development, mentor expectations, and the opportunities that mentors provide, or fail to provide, to youth of color.

In addition to the racial and economic differences between youth and mentors, many volunteer mentors may be unprepared to deal with racism that arises in their interactions with youth, which may further perpetuate racial inequality. Research on youth-staff interactions in the context of out-of-school-time activities supports this idea. For example, a qualitative study [10] of 33 racially/ethnically diverse program leaders of after-school programs serving racially/ethnically diverse youth revealed that many felt ill-equipped to deal with incidents of discrimination (i.e., around race, ethnicity, immigration, religion or language) that occur in youth programs. Some staff used a “race-blind” approach to these incidents; specifically, these staff reported that there were no issues related to culture or race in their program, that race and culture were not important, and that this approach was a fair way to treat youth. The authors argued that not challenging the privileges of dominant groups may isolate and alienate youth from nondominant groups. Other staff responded to the incidents in a limited way. They stated that the incidents were important to address, but did not feel comfortable or adequately prepared to respond. The authors [10] thus highlighted the importance of training staff to examine their assumptions and teach them how to respond to racism to increase staff confidence, which could lessen their role in perpetuating racial inequalities.

How mentors respond to discrimination and the extent to which they are culturally responsive could influence the development of their relationship with their mentees. For example, Spencer [11] revealed that one of the reasons youth mentoring relationships fail is because some mentors are unable to bridge and respond effectively to cultural differences between themselves and their mentees. A qualitative study [12] of mostly White volunteer mentors serving a majority Latinx male mentee sample involved with the justice system found that mentors who had shorter relationships with their mentees were more likely to have racist views about the youth and their families, whereas those in longer-term relationships had a critical awareness of the issues their mentees were experiencing. Another study [13] of teen girls of color matched with White female adult mentors in a BBBS program found that girls’ higher cultural mistrust toward White people at baseline predicted less instrumental satisfaction in their mentoring relationships (i.e., mentees’ perceptions that the relationship had helped them grow) at 12 months, controlling for instrumental satisfaction at 3 months. Thus, mentees’ cultural mistrust, which may be rooted in their past experiences with racial discrimination, can also serve as a barrier to the development of beneficial relationships with the adults with whom they are paired in programs.

These differences in relationship length and quality for many youth of color are important because they are linked with disparities in program benefits [14,15,16,17]. For example, in a large-scale BBBS study [17], youth in relationships lasting at least 12 months experienced improvements (relative to controls) in academic, behavioral and psychosocial outcomes; fewer benefits were experienced by those in matches ending prior to 12 months, and some decrements were noted for those youth in matches ending prior to 6 months. Longer and/or higher-quality relationships are associated with better developmental outcomes [14], such as self-esteem [17], ethnic identity [18], efficacy coping with discrimination [19], academic efficacy [20], and mental health [21]. Given the rewards of higher-quality and longer-term relationships, it is important to help mentors overcome barriers to developing strong and sustained relationships with their mentees.

One potential way to reduce bias and racism from adults who work with youth of color and help them develop higher-quality relationships with youth is by training and supporting cultural humility in mentors. Cultural humility is a lifelong learning process that “focuses on knowledge of self in relation to others, acknowledges the dynamic nature of culture, and challenges barriers that impact marginalized communities on both individual and institutional levels” (p171) [22]. Cultural humility in youth mentoring includes being reflective of oneself in the relationship and of how one’s identity (and the mentee’s identity) is positioned within systems of privilege and oppression, rather than simply learning about “other” marginalized groups [23]. An important aspect of cultural humility in mentoring relationships with youth of color is the mentors’ awareness of, and skills to support, their mentee’s ethnic/racial identity development and the ability to have supportive conversations around racism.

Research supports the potential for cultural humility training to change mentors’ attitudes. A case study [24] of six college students in a seven-week “critical mentoring” preparation course found an increase in mentors’ awareness of the racialized sociopolitical histories in their community, their own racial biases and stereotypes, and the negative impact of White saviorism in mentoring. Other studies [23, 25] have found that volunteer mentors who participated in a cultural humility training showed an increase in their self-efficacy to provide ethnic/racial support to their mentees (e.g., confidence in helping their mentee feel good about their ethnic/racial group, confidence in helping their mentee cope with prejudice and discrimination) relative to mentors in a control group who did not receive this training. In a study [26] of predominantly White, middle-/upper-class mentors matched with predominantly Black, low-income high school students, after receiving training on race, class, and White privilege, mentors’ journals and papers indicated that they reflected on both assets and negative conditions of their mentees’ communities. Furthermore, mentors described increased recognition of their own biases and stereotypes, although some continued to perpetuate blame toward their mentees rather than recognizing the role of social context [26], suggesting that further research is needed to determine why some mentors begin to disrupt inequality while others reinforce it. A limitation of the aforementioned research is that only one study [23] utilized a randomized controlled trial (RCT) design. Further, researchers have mostly examined the impact of these trainings on mentor attitudes, but without any consideration of youth’s age and development, and as such, it is unknown whether these trainings impact mentors’ interactions with youth and ultimately youth outcomes.

The aim of this study is to investigate an innovative set of trainings and ongoing supports for volunteers in the BBBS mentoring program that orients them to cultural humility in their mentoring relationships with youth of color and provides strategies for integrating cultural humility into these relationships. The goal of the intervention is to foster mentors’ skills to support youth of color in developing a positive ethnic/racial identity and enable them to have supportive conversations about racism and social justice issues. To address gaps in the literature regarding the effects of cultural humility training, the study includes an RCT to rigorously test the effects of the intervention on mentor attitudes, their interactions and relationships with youth of color, and youth outcomes. We hypothesize that the training and support will have favorable effects on: a) mentor cultural humility; b) culturally attuned interactions between mentors and youth; and c) positive youth developmental outcomes (see Fig. 1: Hypothesized theory of change). Further, a qualitative study with a subset of youth-mentor dyads will be conducted to understand how the intervention works and gain an in-depth understanding of potential intervention effects on the mentors, mentoring relationships, and youth.

Fig. 1

Hypothesized theory of change

Methods/design

Study setting

This study is being conducted with Big Brothers Big Sisters of America (BBBSA) in the U.S. and seven of its affiliate agencies across the U.S. that provide mentoring services. BBBSA is the largest network of mentoring programs in the U.S. and serves approximately 159,000 youth annually [27].

Overview of research design

The study uses a mixed-methods, experimental convergent design [28]. The quantitative component is an RCT that tests the effectiveness of the Justice, Equity, Diversity, and Inclusion (JEDI)-Culturally Smart Relationships (CSR) intervention. The qualitative component is embedded within the quantitative study and aims to understand how and why the intervention works, how participants are experiencing the intervention, and relationship development over time, in addition to gathering in-depth information about our study’s theory of change. Two flowcharts (see Fig. 2: Study flow diagram for case manager participants and Fig. 3: Study flow diagram for youth, parent, and mentor) illustrate the timeline of participant enrollment, assessments, randomization, and intervention components, as recommended in the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) Statement [29].

Fig. 2

Study flow diagram for case manager participants

Fig. 3

Study flow diagram for youth, parent, and mentor participants

Randomization occurs at two levels. First, case managers, the agency staff who provide supervision services to youth-mentor relationships (i.e., “matches”), are randomly assigned to either the treatment or control condition by the research team. Agency administrators who supervise the case managers assign each case manager a code, maintain the list of codes, and complete short surveys about each case manager, which includes questions about the case manager’s race, tenure at the agency, ability to provide effective match support, whether they provide match support in a language other than English, and whether they supervise matches in specific geographic locations. The researchers use the de-identified survey responses to pair each case manager with a similar case manager at their agency and then randomly assign one case manager in the pair to the treatment condition and the other to the control condition. Pairing in this way aims to balance the groups on characteristics that may influence supervision quality. Second, randomization occurs at the mentor-youth dyad level. When a new volunteer mentor enrolls in the study and is matched with a study-eligible youth (see eligibility criteria below), the youth-mentor dyad (i.e., study match) is randomly assigned to either the treatment or control condition and is added to the caseload of a case manager in their respective condition. For mentors in existing matches (i.e., in an ongoing mentoring relationship with a study-eligible youth for up to 2 years), the dyad is assigned to the condition previously assigned to their case manager. This process allows agencies to make the best matches in accordance with evidence-based practice standards, rather than study enrollment needs.

At two agencies, whether a match is assigned to the treatment or control condition is determined by their geographic area due to the case manager structure at those agencies. Specifically, only one case manager works with all matches that are served in a particular region which makes random assignment not possible. Not conducting random assignment at these agencies reduces interference by the research team in program services (e.g., requesting that two case managers serve a particular region to enable youth to be randomly assigned to either condition within that region).

Allocation sequences for random assignment of both case managers and matches are computer-generated by the research team and are unavailable to BBBS staff in view of their responsibilities involving study enrollment. For matches, separate allocation sequences in blocks of a specified size (not reported here to reduce predictability) are generated for each agency to allow for stratification of assignment by agency. The research team informs the study liaison at the relevant agency of each assignment; in the case of matches, the research team also notifies the youth and their caregiver as well as the youth’s mentor. Researchers are not blinded to the treatment condition throughout the data collection process as it was not practical to do so given the differences in content of the surveys completed by those in the treatment and control groups. Given the nature of the intervention, agency staff are also not blinded to treatment condition.

Eligibility criteria and recruitment

Affiliate agencies were selected based on size (i.e., serving at least 250 new mentees each year), youth population (i.e., at least 200 new mentees each year identify as African American/Black and/or Hispanic/Latino), and having a majority of matches that are cross-race (i.e., youth and mentor have different racial/ethnic identification). The goal was to recruit 6 to 8 agencies that could collectively enroll at least 500 matches to participate in the study. This sample size was selected on the basis of statistical power analyses conducted using the Optimal Design software, which indicated that it would provide adequate power (0.80) for detecting small-to-medium impacts of the intended enhancements on outcomes (i.e., standardized mean difference between treatment and control groups of 0.35). Due to challenges associated with COVID and other factors, the target sample size was subsequently lowered to 240 matches. Power remained adequate (0.80) with this sample size in the context of an exploratory mode for hypothesis testing (i.e., p < 0.10 rather than p < 0.05). This approach is defensible in view of the very nascent stage of research on interventions of the type under investigation, which argues for minimizing “false negatives” (i.e., Type II errors of incorrectly concluding there are no effects) in order to avoid prematurely diverting from a promising line of inquiry.

Selected affiliate agencies (n = 20) that met the above criteria were invited to attend an informational presentation about the study. Eight agencies joined the study after expressing interest following the presentation, and then one of these agencies ultimately dropped out prior to initiation of participant enrollment.

All case managers in both study conditions are eligible and recruited for study participation. Their voluntary research involvement is separate from their role in delivering the intervention (for those in the treatment group) as part of their role as case managers. Case managers are recruited for the study via email by research staff, and those who are interested complete an online consent form (See Appendix).

Youth are eligible if they: 1) are between 9 and 17 years old; 2) identify with at least one of the following racial or ethnic groups: African American/Black, Hispanic/Latino, Native American, Asian American/Pacific Islander, Middle Eastern/North African or Arab American; 3) are newly applying to a community-based program at a participating agency and are later matched with a mentor who consented into the study, or are in an existing youth-mentor dyad (of no more than 2 years in duration) in a community based program in which the mentor has consented into the study and is in the caseload of a case manager already in the study; and 4) do not have a cognitive or learning disability that would interfere with completing study surveys. Parents of enrolled youth are eligible if they are able to complete study measures in English or Spanish. Typically, agency staff inform families about the study, and then those who are interested are referred to the research team. Research staff then conduct informed consent and assent procedures with caregivers and youth in person or on the phone. In some cases, agency staff who have been trained in human subjects research ethics conduct the consent and assent procedures. Consent and assent forms are available in English and Spanish and are completed online or on paper.

Mentors are eligible for the study if they are newly applying to a participating agency and are later matched with a study-eligible youth or if they are in an existing match of up to 2 years with a study-eligible youth and in the caseload of a case manager already in the study. Agency staff inform eligible mentors about the study and then refer them to the research team. Research staff conduct informed consent, and consent forms are completed online.

Intervention

The JEDI-CSR intervention was developed collaboratively by the research team and BBBSA staff. Input was also gathered from expert researcher and practitioner consultants and informal group discussions with youth from each of the participating agencies. Training and ongoing support are offered to both the mentors who receive the intervention and case managers who deliver the intervention.

Treatment mentors

Treatment mentors receive the JEDI-CSR intervention series over 12 months, which includes: (1) three self-paced online trainings; (2) an online live training; (3) booster emails; and (4) JEDI match support calls. Mentors are compensated $50 for completing the whole training series.

Self-paced online training

The self-paced training is available on the Learning Exchange, a platform managed by BBBSA and available to all agencies. The training is divided into 3 parts, each taking approximately 30 min to complete. The first module is designed to orient mentors to BBBSA’s youth empowerment and JEDI frameworks in their mentoring relationships. The orientation introduces key historical racial events in the U.S. and defines concepts such as race, ethnicity, and racism. The second module focuses on social identities. Mentors build an awareness of their own multiple identities (emphasizing ethnic and racial identity) and how these identities may influence their mentoring relationship. Key concepts that are introduced include social identity and being ‘color brave’ by not avoiding conversations about ethnic and racial identity. The final module aims to enhance mentors' understanding of how social group memberships influence one’s experiences and relationships. Mentors are prompted to reflect on how their identity groups are connected to systems of privilege and oppression and how these identity groups may differentially impact themselves and their mentee. Mentors are encouraged to complete the training at their own pace within months one to three of their study participation (or start of their match in the case of newly matched mentors), and reminder emails are sent to mentors who have not completed them within the requested timeline.

Live online training

During months 3 to 4 of their study participation or start of their match, mentors are asked to attend a live Culturally Smart Relationships (CSR) training via Zoom that lasts 1.5–2 h. The purpose of this training is to help mentors learn and practice strategies for integrating JEDI principles into their match. Topics include how to support youth of color with developing and maintaining a positive ethnic/racial identity and specific strategies for active listening that can be used in supportive conversations about racism and more generally to promote youth-centered relationships. Mentors practice these strategies with their peers. The training is co-led by two trained BBBS agency staff members and includes presentation by facilitators, small and large group discussions, use of Zoom features (i.e., polls, chat), and role plays in break-out rooms. A webinar version of the CSR training is also made available to mentors who have not completed the training after 4 months of study participation.

Booster emails

Upon random assignment of a match to the treatment condition, the mentor begins receiving JEDI support emails from the research team using MailChimp. In the first five months, mentors receive emails that orient them to the intervention, give instructions on how to access training materials, and provide reminders about upcoming training. Throughout the 12 months of the intervention, mentors receive 13 emails (roughly once a month) that are aligned with the intervention series to reinforce content presented in the training, provide additional resources, and suggest relevant mentoring activities. Relevant mentoring activities for different youth age groups are included in these booster emails.

JEDI match support calls

As part of support calls with mentors, which take place monthly during the first year of the match in the BBBS program, case managers provide treatment mentors with support to apply JEDI-CSR concepts and skills. Case managers are also encouraged to ask questions about mentors’ application of JEDI-CSR training content in their mentoring relationship. The support calls reinforce content from the trainings and are meant to help the mentor support their mentee in the development of their ethnic/racial identity, engage in activities and conversations relevant to their mentee’s race/ethnicity (e.g., read relevant books together, attend cultural museums), and work through JEDI-related challenges or barriers.

Treatment staff

In addition to receiving the same self-paced training given to treatment mentors as described above, case managers attend: (1) an initial case manager training; and (2) quarterly discussion sessions. Case managers complete intervention activities as a part of their job responsibilities (except where precluded by state law).

Staff training

All treatment case managers and staff designated to facilitate the live CSR training participate in a two-hour JEDI-CSR Orientation held via Zoom. The session provides an overview of the research study and an introduction to the CSR training content. Treatment case managers then participate in an additional two-hour JEDI-CSR Match Support training held via Zoom. This session covers the logistics of providing JEDI match support, an overview of the JEDI-CSR content of match support calls, and practice applying the content to scenario examples. Finally, CSR training facilitators attend a training on how to facilitate the live CSR training.

JEDI quarterly discussion groups

All treatment case managers are asked to participate in quarterly JEDI discussion groups (i.e., ten total sessions). Sessions are conducted via Zoom and last approximately one hour. Research team members and two experienced mentoring practitioners facilitate the sessions, which include project updates and discussion of successes and challenges with providing JEDI match support to matches on their caseload. While this general structure is used for each session, specific discussion prompts vary based on the stage of the project and ongoing needs expressed by participating agency staff.

Control participants

Control case managers provide their usual supervision to control mentors on their caseload for the duration of the study. This includes monthly support calls by phone. They do not participate in any of the ongoing study meetings or self-paced trainings. Control mentors receive “services as usual” and have access to the usual trainings available to all BBBS mentors and general monthly support calls with their case manager. They do not have access to any of the JEDI trainings (self-paced or live) or ongoing support described above. Services-as-usual was selected as the choice for control in order to examine if the intervention changed mentor, relationship and youth outcomes above and beyond what the agencies typically provide.

Quantitative methods

Survey procedures

Mentors complete online surveys at three time points: baseline (prior to randomization), 4 months after baseline or start of youth-mentor match, and 12 months after baseline or start of youth-mentor match. In youth-mentor dyads that end prior to 12 months, the mentors are asked to complete a survey when the match ends, which includes the measures asked at 12 months. Case managers complete a baseline survey prior to being informed of randomization results and then a follow-up survey 18 months later. Case manager surveys include questions about their racial and cultural humility attitudes, caseload, and previous training experience. The surveys completed by treatment case managers also include questions about their experience with the intervention. Upon study enrollment, youth complete an online baseline survey and then a 12-month survey. Youth complete the surveys with a researcher on the phone who either reads the questions aloud as the youth completes the survey silently or remains silent and available in case the youth has questions. Youth outcomes will also be gathered from BBBS case records, which are collected by agency staff through the Youth Outcomes Survey (YOS) and Strength of Relationships (SOR) survey. If the staff do not administer these surveys to a youth (e.g., because the youth withdrew from the BBBS program) and the youth consented to be in the study, then the research team administers these surveys at 12 months. Each survey takes about 15—25 min to complete, except the 12-month youth survey which takes about 40 min. Participants receive gift cards for completing each survey: parent baseline $10, child baseline $20, child 12-month $20, mentor baseline $10, mentor 4-month survey $20, mentor 12-month survey $20, case manager baseline $10, case manager 18-month survey $15. Youth also receive a $10 gift card in between the baseline and 12-month survey to stay in touch and keep the family engaged in the study.

Measures

Information about the primary and secondary outcome measures is provided in Table 1.

Table 1 Description of primary and secondary outcome measures

Intervention assessment and fidelity

Completion of the training activities among treatment mentors is tracked bi-weekly monitoring on the Learning Exchange, the BBBSA platform. Attendance at the live CSR training is monitored in the Zoom meeting attendance logs. The 12-month mentor survey also includes questions about their receipt of the trainings and ongoing support, the extent to which they implemented recommendations and found them feasible, and perceptions of the quality of the trainings, monthly JEDI supervision calls and booster emails.

Training facilitators complete a fidelity measure after each training to record the extent to which they covered each of the training topics and activities. They also report their experience in preparing for and facilitating the training, their reflections about what worked well and what was challenging, and recommendations for improving the training. Researchers will review the audio recordings, chat transcripts and poll responses from the live Zoom trainings and complete a measure of intervention fidelity that assesses adherence, exposure, facilitator quality and participant engagement.

After each match support contact, treatment case managers complete a brief survey to report on the mentor's engagement with intervention-related activities in the last month. The survey consists of two multiple-response-option items in which case managers report on: JEDI activities and discussions the mentor incorporated into their time with their Little (e.g., using PLAY (Pause, Listen to Learn, Affirm Youth), supporting their Little's ethnic/racial identity); and JEDI monthly email suggestions used (e.g., activity, reading, or reflection for the Big to do independently). A scaled response item measures the extent to which their contact focused on JEDI-related topics (1 = not at all to 4 = extensively), and a single-choice item asks case managers to evaluate the mentor's current engagement with JEDI (e.g., not yet engaged—open to it, trying it—finding it challenging). An open-ended response item captures any additional information the case manager would like to share.

Qualitative methods

Participants

Study participants who gave permission at enrollment to be contacted for potential participation in interviews will be recruited to participate in the qualitative phase of the study by email, text, and phone. We will recruit approximately 24 youth-mentor dyads (18 treatments and 6 controls; the majority of the dyads will be cross-race), the youth’s parent/guardian, and 12 to 18 case managers (half to two-thirds in the treatment group) who supervise at least one participating dyad. Baseline data will be used to select mentor-youth dyads (e.g., age, gender, mentor’s self-efficacy, youth’s cultural mistrust) and case managers (e.g., number of years working at BBBS, awareness of racial privilege). Participants will be given information about the interviews and asked to complete online consent and assent forms via REDCap or hard copy as needed. Parental consent forms will be available in English and Spanish. After the consent and assent forms are completed, the research team will contact participants to schedule the interviews.

Photo elicitation

To facilitate dialogue about the intervention and the development of the mentoring relationship, photo elicitation [67] will be incorporated into the individual and dyadic youth-mentor interviews. Prior to each interview, youth and mentors will be asked to share three photos or images that reflect their mentoring relationship (e.g., how the mentor supports youth, a challenging time in their relationship) using a secure link provided by the research team.

Interview procedures

All interviews will be conducted via Zoom or phone and audio recorded for transcription. Two one-on-one interviews will be conducted with youth and mentors at 4 to 8 months and then at 12 months into study participation to examine relationship development over time. Participants will receive a $20 gift card for the first interview, and a $25 gift card for the second interview. Parents will participate in an individual interview at the 12-month time point, and case managers will participate 18 months into their study involvement to examine their experience with the intervention. Each of the individual interviews will be about 30 to 60 min, and parents and case managers will each receive a $25 gift card. If both the youth and mentor within the dyad consent to participate in the qualitative study, then both will be invited to participate in an additional 30- to 45-min dyadic interview on Zoom 12 months after their start of study participation. Youth and mentors will each receive a $15 gift card for this interview. All interviews will be audio-recorded and transcribed.

Interview measures

The first individual interview of mentors and youth starts by asking the participant to describe the submitted photos that reflect their relationship. Then they are asked questions about their relationship progress and racial dynamics (e.g., how racial/ethnic differences or similarities between mentor and youth impacts their interactions, how race/ethnicity comes up in conversations and activities) within their relationship. Treatment mentors are asked an additional series of questions about their experiences with the intervention activities, supports and resources, and their effects on their mentoring relationship.

The second individual interview with mentors and youth will also include discussion of the submitted photos/images, and then they will be asked to reflect and discuss important events, turning points, and challenges that took place in their relationship in the last 12 months. The remaining portion of the interview will include questions developed from preliminary thematic analysis of the first interviews and exploration of survey data. For example, questions may ask participants to discuss what they would have done differently, how they think the relationship has changed over time, how racial/ethnic differences or similarities between the mentor and youth has impacted the relationship, comfort level and challenges between the mentor and youth’s family, and benefits of being in the mentoring relationship. Mentors in the treatment condition will also be asked about their experiences with the intervention.

Case managers will be asked to complete one interview about 18 months after the start of their study participation. All case managers (treatment and control) will be asked about their perceptions of how racial differences impact mentoring relationships, how mentors support youth’s ethnic/racial identity, mentors’ comfort in talking about race and identity in their matches, racial challenges and conflicts in mentoring relationships, mentors’ and youth’s engagement in activities that are relevant to race/ethnicity, concerns and feedback parents raise related to race, their comfort in supporting mentors on areas of race and identity, and areas in which case managers need more support. Treatment case managers will be asked whether intervention components seemed to influence attitudes among the mentors they supervise, the intervention-related activities in which mentors and youth engage, and mentors' approach to these interactions and youth’s responses. Treatment case managers will also be asked about their experience delivering the intervention and its impact on the match support they provide.

Parents/guardians will complete an interview 12 months into their child’s study participation. The interview will focus on the development of the relationship between their child and the child’s mentor, racial dynamics, and conversations or activities with the mentor and child related to race, ethnicity, or social justice.

Dyadic interviews will focus on how the mentoring relationship has changed over time and how their social identities have affected their relationship. Youth and their mentors will be asked to bring three photographs to the interview respectively to describe their mentoring relationship. The dyad will be asked to discuss how they would respond to a hypothetical scenario about a group that is protesting in their local community because a group of youth of color was kicked out of a park for loitering.

Quantitative analyses

Quantitative data will be analyzed to test hypothesized effects of the intended enhancements on both primary and secondary outcome measures (see Table 1). These analyses will be conducted both from an “intent-to-treat” (ITT; i.e., the effect of assignment to the group receiving the intended enhancements regardless of exposure/participation level) and “treatment-on-the-treated” (TOT; i.e., intervention effects when a designated threshold of sufficient exposure or participation is met) perspective. For ITT analyses, regression analyses will test for a group difference (i.e., those assigned to receive standard services and the intended enhancements, respectively) in each outcome at the relevant time of follow-up (4.5 or 12 months), controlling for baseline level of the outcome, planned covariates as assessed at study baseline (youth gender, family low-income status, single- vs. two-parent household, status as child of an incarcerated parent, level of involvement in organized out-of-school activities, youth and mentor age and race/ethnicity, and mentor reports of prior experience working with youth, volunteering, receiving cultural humility and/or social justice training, contact with individuals of various racial/ethnic groups), and any other baseline measures for which there is a standardized mean effect size difference between treatment and control groups of 0.05 or greater, per What Works Clearinghouse Standards [68]. The foregoing analyses also will be conducted for the post-training measures of mentor attitudes and behaviors to assess short-term effects of the training on these outcomes. To control for selection bias for who receives the threshold dose of the intervention, TOT effects will be investigated using Complier Average Causal Effects (CACE) models that are estimated as mixture models. The threshold for sufficient intervention participation/exposure will be that a mentor whose match was assigned to the intended enhancements group completed both the self-paced and live training components of the intended enhancements. Given the exploratory nature of the trial, no adjustments are planned for testing of multiple outcomes.

To account for potential non-independence of data associated with nesting of participants within agency and case manager, standard errors will be adjusted in all tests of significance using the Type = Complex option in Mplus. Missing data will be handled by multiple imputation or, where feasible such as in the SEM analyses, using the asymptotically equivalent approach of full-information maximum likelihood (FIML). Analyses will take into account the distributional properties of each outcome measure (e.g., ordinal regression will be used when responses are discrete and ordered but not plausibly continuous and right censoring will be specified for match length when examining it as an outcome to account for matches that have not yet ended).

Several additional sets of analyses also will be conducted. First, to explore potential moderators of intervention effects, we will use the regression trunk approach (RTA) as implemented in the R package STIMA. RTA is appropriate when strong a priori hypotheses about intervention effect moderation are lacking, as is the case in this research. It has the advantage of enhancing sensitivity to relatively more complex patterns of program effect moderation (e.g., involving two or more moderators). Candidate moderator variables in these analyses will align with those included in the theory of change shown in Fig. 1 (e.g., youth age to allow for potential developmental differences in effects).

Second, analyses will be conducted to explore indications of potential effects of the training that case managers receive, combined with their experience delivering components of the intervention to mentors, on the measures of cultural humility, social justice and racial attitudes that they complete at the end of the study intervention period. These analyses will be conducted using both an ITT approach (i.e., comparing case managers assigned to the standard services condition and those assigned to deliver the intended enhancements to study matches on their case loads) and a TOT approach that takes into account varying thresholds of involvement in intervention delivery (e.g., number of study matches on the case manager’s caseload).

Third, fidelity of implementation for the intended enhancements will be examined using relevant data (e.g., the facilitator self-ratings and observer ratings will be used to assess implementation fidelity for the live mentor training session). To the degree that there is noteworthy variability on fidelity measures, analyses will also examine potential associations of these measures with outcome measures to inform understanding of potential implications of implementation fidelity for effectiveness of the intended enhancements.

Finally, structural equation modeling will be used to evaluate the hypothesized theory of change for the intended enhancements (Fig. 1). Where available, multiple measures will be used as indicators of model constructs (i.e., latent variables) to enhance reliability and baseline scores on measures (e.g., youth outcomes) will be controlled for to better support causal inference. The primary model tested will include only paths among latent variables that are consistent with the theory of change as shown in Fig. 1. A final best-fitting model that incorporates any modifications of the hypothesized model that are both empirically and theoretically justified will be identified. Determination of the best-fitting model will be based on combined consideration of the following model fit indices: Comparative Fit Index (CFI), Root Mean Square Error of Approximation (RMSEA), and Root Mean Square Residual (RMSR). Minimum criteria for adequate fit will be CFI > 0.95, RMSEA < 0.05, and RMSR < 0.08 [69].

Qualitative analyses

A case study approach will be used to organize and analyze our qualitative data. To develop descriptive accounts of each mentoring relationship, all relevant data for each dyad will be compiled to review as a “case,” which will include: (1) interview transcripts of the mentor (4–8 & 12 months), youth (4–8 & 12 months), caregiver (12 months), and case manager (if applicable; 18 months); (2) transcript of the mentor-youth dyad video interview (12 months); (3) photos provided by youth and mentors; and (4) case manager supervisory notes. Two research team members will review these data for each case and write a rich description of the mentor and youth, the youth’s family and neighborhood context, mentor’s experience with the intervention (if in the treatment group), the mentor’s and youth’s attitudes and how they changed over time, features of the mentoring relationship, and how the youth appears to have benefited from the relationship. As part of the case description, a timeline will be created to capture key moments and events in the mentoring relationship and intervention components to discern potential connections.

​​The qualitative data will be coded thematically [70] using Dedoose, a cloud-based, qualitative software program, by a team of at least two coders. Following a multi-step approach, the coders will first familiarize themselves with the data by reading through the transcribed interviews and the case description of each youth-mentor dyad. In this initial review, interview text that is relevant to the research questions will be highlighted and grouped into repeating ideas. Coders will independently develop initial codes based on the repeating ideas and develop a codebook. The repeating ideas will be organized by the topics addressed in the interviews (e.g., learnings from training and supports, relationship challenges, perceptions of how mentors support youth in their race/ethnicity). These codes will be refined through an iterative process, and the final codebook will be used to code all interviews. Codes will be sorted into conceptually clustered matrices [71] organized by focus area to identify themes and sub-themes and examine associations among them.

Mixed methods integration

Mixed methods data integration will also be conducted. The themes that emerge in the qualitative analysis will be examined in conjunction with scores from the quantitative measures to achieve a contextualized understanding of the constructs examined. A case-oriented merged analysis display will be constructed in which baseline and follow-up scores associated with the mentor-youth dyad and individual interviews will be arrayed along the scales for select measures. For example, themes from mentors’ descriptions of their relationship challenges will be displayed alongside their scores on the scales used to measure relationship challenges. Results of one method also will be used to inform data analysis in the other method. For example, initial quantitative results based on the baseline and 12-month survey data may direct us to examine specific themes in the qualitative data to better understand significant (or nonsignificant) differences in the quantitative data. We will also use the qualitative data about the experiences of mentors in the intervention (e.g., aspects of the intervention that worked well or did not work well, why, and what could be done differently) to better understand the quantitative results. Finally, the qualitative findings may point us to specific moderators or mediators that should be examined in the quantitative data to help us understand for whom the intervention is more effective or the processes by which the intervention strengthens positive youth resources. Whenever possible, meta-inferences will be made about the impact of the intervention on mentors, youth-mentor interactions, and youth outcomes based on both the qualitative and quantitative data.

Discussion

The aim of this study is to investigate an innovative set of training and ongoing support for volunteer mentors that orients them and provides them with strategies for integrating cultural humility in their mentoring relationships with youth of color. The goal of the JEDI-CSR intervention is to foster mentors’ skills to support youth of color in developing a positive ethnic/racial identity and enable them to have supportive conversations with mentees about racism and social justice issues. The study utilizes a mixed-methods design that includes an RCT to rigorously test the effects of the intervention on mentors’ attitudes, their interactions and relationships with youth of color, and youth outcomes. It is expected that the intervention will improve youth outcomes through enhanced mentor cultural humility, more culturally attuned mentoring interactions, and greater mentoring relationship quality. Outcomes relevant to the healthy development of youth of color will be examined, such as strengthened ethnic/racial identity, sense of mattering, coping efficacy with discrimination, and academic self-efficacy. Integration of quantitative and qualitative data will aim to better understand whether and how the intervention works in changing mentor attitudes, mentor-youth interactions and relationship quality, and youth outcomes.

There are several limitations in this study. First, the national attention on JEDI topics and state legislation across the U.S. to restrict JEDI efforts (e.g., banning instruction of race and diversity in Florida) may influence the types of participants who enroll in this study, potentially biasing the sample towards mentors inclined to learn about JEDI topics, and youth of color and families for whom race and ethnic/racial identity are important. Second, we will have limited youth outcome data due to the inclusion of study matches in which youth decline participation and consequently will not complete any of our surveys. The relatively smaller sample size will also limit the number of study matches that consent to be in the qualitative study which might lead to less variability in relationship quality, intervention experiences, and participant demographic characteristics in the qualitative sample. Another limitation is that the study is limited to youth who come from families who speak English and/or Spanish, impairing the generalizability of the findings to families speaking other languages. Finally, we are not conducting long-term follow-ups of intervention effects on mentoring relationships and youth.

Despite these limitations, there are study strengths. The mixed methods design will allow us to examine how the intervention works and why the intervention is effective or not. Further, surveys and interviews are administered with various sources, including youth, parents, mentors and case managers, and as such, triangulation of data will strengthen study findings. The multiple time points during a 12-month period will allow us to understand the intervention’s impact on relationship trajectories. For example, mentors and youth may not engage in conversation or activities related to identity early in the relationship as they get to know one another but they may feel comfortable to do so later in the relationship. Results from this investigation will have implications for understanding how these types of interventions work despite societal backlash against JEDI in the U.S.. Finally, this investigation provides insight on how to conduct and improve this type of intervention in community-based settings.

Given the moderate effects of youth mentoring programs [2,3,4] and that the majority of youth served in mentoring programs are children and adolescents of color [5], this culturally tailored training and support intervention for volunteer mentors may be one way to increase the effectiveness of these programs. The study findings will not only have implications for youth mentoring programs but also for other settings (e.g., schools, after school programs) in which children and adolescents form relationships with adults, particularly in how to support adolescents of color in their positive ethnic/racial identity development and in how to have supportive conversations around racism and social justice topics.