Preliminary evaluation of a gamified smartphone intervention (O2O-PEP) for enhancing HIV post-exposure prophylaxis uptake in men who have sex with men: pilot feasibility study

Abstract

Background

Addressing the burden of human immunodeficiency virus (HIV) infection among men who have sex with men (MSM) has become a major priority. Over the years, post-exposure prophylaxis (PEP) has proven efficacy in preventing HIV transmission. However, the widespread underutilization of PEP undermines its protective potential. To address this issue, we developed an innovative, gamified mini-app embedded in WeChat, known as O2O-PEP (online-to-offline collaborative model for HIV postexposure prophylaxis), that targets PEP uptake.

Objective

We aimed to conduct a pilot feasibility study to assess the feasibility, acceptability, and preliminary efficacy of O2O-PEP in promoting PEP uptake among Chinese MSM.

Methods

A single-arm, pre-post feasibility study combing quantitative and qualitative methods for data collection was performed in Qingdao, China. Eligible MSM were invited to use the O2O-PEP for 2 weeks. The mini-app provides an online-to-offline service model, enabling MSM to access PEP-related character-based narratives, share on moments, sexual transmitted disease picture test quizzes, e-Consultation, and referrals to offline PEP service providers. Additionally, gamification features, such as reward-based engagement and social support, are incorporated to enhance user motivation. Intervention feasibility was tested by looking at participant engagement data. The acceptability of the intervention was explored with System Usability Scale (SUS) and in-depth interviews. Preliminary efficacy was measured by assessing HIV PEP knowledge and HIV PEP uptake.

Results

A total of twenty Chinese MSM were successfully enrolled. All participants completed baseline outcome measures, and 19 (19/20, 95%) completed outcome measures at 2 weeks. Participants had a mean age of 29.65 years (SD 9.17). PEP knowledge scores increased from 7.79 (SD 1.69) at baseline to 9.68 (SD 2.86) at the two-week follow-up, reflecting a large Cohen’s d effect size of 0.8. One participant successfully accessed PEP within 12 h after engaging in condomless anal sex with a stranger during the intervention period. On average, participants spent 35.84 min using the mini-app over a two-week period, though app usage declined over the study period. Acceptability ratings ranged from moderate to extreme across several dimensions, including satisfaction, usability, recommendation of the mini-app to others, and user-friendliness. Ratings for the difficulty of using O2O-PEP was somewhat lower. Regarding the SUS score, 89.5% of participants rated the mini-app within the acceptable usability range (≥ 50), indicating that the majority found the mini-app to be user-friendly. Qualitative analysis of follow-up assessments identified areas where O2O-PEP needed to be improved to enhance user engagement, including a desire to additional functions of psychological and sexual transmitted diseases counseling.

Conclusions

This study pioneers the application of gamification to PEP uptake, demonstrating the feasibility and acceptability of O2O-PEP. A further efficacy trial is currently underway.

Trial registration

This study was prospectively registered at the Chinese Clinical Trial Registry (No. ChiCTR2200062538) on 11 Augst, 2022.

Introduction

Men who have sex with men (MSM) constitute the demographic most severely affected by human immunodeficiency virus (HIV) and acquired immune deficiency syndrome (AIDS) globally [1]. Between 2010 and 2022, HIV infection rates amongst MSM has increased by 25% [2], with an HIV prevalence of 8.5% in 2023, which is 9.6 times higher than that in the general population [3]. In China, MSM faced an HIV prevalence of approximately 8% over the past five years [4], a significantly higher rate compared to other high-incidence key groups. Addressing this persistent HIV epidemic necessitates innovative HIV prevention strategies tailored to the unique needs of MSM, such as the needs for MSM-friendly sexual health services.

A series of new tools complementary to the use of condoms have been introduced for comprehensive prevention with emphasis on biomedical prophylaxis, such as pre-exposure prophylaxis (PrEP) and post-exposure prophylaxis (PEP). PrEP is increasing in the world, but not reaching all populations at risk, for example, the proportion of PrEP usage among Chinese MSM in 2021 was only 18% [5], thus additional preventions are still needed. HIV PEP has demonstrated high efficacy in preventing HIV infection following a recent exposure [6, 7] and has been widely adopted in numerous countries [8, 9]. The guidelines from the World Health Organization recommended people with suspected or unknown exposure to HIV take PEP as early as possible, ideally within 24 h but no later than 72 h [10]. However, PEP remains underutilized as an HIV prevention strategy in China. Chinese Center for Disease prevention and Control (Chinese CDC) released its first guideline for HIV PEP in October 2020. The guideline delineated the criteria for PEP eligibility, which are as follows: individuals should test negative for HIV, the exposure should have occurred within the preceding 72 h; there should be an assessed high risk of acquiring HIV post-behavioral evaluation, and individuals should consent to adhere to the PEP regime and subsequent follow-up procedures [11]. Despite strong interest, studies consistently showed low PEP uptake among Chinese MSM [12,13,14]. Factors hindering PEP scale-up include low awareness of HIV infection risks and concerns among HIV healthcare providers that PEP availability may encourage high-risk behaviors or contribute to HIV drug resistance among MSM [12, 15]. PEP’s limited accessibility, with challenges such as waiting times in offline designated hospitals for HIV treatment, part-time medical staff providing PEP services, and the absence of specialized PEP service center, exacerbate the demand-supply conflict [16]. Therefore, exploring alternative PEP delivery models is imperative.

With the widespread usage of smartphones globally, MSM have shown high engagement with mobile technology, particularly for health-information seeking, dating, and mobile health (mHealth) interventions. This highlights the potential of mHealth as a scalable, cost-effective, and user-friendly approach to health service delivery [17]. In December 2021, the Chinese expert consensus on HIV-related medical services by online health platforms provided a foundation for collaboration between researchers and app developers [18]. This collaboration aimed to integrate innovative elements, including gamification, into apps to reduce risks of acquiring HIV among MSM. Gamification involves applying game components (e.g., leaderboards, gaining points, and badges) to nongame contexts, leveraging the motivational power of health behaviors within an engaging framework [19]. While previous studies have explored gamification in promoting desired health behaviors, such as physical activity and cognition training [20, 21], our literature review as of January 2024 found no randomized controlled trial (RCT) was conducted to evaluate the effect of the gamified digital intervention on PEP uptake [22]. Most of the current studies explored the status of PEP uptake and its associated factors among MSM [14]. In 2017, Chinese CDC piloted a project to explore a digital intervention combing HIV self-risk assessment with online medication supply, but its impact on PEP uptake was still uncertain [23]. To bridge this gap, we have developed a smartphone intervention, initially as a WeChat mini-app named “O2O-PEP” (online-to-offline collaborative model for HIV postexposure prophylaxis) [24], incorporating gamification components to enhance PEP uptake. In brief, O2O-PEP includes several features, which mainly include (1) PEP related medication aide, directing participants to an online health platform “HeHealth” or an offline designated hospital capable of offering PEP; (2) Share on Moments, which provided a platform for asynchronous peer-to-peer interaction and social connectivity; (3) Character-based narratives; (4) Sexual transmitted diseases’ (STDs) quiz; and (5) E-counseling. An efficacy study is currently underway to evaluate the impact of O2O-PEP in promoting HIV PEP uptake among Chinese MSM. Prior to this, we conducted this feasibility study using a single arm, pre-post feasibility study design to assess the acceptability, feasibility, and preliminary efficacy of O2O-PEP over a two-week period. This feasibility study aimed to collect user feedback to refine the prototype and inform future real-world scale-up efforts. We hypothesized that O2O-PEP would be acceptable, feasible, and demonstrate preliminary efficacy.

Methods

Study design

A single-arm, pre-post feasibility study combing quantitative and qualitative methods for data collection was performed in Qingdao, China, to assess the acceptability and feasibility of the intervention [25]. The city Qingdao was selected for our study setting as our previous survey has found the proportion of PEP usage among MSM in Qingdao was still very low [26], which would contextualize the need for this intervention. This study was prospectively registered at the Chinese Clinical Trial Registry (No. ChiCTR2200062538) on 11 Augst, 2022, and the study protocol was previously published [24].

Ethical considerations

Ethics approval for this study was obtained from the institutional review board of Binzhou Medical University (IRB# 2021-007). Our study adhered to the Declaration of Helsinki. Written informed consent was retrieved from all study participants upon enrollment in the study. Data were deidentified before analysis. No additional compensation was provided for participants.

Participants

Participants were recruited through online and offline advertisements, as well as staff recommendation from a local community-based organization (CBO), between May and June 2023. To ensure sample diversity, participants were purposively enrolled from both age groups (under 30 and 30 or older) and included individuals with and without previous experience with digital interventions. Additionally, we intentionally recruited individuals familiar with CBO staff, such as those who have previously attended events organized by the CBO. This criterion was included to ensure participants felt a sense of safety and were well-equipped to provide enrich information regarding the use of O2O-PEP mini-app. Potential participants underwent assessment by CBO staff through phone or in-person interactions. Those expressing interest provided signed informed consent and underwent eligibility confirmation before enrollment. Inclusion criteria were as follows: (1) age 18 years or older, (2) assigned male at birth, (3) self-reported engagement in anal or oral sex with men, (4) HIV-negative status, (5) no pre-exposure prophylaxis (PrEP) use within one week before enrollment, (6) ownership of an iOS or Android smartphone compatible with the O2O-PEP mini-app, (7) residency in Qingdao, and (8) proficiency in Chinese. Exclusion criteria included a history of medical or psychiatric conditions that could interfere with study participation. We would stop recruit participants based on data saturation through ongoing discussions of interviews, and we finally recruited 20 MSM, this is consistent with research which showed saturation is reached with about 12 interviews [27, 28].

Pilot test procedures

Collaborating with the local CBO, we integrated the O2O-PEP mini-app into an existing WeChat public account (Qingdao Qingtong Anti-HIV/AIDS Volunteer Service Center) to enhance its acceptability, scalability, and potential for widespread implementation. The CBO was selected based on its extensive experience serving MSM since 2015, and its prior involvement in research projects also facilitated smooth collaboration with the study. CBO staff provided a demonstration of the app’s functions and operations to participants. To restrict access to O2O-PEP to eligible study participants, a unique PIN was provided. After entering the PIN, participants completed the electronic informed consent for study participation, and underwent a baseline assessment. Subsequently, participants navigated to the O2O-PEP home screen, featuring six key function icons targeting PEP uptake: my profile page, share on moments, character-based narratives, STDs picture choice quizzes, medication aide, and e-consultation. Table 1 lists detailed intervention components within each function icon. Figure 1 presents a screenshot example of the O2O-PEP mini-app.

Table 1 Description of the O2O-PEP intervention mini-app

Participants were allocated a 2-week period to engage with the mini-app, with strong encouragement to utilize it on a daily basis. This 2-week period duration allowed participants to interact with various app features and identify any significant usability issues, while maintaining a focus on short-term usability rather than long-term engagement. Throughout this duration, participants were not only given the flexibility to communicate with CBO staff and researchers at any time but were actively encouraged to do so. Upon the conclusion of the 2-week pilot trial, all participants were invited to participate in a posttest computer-assisted self-interview survey. Additionally, they took part in a qualitative exit interview conducted online through licensed videoconferencing software (Tencent meeting).

As an incentive for engagement, participants received rewards in the form of points and game cards for specific activities carried out within the mini-app. These activities included earning 1 point for each login and check-in, 2 points for watching one story, and 2 points for creating a post or leaving a comment in the Share on Moments section. Furthermore, participants had the option to redeem their accumulated points and game cards for in-kind goods such as lubricant or JD Ecards which are similar with the Amazon gift cards.

Fig. 1

A screenshot example of an intervention component of the O2O-PEP mini-app, featuring character-based narratives. The text above reads: “the two met at a hotel in the evening…”, and the text below states: “Fear after passion.”

Assessments

Our study aimed to evaluate the feasibility, acceptability and preliminary efficacy of the O2O-PEP intervention. Specifically, feasibility was assessed through participant engagement data on O2O-PEP, while acceptability was assessed by a 4-Likert point intervention acceptance scale, SUS score, and an in-depth review. Preliminary efficacy was evaluated based on PEP knowledge scores and uptake of PEP during the two-week follow-up visit.

Participant characteristics

Baseline and follow-up assessments were self-administered through a Web-based survey. Upon securing informed consent, participant information was collected, including demographics (age, marital status, education, personal monthly income, etc.), and potential confounders, such as sexual risk behaviors, self-perceived risk of acquiring HIV, MSM-related stigma, PEP and PrEP usage history, and psychiatric information [31, 32]. The 8-item HIV RISK Assessment tool for MSM, recommended by Chinese CDC, was administered to assess risk of acquiring HIV over the past 6 months [33, 34]. The total score of the scale ranges from 0 to 15, with participants scoring less than 5 classified as “low risk of acquiring HIV” and those scoring 5 or higher categorized as “high risk of acquiring HIV”. MSM-related stigma was assessed using the Neilands’ homosexual stigma scale, which consists of 10 items, each scored on a scale from 0 (never) to 3 (many times), with higher score indicating a greater level of stigma [35]. The severity of depression symptoms over the past 2 weeks was assessed using the 9-item Patient Health Questionnaire-9 (PHQ-9), the total score for this scale ranges from 0 to 27, with higher score reflecting greater severity of depression [36]. Overall, the baseline assessment would take 10–15 min to complete.

Feasibility of the O2O-PEP mini-app

To assess feasibility, user statistics of various app features were collected in a secure web-based cloud environment (Amazon Web Service). This included the average time spent using the mini-app, daily check-ins over the 2-week period, frequency of use of different intervention components, and the number of posts and comments in the Share on Moment section.

Acceptability of the O2O-PEP mini-app

Acceptability was assessed by five questions we adopted from previous studies [37]: what was your overall satisfaction with O2O-PEP? How usable was O2O-PEP on a daily basis? How difficult was it to learn how to use O2O-PEP? would you recommend O2O-PEP to a friend who would need it? How user-friendly was O2O-PEP? employing a Likert scale from 1 (not at all) to 4 (extremely). The System Usability Scale (SUS) was used to assess the usability of the mini-app. Participants rated 10 items on a 5-point Likert scale (from “Strongly Disagree” to “Strongly Agree”). For positively worded items, scores were calculated by subtracting 1 from the raw score. For negatively worded items, scores were calculated by subtracting the raw score from 5. The scores for each item were summed, and the total score was multiplied by 2.5 to calculate the final SUS score, which ranges from 0 to 100. Higher scores indicate better perceived usability, with scores above 50 considered acceptable [38]. Semi-structured, in-depth interviews with MSM were conducted online to qualitatively analyze their experiences and perceptions of O2O-PEP. Interview questions covered topics such as liked or disliked features, factors influencing daily use, suggested improvements or additions, and the likelihood of future use. The interview questions can be found in Multimedia Appendix 3. All interviews were audio recorded and transcribed verbatim with participants’ consent.

Preliminary efficacy

PEP knowledge was assessed using a 11-item PEP knowledge scale at both baseline and follow-up visits [39]. The total score of the scale ranges from 0 to 11, with a higher score reflecting greater level of PEP knowledge. Information regarding PEP uptake was collected during the follow up.

Data analysis

Descriptive statistical analysis was employed to portray participant socio-demographics, acceptability, feasibility, and preliminary efficacy. Means and standard deviation (M, SD) were calculated for continuous variables, while frequencies (proportions, %) were determined for categorical variables. Specifically, intervention acceptability was evaluated through two approaches. First, individual responses to acceptability questions (satisfaction, usability, difficulty, willingness to recommend, and user-friendliness) were descriptively analyzed. Second, SUS scores equal to or exceeding 50 were considered acceptable. For qualitative interview data, participant comments were systematically summarized and examined for themes related to mini-app functions. Two researchers utilized an open coding approach to categorize and analyze the data. Relevant quotes for each theme were included in the manuscript. For the assessment of preliminary efficacy, the PEP knowledge score was compared between baseline and the two-week follow-up visit. Cohen’s d was used to quantify the effect size between the two time points. Cohen’s d is calculated as the difference between the means of PEP knowledge scores of the two time points divided by the pooled standard deviation:

$$\:d=\frac{{M}_{1}-{M}_{2}}{{SD}_{pooled}}$$

Where M₁ and M₂ represent the means of PEP knowledge score at baseline and the two-week follow-up, respectively, and SD_pooled is the pooled standard deviation. According to Cohen’s guidelines, an effect size of 0.2 is considered small, 0.5 represents a medium effect size, and 0.8 or higher indicates a large effect size [40]. Microsoft Excel 2019 (Microsoft Corp., WA, USA) facilitated data management and analysis throughout the study.

Results

Study population characteristics

A total of 20 participants were successfully enrolled during the index visit. Baseline characteristics of the study population are presented in Table 2. The participants had a mean age of 29.65 years (SD 9.17, range: 20 to 53 years), and the majority identified as single (95%, 19/20). About 55% (11/20) had completed at least a college level education, while 80% (16/20) had undergone HIV self-testing, and 90% (18/20) had received facility-based testing in the past year. A significant portion of participants had their sexual debut with a man at less than 19 years old (30%, 6/20), and 85% (17/20) perceived themselves as having no or low risk of acquiring HIV. The sample retention rate was 95% (19/20) at the conclusion of the two-week period. One participant was lost to follow-up due to refusal to participate in the review.

Table 2 Baseline characteristics of participants (N = 20)

Preliminary efficacy

The mean PEP knowledge score exhibited a significant increase from 7.79 (SD 1.69) at baseline to 9.68 (SD 2.86) at the two-week follow-up visit. This change demonstrated a large effect size with a Cohen’s d of 0.80. Notably, the proportion of participants correctly answering the item “A person does not need to take PEP after having unprotected sex with people with unknown HIV status” increased from 75% (15/20) to 89% (17/19). Importantly, one participant successfully obtained PEP from the online health platform HeHealth within 12 h after engaging in condomless anal sex with a stranger.

Feasibility

Over a two-week follow-up period, the mean total time spent using the mini-app was 35.84 min, ranging from 10 to 61 min. Participants engaged with the app for an average of 7 days, and the number of check-ins displayed a decline over the pilot trial duration (Fig. 2). A total of 45 posts were made in the Share on Moments section, averaging 3.21 posts per day, with 47% (9/19) of participants contributing at least one post during the pilot. The mean number of clicks on “narrative-based characters” and “STDs picture choice quizzes” per participant was 2.32 (SD 1.67) and 1.37 (SD 1.01), respectively. Notably, 9 participants continued to use the mini-app beyond the two-week pilot trial period.

Fig. 2

Number of check-ins of O2O-PEP (N = 20)

Acceptability

In Table 3, participants’ acceptability responses for the O2O-PEP mini-app are summarized. The average level of acceptability varied from moderate to extreme for aspects such as intervention satisfaction, usability, recommendation of the mini-app to others, and user-friendliness. Assessing the difficulty of using O2O-PEP, the average rating fell within the “somewhat” range.

Table 3 Responses on a 4-point likert scale for the intervention acceptance

In Table 4, participants’ acceptability for SUS scores are summarized. The total mean score was 60.26 (12.35). Using a threshold score of ≥ 50 on the SUS as an indicator of O2O-PEP user-acceptability, it was found that 90% (17/19) of the sample surpassed this cutoff, indicating their acceptance of the mini-app.

Table 4 Responses on a 5-point likert scale for the system usability scale (SUS)

In-depth review

In a comprehensive review involving 19 participants at the conclusion of the pilot trial, we sought to understand their perspectives on the O2O-PEP intervention. Categorizing their opinions into four distinct areas, we explored: (1) their preferences and reservations regarding intervention features, (2) factors influencing daily use, (3) suggestions for feature enhancements or additions, and (4) their likelihood of using O2O-PEP in the future.

Intervention features being liked and disliked

The overall consensus is that O2O-PEP moderately fulfills the need and expectations of its user base. A prominent feature that generated significant interest was the “Share on Moments”, with 11 participants expressing appreciation. One participant highlighted the value of this component, stating, “You can read stories from others, like how he found his sexual partner, how he was infected with HIV. Very scary, which can also warn others” [30-years old MSM]. However, not every participant favored this feature; one participant explicitly disliked it, stating, “The topics are irrelevant to my life, I don’t care” [31-year-old MSM]. Other liked features included character-based narratives (n = 10), and medication aide (n = 5). A few participants disliked these features, including character-based narratives (n = 7), and medication aide (n = 4). For example, a 21-year-old MSM stated, “the character-based narratives, combined with card-based rewards, provide limited entertainment value and interactivity. The frequent flashing of cards on the screen can be somewhat annoying”. As our intervention intended to overcome PEP inaccessibility to improve PEP uptake amongst MSM, we sought to understand why participants dislike the function of medication aide, reasons can be summarized as following: (i) the online health platform (HeHealth) webpage complicated and were unsure how to quickly access PEP (n = 2), (ii) online pre-purchase physician evaluations burdensome (n = 1), and (iii) concerns that easy access to PEP might encourage risky sexual behaviors (n = 1). Regarding character-based narratives, some participants disliked them (n = 7), 5 participants suggesting the storylines were too simple and lacked comprehensive integration of PEP knowledge, another two participants mentioned that the animation-based design was only suitable for younger MSM and less so for older individuals.

Factors affecting daily use

The majority of participants found the mini-app easy to navigate (n = 15), with comments like “It is really straightforward. You know, I even don’t need to spend more time to learn how to use the mini-app after the staff shows it to me” [21-year-old MSM].

However, four participants encountered barriers to daily use, primarily related to technical challenges. Examples included app flashback and slow loading, as expressed by one participant: “It was a little bit laggy when I first tried to click into my profile page” [23-years old MSM]. Additionally, one participant highlighted the limitation of having only one offline hospital for PEP services, stating that it was inconvenient for those living far from the designated hospital [22-years old MSM].

Additional features to be added to O2O-PEP

Eight participants expressed a desire for additional functions in O2O-PEP. These included proving information about psychological counseling and mental health service (25%, 2/8), STDs (25%, 2/8, e.g., syphilis and Condyloma acuminatum), PrEP (25%, 2/8), HIV testing locations (2/8), dating function (1/8), and more introductory information about CBOs (13%, 1/8).

Likelihood to use O2O-PEP in the future

Eleven participants expressed a positive intent to continue using O2O-PEP even after the research program concluded. One participant stated, “This is really a good app, I would recommend it to my friends if they need it” [25-year-old MSM]. However, seven participants indicated that their future use depended on factors such as their available time and the update speed of the mini-app. Only one participant stated he would never use the mini-app again due to time constraints, expressing, “absolutely not, it’s too much a hassle, and the time investment is excessively high” [30-years old MSM].

Discussion

Principal findings

In the context of growing interest in gamifying digital interventions for HIV prevention, our study aimed to test the impact of a gamified mHealth mini-app, O2O-PEP, on promoting PEP uptake among Chinese MSM. Developed internally with multidisciplinary support, the study evaluated acceptability, feasibility and preliminary efficacy in 19 participants, revealing promising potential for enhancing PEP uptake.

Our study findings suggest that the mini-app was both feasible and acceptable during the 2-week evaluation period, although the in-depth interviews highlighted areas requiring improvement. Notably, two gamified elements—a point-based reward system and social connectivity—were creatively integrated into various intervention modules to enhance engagement. For instance, users could accrue points and game card-based rewards upon completing different intervention components, and the “Share on moments” module fostered a sense of social connectivity. The point-based reward system operates on the principle that individuals are motivated to repeat desired behaviors to obtain further rewards. Due to its straightforward reinforcement nature, this system has found widespread application in various gamified interventions [21, 41]. Another crucial element, social connectivity, involves behaviors inspired by what other people think, say, or do. However, our data revealed a decline in app usage over the study duration, with some users expressing dissatisfaction, particularly with game card-based rewards, describing them as annoying. This observation aligns with findings reported in other studies [42]. Notably, several factors, both within and external to the mini-app, may impact usage. These encompass diverse game-based elements, users’ perceptions of app usability, and the speed at which intervention components are updated [43]. Cugelman B has presented seven criteria that intervention developers can leverage to craft a promising gamified intervention, namely goal setting, capacity to overcome challenges, providing feedback on performance, reinforcement, compare progress, social connectivity, and fun and playfulness [44]. Therefore, study researchers should collaborate with companies specializing in gamification technology and explore opportunities to integrate additional game components into the future development of digital interventions, aiming to enhance user engagement and overall effectiveness.

HIV PEP continues to be an underutilized HIV prevention measure in China, facing numerous barriers that hinder its widespread adoption. Among the challenges reported by MSM are the tedious administrative procedures in HIV/AIDS designated hospitals, characterized by awkward access to PEP clinics and long waiting times. Additionally, individuals expressed a sense of shame during the PEP consultation process in a traditional doctor’s office setting [45]. To address these issues, our study incorporated the “medication aide” module, directing participants to the online health platform HeHealth. HeHealth offers a sequential pathway for MSM to access PEP, including HIV risk assessment, pre-PEP usage counseling, PEP prescription, medication delivery, and follow-up phone calls. This digital approach aims to circumvent face-to-face visits to health facilities, particularly in environments perceived as unfriendly [16]. However, participants noted that pre-PEP counseling in the online platform (HeHealth) introduced its own set of challenges, describing it as a stressful and burdensome situation. Individuals exposed to increased risks of acquiring HIV often prioritize initiating PEP as quickly as possible. This sense of urgency may result in a reduced emphasis on or even disregard for pre-PEP counseling services. For instance, participants may focus solely on absorbing critical information shared during the counseling session without fully engaging in a two-way discussion, as their primary concern is to begin PEP promptly. Therefore, there is a call for the integration of more online mental health service for MSM during this critical phase. The mini-app’s most appreciated feature, “Share on Moments”, creates a platform for asynchronous peer-to-peer interaction and social connectivity. This aligns with the preferences of MSM, who have consistently advocated for such features in the design of HIV prevention interventions [46]. Despite its popularity, user analytics revealed that over half of the participants did not actively post or interact with others during the pilot trial. This reluctance may stem from concerns about privacy and perceived stigma associated with homosexuality, potentially deterring participation in HIV prevention programs [47].

Our data indicated a significant improvement in users’ PEP knowledge following the two-week intervention, with a Cohen’s d effect size of 0.8. In contrast to previous research on PEP, which has predominantly focused on aspects such as awareness, intention to use PEP, uptake, adherence, and their influencing factors [30, 48], there has been a notable gap in assessing in-depth PEP knowledge among MSM [49]. The observed enhancement in PEP knowledge scores over time suggests that O2O-PEP may have efficiently addressed a critical unmet need in the MSM community. However, despite the improvement in knowledge, a substantial gap persists between knowledge and healthy behaviors [50], To establish the true effectiveness of O2O-PEP in enhancing PEP uptake, future research is essential, emphasizing the need for a well-powered study.

Given the nearly universal ownership of mobile phones among MSM, mobile health technology has emerged as an ideal platform to reach individuals who may face challenges attending in-person health care services. The mobile mini-app facilitates the delivery of high-quality behavioral intervention to MSM via smartphones, allowing them to engage at their convenience in a private setting. While challenges in developing mHealth interventions, such as additional resource and time for developing, testing, and debugging, have been reported [51], there is a growing shift towards adapting existing evidence-based platforms that share common goals and functions to promote health behavior change [52]. For instance, Liu S et al. introduced PATHVERSE, a no-code mHealth app design platform, enabling researchers to design customized app content based on their specific research topics [53]. Although PATHVERSE has yet to be applied in the field of HIV prevention and care, future studies are warranted to explore its potential contributions to HIV related digital interventions.

Study strengths and limitations

This study demonstrated that the gamified mHealth-based intervention O2O-PEP is both feasible and acceptable among Chinese MSM. These results provided a strong foundation for expanding the trial to a larger sample size, which will facilitate a more comprehensive assessment of its efficacy in promoting HIV PEP usage. While our study findings should be interpreted in light of certain limitations. Firstly, the small sample size and purposive sampling represents a significant constraint. For instance, recruiting participants familiar with CBO staff may have introduced selection bias into the study sample, potentially overrepresenting individuals who are more engaged with HIV prevention initiatives. This may limit the generalizability of our findings to those less connected to CBOs or unfamiliar with HIV-related services. To address this limitation and strengthen the evidence base, we intend to enroll a more diverse cohort of MSM in future research. Secondly, the absence of a control group limits the validation of the intervention’s efficacy in PEP uptake and related behavioral health outcomes. Thirdly, the delivery of interviews through remote telephone contact posed a constraint, as it restricted our ability to observe participants’ body language and the surrounding environment, factors that could influence their feedback to the interview guide. Fourthly, the study did not analyze the effect of O2O-PEP in different subgroups of the population due to limitations in the sample size. Future research should explore the efficacy of O2O-PEP across highly vulnerable subgroups of MSM, such as young versus old MSM or single versus married MSM. Fifthly, numerous studies have confirmed that cost is a significant barrier to PEP usage among Chinese MSM. However, our study did not account for its impact on HIV PEP usage. Further research should explore how policy interventions, such as cost subsidies, can enhance the uptake of HIV PEP among MSM. Lastly, the study did not delve into recommendations for scalable app domains, like character-based narratives, primarily due to development cost considerations. This aspect is earmarked for prioritization in a subsequent study.

Conclusions

Overall, our study demonstrates the feasibility, acceptability, and preliminary efficacy of O2O-PEP among Chinese MSM. Future well-powered research is needed to assess its effectiveness in improving PEP uptake, emphasizing the potential impact of gamified digital interventions in HIV prevention and care.