Usability and applicability of a mindfulness based online intervention developed for people with problematic internet gaming behavior
Published: May 31, 2022
In this study, the usability and applicability of a mindfulness based online administered gamified short-term intervention for people suffering from internet gaming disorder is investigated in a subclinical sample (N = 49). Within a four-week intervention program (Room2Respawn) participants were randomly allocated to a gamified condition (experimental) or a neutral condition (control). Participants were provided with three intervention sessions per week consisting of psychoeducation on internet gaming disorder and mindfulness-exercises. Most of the intervention content has been adopted from a group-therapy intervention to an online platform. The 39 participants who participated over the course of the whole intervention rated the usability of the online-intervention overall and across all measures as good or very good. There was no evidence for beneficial effects of gamification on motivation when compared with ratings from a control condition. Areas of improvement and study limitations are discussed. Due to an easy applicability and very good usability, Room2Respawn shows potential to develop into a low-threshold treatment option, which, after rework, needs to be put to test on a sample showing clinical symptoms of pathological internet gaming.
Applicability., mindfulness, internet gaming, internet gaming disorder, usability, gamification
1. The usability of a newly developed mindfulness based online-intervention for people suffering from internet gaming disorder is under Scrutiny.
2. Content from a former group-therapy intervention was adopted and adjusted for digital use.
3. Gamification principles are used and described which were installed to increase motivation and treatment adherence.
Usability of a mindfulness based online intervention developed for people with problematic internet gaming behavior
Most internet gamers are casual players; however, some develop an attachment to internet games that can be considered pathological. The Internet Gaming Disorder (IGD) was included in the latest version of the Diagnostic Statistical Manual of Mental Disorders (DSM-5; American Psychiatric Association, 2013) as research diagnosis. The reported prevalence rates for IGD vary from less than 1 to 10% (Lemmens et al., 2015; Petry et al., 2014; Wartberg et al., 2017) and comorbid disorders such as social phobia, attention deficit disorder, depression and obsessive-compulsive disorder appear commonly (Carli et al., 2013; Mallorquí-Bagué et al., 2017). The internet gaming population still suffers stigmatization and since prevalence rates IGD are relatively low, the risk of pathologizing normal behavior is comparably high (Kardefelt-Winther et al., 2017).
Assessment methods and construct conceptualizations for IGD are quite diverse (King et al., 2013). Literature focusing on the treatment of IGD however, is relatively sparse in comparison to other potential behavioral addictions and more readily available treatment options (advisory interventions and psychopharmacotherapy) are often associated with high costs and/or side effects.
However, focusing on treatment effectiveness the research on psychotherapeutic interventions in the form of cognitive behavioral therapy seems promising, especially when combined with psychopharmaceutic interventions (Han et al., 2010; King and Delfabbro, 2014). Overall, the IGD population is heterogeneous, with a lack of recommended treatment strategies, large numbers of relapses and treatment often involves the necessity of high-cost interventions (for an overview of reviews and studies, see King et al., 2017; King et al., 2017; Torres-Rodríguez et al., 2018).
These high costs often lead for people that suffer from symptoms of IGD to refuse treatment or become non-compliant during the process of abstinence due to a lack of alternative coping strategies. The research on interventions with better accessibility, such as online-interventions, for IGD is especially sparse. Due to the associated proximity of the dysfunctional behavioral patterns in IGD (i.e. online-gaming) and the computerized intervention it is assumed that this might lead to an increase in severity of IGD related symptoms. Yet, other than arguments drawing from the theory of stimulus control (i.e. increasing distance between the problematic stimulus) including an abstinence focus, to the knowledge of the authors, regarding their validity these assumptions have not been further scrutinized. Also, in our increasingly digitized world an avoidance focus regarding computers and smartphones leads to a cascade of resulting issues, such as feelings of exclusion, not being able to participate in social interactions and so on. Thus, a restricted use of computers and smartphones cannot be regarded as a sustainable intervention approach.
This study introduces an intervention with a low entry threshold and short-term focus which encompasses resulting trans-diagnostic psychological deficiencies of people suffering from IGD such as emotion regulation problems, attentional disturbances, automatic dysfunctional behavioral patterns and diminished awareness towards deprived needs. It was decided to transfer and expand an already established alternative method for people suffering from addictions which is Mindfulness-Oriented Recovery Enhancement (MORE; Garland, 2013).
Originally, MORE is applied in a group-setting over eight or more consecutive weeks. Evidence for the effectiveness in treating IGD could already be provided (Li et al., 2017). MORE combines mindfulness meditation, elements of reappraisal of negative events and positive psychology into an integrative treatment to disrupt dysfunctional behavioral and cognitive patterns and to re-establish awareness for deprived needs. The benefits of mindfulness interventions for treatment and motivation for people suffering from (substance) addiction (Li et al., 2017b) and gambling addiction (Toneatto et al., 2014) is plentiful.
To circumvent the treatment gap (Kohn et al., 2004) and add flexibility to the application of the intervention, a transfer of the adopted content to an online setting seems promising. The effectiveness of online based interventions to bridge waiting period was already shown (Fuhr et al., 2018). Furthermore, an online application of an intervention represents a low-threshold and cost-effective possibility to reach people who otherwise would not have been able to find help.
In the best-case scenario an intervention should be as motivating as internet gamer are captivated by internet games. Regarding underlying motivational mechanisms, Yee (2006) offered a conceptualization of player types via the distinction into three categories (Achievement, Social, Immersion). The components of “Achievement” entail advancement, mechanics and competition. “Social” comprises socializing, relationship and teamwork whereas “Immersion” contains discovery, role-playing, customization and escapism. Yet, these motivational concepts described above are not only present in games but have also been adopted via the use of operationalized gamification principles in a variety of other domains.
Implementing gamification principles can enhance work or learning processes, increase motivation and subsequently performance (Sailer, 2016). The stigma around gamification principles arises due to the underlying framework for which they are used in online-games (such as increasing the volume in micro-transactions and furthering gambling experiences [i.e. loot-boxes] to increase commitment). It can be argued that using the familiarity with gamification principles by applying these principles to an intervention for people suffering from pathological internet gaming offers a non-intrusive way to help shift dysfunctional behaviors to a functional need fulfilling behavior, giving a functional framework. For instance, in a study by Miloff et al. (2015), a gamified intervention for people suffering from social anxiety was introduced which incorporated a high degree of personalization, anonymous social interactions and further gamification techniques. Instead of condemning addiction maintaining learning mechanisms, an implementation of those same mechanisms for a functional transfer to alternative coping behaviors seems helpful and resource oriented. The developed intervention Room2Respawn uses gamification principles in its design by focusing on the following dimensions: points as a reward system, see also (Sailer et al., 2017; Werbach and Hunter, 2012), levels to structure tasks and increase salience of goals (Stinson et al., 2013), badges as collectibles to promote motivation (Eickhoff et al., 2012), personalized contact to increase identification (Reeves and Read, 2009) and leaderboards for social comparison and competition (Burguillo, 2010; Crumlish and Malone, 2009).
This study aims at exploring the applicability and usability of an online administered treatment with mindfulness elements for people suffering from symptoms of IGD. This problematic IGD related behavior is defined as people showing symptoms, such as withdrawal, preoccupation, tolerance and further, being measured in the IGDT-10 (Király et al., 2017). However, surpassing a certain threshold of problematic IGD related behavior was no necessity for participation. Instead the population of interest were people with experience in online-gaming related behaviors to firstly test the applicability and usability of the newly developed intervention and circumenvent possible adverse effects for severely IGD disordered people arising due to an unclear non-applicability of the proposed intervention. To the authors, it is of utmost necessity for any intervention with no in-person contact (in which a direct clinical control is not possible) to take precautions and first explore the interventions validity with a more functional population, such as with people having experience with online-games and only show some signs of IGD. Gamification principles were incorporated to deliver the intervention content in a known framework for an internet gaming population. Since the usability of a gamified intervention for people suffering from IGD has to the author’s knowledge never been tested, a framework with a gamified experimental condition and a balanced, neutral condition was chosen as control. Of primary interest for this investigation is an adequate usability over the duration of the intervention (four weeks). It is hypothesized that participants will rate the usability of the intervention elements across all measures as above average. On a secondary level it is hypothesized that the reported (intrinsic) motivation for people to reduce internet gaming behavior will increase after the intervention. Overall, it is expected that usability and motivation will be rated higher for people in the gamification condition than for people in the control condition. On an exploratory level the comments made by participants regarding the intervention will be qualitatively investigated. Lastly, participation and dropout-rates between groups are explored. A subclinical sample with gaming experience was chosen as a first step for further optimization of the intervention.
Inclusion criteria were age between 18 and 40 years, German mother tongue and gaming experience. Exclusion criteria were the use of illegal substances, mental illness and pathological signs of internet gaming disorder. To take precautions, prospective participants that were showing strong signs of IGD were forwarded to specialized clinics offering already established treatment options. Interested participants who reported having experiences with internet gaming were chosen and randomly allocated to the neutral control or gamified experimental condition. Participants were recruited via flyers on the university premises, in online forums and social media. The starting sample consisted of 49 participants (24 females; 25 males) between 18 and 35 years of age (M = 24.41, SD = 3.72 years). Participants missing four or more sessions were excluded (dropout: four participants experimental condition, six participants control condition) resulting in a final sample of 39 participants for analyses. The remuneration of the participants was either financial (50€) or with credit points as a trial subject. The latter are a necessity for students to obtain their bachelor degree at the University of Düsseldorf. Informed written consent was obtained from all participants. The study was approved by a local Ethics Committee.
Instruments and Intervention
The German version by Böckermann et al. (2015) of the CSUQ (Lewis, 1995) was used and for study purposes, the original 19 items were extended by 11 additional items to capture usability ratings regarding email interactions. Statements range on a Likert scale from 1 = don’t agree at all to 7 = completely agree.
Two questions succeeding every intervention session were implemented to control whether the participants revised the exercise content for each session. The questions were designed by the authors and presented in a multiple-choice format with one correct answer possibility.
The IMI (first used in Ryan, 1982) is rooted in Self Determination theory and consists of 45 items on six subscales. These were reduced to 27 items with a changed reference from this activity to this exercise and translated into German for the purpose of this study. Answers are recorded within a 7-point Likert scale ranging from 1 to 8 with higher overall values indicating stronger approval of the respective subscale (1 interest or pleasure, 2 perceived competence, 3 perceived choice, 4 effort, 5 feeling pressure or tension, 6 usefulness); Cronbach’s α ranging from .82 to .91 (Monteiro et al., 2015).
In addition to age, gender, education and knowledge of German, the sociodemographic questionnaire also covered previous experience with meditation, diagnosed mental illnesses, illegal substance use and frequencies of digital media and video game consumption of the participants.
The IGDT-10 (Király et al., 2017) contains 10 items which measure the extent of pathological internet gaming behavior based on the DSM-5 criteria for Internet Gaming Disorder. Item nine and ten refer to the same facet of the disorder and contribute each 0.5 points towards the sum of the scale. The IGDT-10 was translated into German for the purposes of the study. A cut-off score of five out of nine points represents a strong indication of an underlying disorder. In the response format, answer possibilities are Never, Sometimes, and Often. The IGDT-10 has an internal consistency of Cronbach's α = .68.
The self-developed questionnaire for the evaluation of the intervention content is divided into four sub-scales: email (five items), psychoeducation (two items), meditation (four items) and intervention in general (two items). Dimensions are length, interestingness, favorability and comprehensibility evaluated on a 7-point Likert scale with corresponding varying anchors. Furthermore, participants could also comment open-ended within a text box. In the experimental group the level of gamification of the intervention content was also evaluated using a questionnaire including 42 items with seven questions each for the implemented gamification elements (pseudonym, daily quiz, score, leaderboard, level and badge). Answers were recorded using a 7-point Likert scale ranging from 1 = not at all to 7 = agree completely and a commentary box for open-ended answers.
In the quasi-experimental design of the study the experimental condition received a gamified intervention content whereas the control condition received a neutral version with the same, balanced content.
The email communication with the participants contained the gamification elements that were implemented. In the gamification condition participants were asked to create a personal username (1a Personalization), interventions were referred to as levels (2a Levels), a leaderboard (3a Leaderboard) showed the current score (correct responses in control questions and working through content awarded points) and ranking in comparison to fellow participants and lastly a badge (4a Badge) depicting a progressively more evolving meditating person after the first and with every three successfully completed session. The gamification elements were balanced as follows in the neutral condition: a standardized neutral code (1b Subject Code), interventions were referred to as sessions (2b Session), an overview (3b Overview) about the weekly content was shown and the logo for Room2Respawn was depicted (4b Logo).
The main content for the intervention has been adopted largely from MORE (Garland, 2013), selected by clinicians for suitability and translated into German for the purpose of this study and adjusted for internet gaming behavior. The content of the intervention (12 sessions) was the same for experimental and control group and can be reviewed in MORE (Garland, 2013) or under http://www3.hhu.de/room2respawn/. All mindfulness-meditations were recorded by a trained psychotherapist and adjusted in content for internet gaming behavior. The fifth session had to be completed solely in writing, all other exercises contained a meditation (7 min and 25 s on average). An overview about the sessions, topics, psychoeducational content, mindfulness-exercises and the implemented questionnaires is given in Table 1.
Interested possible participants contacted the researchers via email and received a link for the online screening. The latter consisted of an informed consent, a declaration of data protection, sociodemographic questions and the IGDT-10. All answers were collected and carried out via the online questionnaire service SoSciSurvey (Leiner, 2018). Subsequently, all included participants received an email with general information about the course of the study and an invitation to participate in the pretesting (t00).
The intervention content was published on each intervention day (three times a week; on Monday, Wednesday and Friday) at 10 a.m. on an intervention blog entitled Room2Respawn which was created for the purpose of this study. After each session, the content remained on the blog for the remainder of the intervention. The content could be edited until 10 p.m. on the evening of the subsequent intervention session. Since participants received an email for each individual session, this email also included a link to the post-session survey. The post-session survey included a question of conscience to check whether the content was revised, a control question in a multiple-choice format regarding the psychoeducation and mindfulness-exercise, and evaluative multiple-choice questions regarding the intervention content. In case a session was missed, participants received an email asking them to attend the next session again, stating that it was not possible to complete the intervention at a later point in time. After having missed three sessions altogether a warning was sent, describing that the next miss would lead to exclusion from participation. At the end of the last intervention session all subjects were invited via email to take part in the final post-intervention survey after which they received an email of appreciation for their participation, a debriefing and their renumeration.
All statistical analyses were carried out at a significance level of 0.05, using IBM SPSS Statistics (25). Independent variables are condition, operationalized as gamification (experimental) or neutral (control) group and time of measurement. Dependent variables were usability ratings (dimensions: email, psych education, meditation, length) and (intrinsic) motivation. Data was collected at 15 timepoints (screening, pre-measurement, 12 interventions, post-measurement). Of primary interest for this study was the evaluation of overall usability measured in time point’s t1, t6 and t13 with the CSUQ and overall time points with the UEQ. For the main analyses, t-Tests for independent samples or analyses-of-variance (ANOVAs) were calculated. A 3x2 mixed-factors ANOVA was employed to explore changes in ratings for the usability throughout the course of the intervention (t1, t6, t13) for the different
conditions (experimental and control). For the applicability of content, the mean values for the scales Email, Psych education and Meditations were computed and together with the dimension of Length, t-Tests between conditions were computed. To test whether a change in (intrinsic) motivation took place during the intervention phase, the subscales of the IMI were analyzed with 2x2 ANOVA with the between-factor conditions (experimental, control) and the within-factor time of measurement (t1, t12).
Statistics and data analysis
Comparison of age, media usage times and computer gaming experience (Table 2) in a sample of N = 39 participants who completed the intervention showed no statistically significant differences between control (n = 18) and experimental (n = 21) condition, except for computer gaming experience in years (Table 2).
The usability of the developed intervention as indicated in the CSUQ was good, M = 6.20, SD = 0.69, ranging from 1 = “not good at all”, to 7 = “very good”. A dependent samples t-Test on a conservatively chosen cutoff of 5 as indication of adequate usability revealed a statistically significant difference from this cutoff (t(31) = 10.12, p < .001), thus, providing evidence for the first hypothesis (Figure 1). There was no statistically significant difference for time, F(2, 60) = 1.75, p = .182, ηp² = .06, groups, F(1, 30) = 1.77, p = .192, ηp² = .06 and no interaction effect (F(2, 60) = 1.00, p = .372, ηp² = .03).
Regarding the overall evaluation of content for emails, psychoeducation and meditations, indications in the UEQ were consistently positive and the length of each intervention session was rated adequate. There were no differences between groups regarding the perceived quality of content (F(3, 35) = 1.63, p = .200) or length (t(37) = 1.75, p = .088, d = .06) of the intervention (Table 3).
Concerning the motivation hypothesis, the mean values and standard deviations for the subscales can be seen in Table 4. The perceived choice was statistically significantly higher after the intervention (F(1, 30) = 5.78, p = .023, ηp² = .16) and indications of the participants regarding usefulness of the intervention was statistically significantly lower after the intervention (F(1, 30) = 5.33, p = .028, ηp² = .15). Except for that, no statistically significant changes in respecting subscales could be observed. There were no statistically significant interaction effects for time and condition.
For exploratory reasons, participation and dropout rates between experimental and control condition were compared. Participants in the control condition missed on average 0.61 (SD = 0.85) interventions, whereas participants in the experimental group missed 0.71 (SD = 0.90) interventions. This difference was not statistically significant, t(37) = 0.37, p = 0.717. Analyses of the dropout rates between control group (25 percent) and experimental group (16 percent) did not conclude statistically significant results, U = 273.000, p = 0.439.
Qualitative evaluation of the main comments regarding improvable aspects of the intervention, revealed an improvement of email layout, rework of badges, clearer instructions, shorter sentence structure and improvement of the text layout. For the meditations, background music, slower pace, and slightly longer durations were suggested.
The goal of the present study was to determine the usability and applicability of a newly developed online intervention (Room2Respawn) for people suffering from IGD. A population of people with internet-gaming experience has been chosen to indicate applicability and usability of the intervention. Overall, the usability of Room2Respawn was rated as good and participants evaluated all associated subscales positively. Applicability of content, email, psychoeducation and length of exercises were rated as good and satisfactorily. Usability ratings did not differ between gamified experimental or balanced control condition. Participants in both conditions reported higher perceived choice and lower usefulness at the end of the intervention. Since these results are inconclusive, the motivation hypothesis needs to be rejected. Additionally, there was no statistically significant indication that the gamified version of the intervention was more motivating for participants. Descriptively, there were fewer dropouts present in the gamified condition.
Participants rated the website as good and easy to use. However, some participants commented that the design could be improved. More experience with digital media means higher expectations regarding the interface (Crumlish and Malone, 2009). Reviewing the comments which the participants made it seems promising to further professionalize the recorded meditations by adding more structure, a slower pace and more breaks. Due to the structure of the intervention, it was not possible for participants to individualize their treatment schedule. Adding more opportunities for personalization could lower dropout-rates and encourage participation. However, finding a sensible balance regarding autonomy and freedom is a challenging task for further redesign.
The effects of gamification are context-dependent and controversially discussed. Benefits on performance and (intrinsic) motivations are reported (Sailer, 2016), yet, the collected data does not provide evidence towards this. Due to continuous exposure to gamified digital content, people become increasingly familiar with its mechanics. Thus, implementing only a few basic options, such as leaderboards, badges, levels and a pseudonym, this could have an adverse effect on participation and motivation by making a lack of quality in implementation salient. The reduced perceived usefulness could be explained by an emerging redundancy in psycho-educational content. The observed increase in perceived choice hints at an increase in perceived autonomy in participants due to the intervention but needs to be scrutinized in a follow-up study. Furthermore, in a revision of Room2Respawn, the inclusion of a chat forum for participants could represent a further step to increase feelings of belonging (Lavigne et al., 2011) and help foster collective growth orientation. The lack of social embeddedness represents one of the most apparent differences of this intervention compared to MORE (Garland, 2013).
One limitation of the present study, besides a larger sample size, is the lack of validation of the translated intervention and meditation texts taken from MORE (Garland, 2013), the translation of the IGDT-10 into German and adjusting the IMI. Also, the CSUQ was extended by adding a dimension of email usability. Additionally, sample characteristics could limit generalizability since the sample was well-educated and had some experience with internet gaming but not in clinical amounts. There were some technical issues due to which some emails were not received by the participants or human error occurred and wrong badges were sent to participants. Besides possible influences on usability that these issues might have caused, reduction of the interference of technology could be achieved by adjusting the intervention content to run on a single platform or by automatizing communication via email.
To summarize, content from a mindfulness-based intervention which has already proven effective for the treatment for IGD was successfully adopted to an online platform (Garland, 2013; Li et al., 2017). Room2Respawn was conceptualized as a short-term intervention that uses a gamified framework to approach people suffering from IGD to alleviate symptoms and promote a needs-congruent way to learning alternative coping mechanisms. Due to the more rapidly growing digital world and the increasing demands on the individual to cope with this, it is crucial for research and treatment to keep up. After taking the results of this study into consideration and redesigning some parts of the intervention, the effectiveness of Room2Respawn should be tested on a sample with participants showing clinical symptoms of internet gaming disorder.
Key practice points
1. Moving from group-settings or face-to-face settings to a digital form of communication with clients suffering from pathological internet gaming is possible and shows promising usability and applicability.
2. Digital and internet-based interventions can be used before, during and after “traditional” face-to-face CBT.
3. For a successful blend of methods, it is crucial that practitioners are familiar with the content of the interventions and its main goals to ensure a synergistic efficacy.
CONFLICT OF INTERESTS
The authors have not declared any conflict of interests.
American Psychiatric Association (2013). Diagnostic and statistical manual of mental disorders (5th ed.). American Psychiatric Publishing.
Böckermann M, Gieselmann A, Sorbi M, Pietrowsky R (2015). Entwicklung und Evaluation einer internetbasierten begleiteten Selbsthilfe-Intervention zur Bewältigung von Albträumen. Zeitschrift Für Psychiatrie, Psychologie und Psychotherapie 63(2):117-124.
Burguillo JC (2010). Using game theory and competition-based learning to stimulate student motivation and performance. Computers and Education 55(2):566-575.
Carli V, Durkee T, Wasserman D, Hadlaczky G, Despalins R, Kramarz E, Wasserman C, Sarchiapone M, Hoven CW, Brunner R, Kaess M (2013). The association between pathological internet use and comorbid psychopathology: A systematic review. Psychopathology 46(1):1-13.
Crumlish C, Malone E (2009). Designing social interfaces: Principles, patterns, and practices for improving the user experience. O'Reilly Media.
Eickhoff C, Harris CG, Vries AP de, Srinivasan P (2012). Quality through flow and immersion. In Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval, ACM Press pp. 871-80.
Fuhr K, Fahse B, Hautzinger M, Gulewitsch M (2018). Erste Erfahrungen zur Implementierbarkeit einer internet-basierten Selbsthilfe zur Überbrückung der Wartezeit auf eine ambulante Psychotherapie. PPmP - Psychotherapie Psychosomatik Medizinische Psychologie 68(06):234-241.
Garland EL (2013). Mindfulness-oriented recovery enhancement for addiction, stress, and pain. NASW Press. Han DH, Hwang JW, Renshaw PF (2010). Bupropion sustained release treatment decreases craving for video games and cue induced brain activity in patients with Internet video game addiction. Experimental and Clinical Psychopharmacology 18(4):297-304.
Kardefelt-Winther D, Heeren A, Schimmenti A, van Rooij A, Maurage P, Carras M, Edman J, Blaszczynski A, Khazaal Y, Billieux J (2017). How can we conceptualize behavioural addiction without pathologizing common behaviours? Addiction 112(10):1709-1715.
King DL, Delfabbro PH (2014). Internet gaming disorder treatment: A review of definitions of diagnosis and treatment outcome. Journal of Clinical Psychology 70(10):942-955.
King DL, Delfabbro PH, Wu AMS, Doh YY, Kuss DJ, Pallesen S, Mentzoni R, Carragher N, Sakuma H (2017). Treatment of Internet gaming disorder: An international systematic review and CONSORT evaluation. Clinical Psychology Review 54:123-133.
King DL, Haagsma MC, Delfabbro PH, Gradisar M, Griffiths MD (2013). Toward a consensus definition of pathological video-gaming: A systematic review of psychometric assessment tools. Clinical Psychology Review 33(3):331-342.
King DL, Kaptsis D, Delfabbro PH, Gradisar M (2017). Effectiveness of brief abstinence for modifying problematic internet gaming cognitions and behaviors. Journal of Clinical Psychology 73(12):1573-1585.
Király O, Sleczka P, Pontes HM, Urbán R, Griffiths MD, Demetrovics Z (2017). Validation of the ten-item Internet Gaming Disorder Test (IGDT-10) and evaluation of the nine DSM-5 Internet Gaming Disorder criteria. Addictive Behaviors 64:253-260.
Kohn R, Saxena S, Levav I, Saraceno B (2004). The treatment gap in mental health care. Bulletin of the World Health Organization 82(11):858-866.
Lavigne GL, Vallerand RJ, Crevier-Braud L (2011). The fundamental need to belong: On the distinction between growth and deficit-reduction orientations. Personality and Social Psychology Bulletin 37(9):1185-1201.
Leiner DJ (2018). SoSci Survey (Version 3.0.06) [Computer software]. Available at:
Lemmens JS, Valkenburg PM, Gentile DA (2015). The Internet gaming disorder scale. Psychological Assessment 27(2):567-582.
Lewis JR (1995). IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction 7(1):57-78.
Li W, Garland EL, McGovern P, O'Brien JE, Tronnier C, Howard MO (2017). Mindfulness-Oriented Recovery Enhancement for Internet gaming disorder in US adults: A stage I randomized controlled trial. Psychology of Addictive Behaviors: Journal of the Society of Psychologists in Addictive Behaviors, 31(4):393-402.
Li W, Howard MO, Garland EL, McGovern P, Lazar M (2017). Mindfulness treatment for substance misuse: A systematic review and meta-analysis. Journal of Substance Abuse Treatment 75:62-96.
Luciano-Soriano MC, Martin SG, Lopez MH, Luque FC (2001). Alcoholismo, evitacion experiencial y terapia de aceptacion y compromiso (ACT) (Translation: Norbert Schneider). Analysis Y Modificacion De Conducta 27:333-371.
Mallorquí-Bagué N, Fernández-Aranda F, Lozano-Madrid M, Granero R, Mestre-Bach G, Baño M, Pino-Gutiérrez AD, Gómez-Peña M, Aymamí N, Menchón JM, Jiménez-Murcia S (2017). Internet gaming disorder and online gambling disorder: Clinical and personality correlates. Journal of Behavioral Addictions 6(4):669-677.
Miloff A, Marklund A, Carlbring P (2015). The challenger app for social anxiety disorder: New advances in mobile psychological treatment. Internet Interventions 2(4):382-391.
Monteiro V, Mata L, Peixoto F (2015). Intrinsic motivation inventory: Psychometric properties in the context of first language and mathematics learning. Psicologia: Reflexão E Crítica 28(3):434-443.
Petry NM, Rehbein F, Gentile DA, Lemmens JS, Rumpf HJ, Mößle T, Bischof G, Tao R, Fung DSS, Borges G, Auriacombe M, González Ibáñez A, Tam P, O'Brien CP.(2014). An international consensus for assessing Internet gaming disorder using the new DSM-5 approach. Addiction 109(9):1399-1406.
Reeves B, Read JL (2009). Total Engagement: How Games and Virtual Worlds Are Changing the Way People Work and Businesses Compete. Harvard Business Review Press.
Ryan RM (1982). Control and information in the intrapersonal sphere: An extension of cognitive evaluation theory. Journal of Personality and Social Psychology 43(3):450-461.
Sailer (2016). Die Wirkung von Gamification auf Motivation und Leistung. Springer.
Sailer M, Hense J, Mandl H, Klevers M (2017). Fostering development of work competencies and motivation via gamification. In Competence-based Vocational and Professional Education. Springer Cham. 23:795-818.
Stinson JN, Jibb LA, Nguyen C, Nathan PC, Maloney AM, Dupuis LL, Gerstle JT, Alman B, Hopyan S, Strahlendorf C, Portwine C, Johnston DL, Orr M (2013). Development and testing of a multidimensional iPhone pain assessment application for adolescents with cancer. Journal of Medical Internet Research 15(3):e51.
Toneatto T, Pillai S, Courtice EL (2014). Mindfulness-enhanced cognitive behavior therapy for problem gambling: A controlled pilot study. International Journal of Mental Health and Addiction 12(2):197-205.
Torres-Rodríguez A, Griffiths MD, Carbonell X (2018). The treatment of Internet gaming disorder: A brief overview of the PIPATIC program. International Journal of Mental Health and Addiction, 16(4):1000-1015.
Wartberg L, Kriston L, Kramer M, Schwedler A, Lincoln TM, Kammerl R (2017). Internet gaming disorder in early adolescence: Associations with parental and adolescent mental health. European Psychiatry: The Journal of the Association of European Psychiatrists 43:14-18.
Wengenroth M (2016). Das Leben annehmen: So hilft die Akzeptanz- und Commitment-Therapie (ACT). Hogrefe.
Werbach K, Hunter D (2012). For the win: How game thinking can revolutionize your business. Wharton Digital Press.
Yee N (2006). Motivations for play in online games. CyberPsychology and Behavior 9(6):772-775.