
==== Front
Internet Interv
Internet Interv
Internet Interventions
2214-7829
Elsevier

S2214-7829(24)00064-2
10.1016/j.invent.2024.100771
100771
Full Length Article
A protocol for a pilot randomised controlled trial of unguided internet cognitive behaviour therapy for grief in adolescents☆
Egan Sarah J. s.egan@curtin.edu.au
ab⁎
Munro Caitlin ab
Pauley-Gadd Sian B. ab
O'Brien Amy ab
Callaghan Thomas ab
Payne Nicholas ab
Raghav Shravan c
Myers Bronwyn ade
Hall Christopher f
Wilson Hayden g
Eisma Maarten C. h
Boelen Paul A. ij
Smith Kirsten V. k
Wild Jennifer kl
Duffy Michael m
Trickey David n
Bills Elizabeth ab
Breen Lauren J. ab
a enAble Institute, Faculty of Health Sciences, Curtin University, Perth, Australia
b Discipline of Psychology, School of Population Health, Curtin University, Perth, Australia
c Independent lived experience consultant, Chennai, India
d Mental Health, Alcohol, Substance Use and Tobacco Research Unit, South African Medical Research council, Tygerberg, South Africa
e Department of Psychiatry and Mental Health, University of Cape Town, Cape town, South Africa
f Grief Australia, Melbourne, Australia
g Child and Adolescent Mental Health Service, Western Australia Country Health Service, Perth, Australia
h Department of Clinical Psychology and Experimental Psychopathology, University of Groningen, the Netherlands
i Department of Clinical Psychology, Utrecht University, the Netherlands
j ARQ National Psychotrauma Centre, Amsterdam, the Netherlands
k Department of Experimental Psychology, University of Oxford, United Kingdom
l Phoenix Australia, Department of Psychiatry, University of Melbourne, Australia
m Queen's University Belfast, United Kingdom
n University College London, United Kingdom
⁎ Corresponding author at: enAble Institute and School of Population Health, Curtin University, Australia. s.egan@curtin.edu.au
03 9 2024
12 2024
03 9 2024
38 10077126 3 2024
5 8 2024
3 9 2024
© 2024 The Authors
2024
https://creativecommons.org/licenses/by-nc/4.0/ This is an open access article under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).
Grief is highly prevalent in adolescents, however, there have been no studies investigating internet delivered cognitive behaviour therapy for grief in adolescents (ICBT-G-A). In this paper, the co-design of an unguided ICBT-G-A intervention is described, and a protocol outlined for a pilot randomised controlled trial of the intervention. Participants will be randomised to the intervention (delivered via eight modules over a four-week period) or a four-week waitlist control. Intervention participants will complete a follow-up assessment at one-month post-intervention (eight weeks from the pre-intervention assessment). The intervention outcomes assessed at pre-intervention, post-intervention and follow-up include wellbeing and symptoms of anxiety, depression, post-traumatic stress, and prolonged grief. User feedback on experiences and acceptability of the intervention will be sought and feasibility assessed via programmatic data on recruitment and attrition.

Highlights

• Grief is common in adolescents.

• No studies to date have evaluated internet grief treatment for youth.

• We co-designed an unguided internet grief treatment with youth.

• The plan is to test the intervention in a pilot trial.

• Accessible interventions are needed for grieving young people.

Keywords

Grief
Loss
Cognitive behaviour therapy
Internet
Adolescence
==== Body
pmc1 Introduction

Grief is normal and refers to the behavioural, cognitive, and emotional response to loss. These losses can include the death of a close person or other losses e.g., death of a pet, parental divorce, and relationship break-up (Field, 2011; Simbi et al., 2020; Uttley, 2019). Grief in children is common, 62 % have experienced bereavement by the age of 10 (Paul and Vaswani, 2020). Children and adolescents who experience bereavement are at a heightened risk of mental health problems (Morris et al., 2016), including anxiety, depression, self-harm, suicide attempts and poorer physical health (e.g., DelCarpio et al., 2020; Harrison and Harrington, 2001; Spuij et al., 2012). Simbi et al. (2020) reported in a meta-analysis that parental loss in childhood through death or parental separation is a risk factor for depression in adulthood. Given studies indicating a heightened risk of mental health problems in children and adolescents who have experienced loss, there is a need for accessible interventions for grief related to both death and non-death loss.

There are a range of psychological interventions for grief including cognitive-behaviour therapy (CBT) for grief, trauma-focused CBT for grief related to traumatic loss, and CBT for prolonged grief (see reviews by Breen et al., 2023 and Salandino et al., 2024 for an overview). CBT for prolonged grief (characterised by a persistent yearning for the deceased and pre-occupation with memories of the person and the circumstances of the death) has focused on intervening for significant impairment in functioning 6 to 12 months or more post bereavement (e.g., Boelen et al., 2021). CBT for prolonged grief includes graded in-vivo behavioural exposure to avoided aspects of the loss (e.g., visiting a grave), cognitive techniques (e.g., cognitive restructuring, behavioural experiments, imaginal exposure), and behavioural activation (Boelen et al., 2021). In a meta-analysis of 28 intervention studies of grief interventions with participants aged 14–24 years, Breen et al. (2023) calculated within-group effect sizes based on single-group, pre-post, randomised controlled trial (RCT) and post-only data from a combination of study designs, including open studies, RCTs comparing interventions to wait-list control, and a small number of RCTs comparing interventions to active treatment. CBT for grief emerged as the most efficacious intervention, with a large effect size reduction in anxiety (pooled d = −0.81) and grief (pooled d = −0.80) and a medium effect size reduction in depression (pooled d = −0.58). Another systematic review including 20 studies (with varying designs including open trials and RCTs) of CBT for grief in children and adolescents reported significant reductions in anxiety, depression, and grief, although effect sizes were not reported as it was not a meta-analysis (Salandino et al., 2024).

Intrusive re-experiencing, distressing memories and images are also common features of traumatic bereavement, for example, related to death through an accident (Duffy and Wild, 2023). A CBT intervention for grief that addresses these symptoms is cognitive therapy for traumatic bereavement (Duffy and Wild, 2023; Murray and El-Leithy, 2023; Wild et al., 2023). This is based on an extension of the cognitive model of PTSD (Ehlers and Clark, 2000) and is supported by research demonstrating that the targets for treatment, namely memory characteristics, negative appraisals and unhelpful coping strategies predict and maintain symptoms of prolonged grief and PTSD following bereavement (e.g., Smith and Ehlers, 2020, Smith and Ehlers, 2023). Components of treatment, including memory rescripting and imagery conversations with the deceased, have not been examined in adolescents. Imagery rescripting is a version of memory updating, which helps to bring in new, useful information to update ‘stuck’ memories (Murray and El-Leithy, 2023). Imagery techniques are typically delivered by a therapist. However, an important question is whether imagery techniques are feasible to use in unguided internet interventions. We sought to incorporate imagery techniques for traumatic bereavement (Wild et al., 2023) into treatment.

Despite CBT for grief being effective in adolescents when delivered face-to-face (e.g., Boelen et al., 2021), CBT for grief is not readily accessible as an internet intervention for adolescents (Breen et al., 2023). Internet interventions can increase accessibility, efficiency, and cost effectiveness of treatments (Andersson, 2016). However, no studies have examined ICBT-G in adolescents (ICBT-G-A). This is an important gap as to address the scalability of interventions an internet intervention for grief is required as there will never be enough trained professionals available to implement face to face treatment in adolescents who require help for their grief. Further, we argue that it is vital to co-design ICBT-G in partnership with people who have lived experience of grief. Co-design is a broad term encompassing a range of approaches which can include co-production methods (Norton, 2021), where individuals with lived experience are involved in all stages of the research. Co-design of interventions is also a broad approach, from individuals with lived experience designing all aspects of an intervention, through to feedback on multiple drafts of an intervention plan and helping to write components of the intervention (e.g., Egan et al., 2023; O'Brien et al., 2022). To date there have been no co-designed ICBT-G-A interventions. We will use co-production methods (Norton, 2021) to involve consumers in designing the intervention and as co-researchers throughout the proposed trial. Co-design of interventions is critical to increase the relevance, quality, and uptake of interventions (Schleider, 2023).

Breen et al. (2023) reported that adolescents and young adults with lived experience of grief recommended interventions should account for bereavement from the death of a close person as well death of pets and non-death losses. The intervention we developed was based on addressing this feedback, to include non-death related loss. Specifically, this paper presents the protocol for a pilot RCT of co-designed ICBT-G-A.

2 Method

2.1 Aims and hypothesis

The aim is to examine (1) preliminary evidence for efficacy on wellbeing and symptoms of anxiety, depression, PTSD and prolonged grief and (2) feasibility and acceptability of ICBT-G-A It is predicted that individuals in ICBT-G-A will report significant improvements in anxiety (primary outcome) and secondary outcomes of depression, wellbeing, prolonged grief, and PTSD at post-treatment, compared to waitlist control, which will be maintained at one-month post-intervention follow-up. Anxiety was chosen as the primary outcome as a review of face-to-face studies of psychological interventions for grief in 14–24 year olds (Breen et al., 2023), found that the intervention resulted in a large effect size reduction in anxiety, while the intervention had a medium effect size reduction in depression, hence we expected a larger effect for anxiety than depression and chose this as the primary outcome. Further, we chose anxiety as the primary outcome due to the inclusion of non-death loss, prolonged grief may not be an appropriate choice of primary outcome since measures of prolonged grief are largely restricted to death-related losses, not all participants would answer a measure of prolonged grief. We also predict feasibility will be demonstrated by sufficient (1) recruitment of at least 85 participants within a 14-week period and (2) attrition defined as 25 % or less of participants not completing post-intervention questionnaires.

2.2 Design

The study is a pilot RCT to examine preliminary efficacy, feasibility, and acceptability of ICBT-G-A. The sample will include 85 adolescents residing in Australia. The RCT was prospectively registered with the Australian and New Zealand Clinical Trials Registry in March 2024 (ACTRN12624000337572). Following screening, participants who are eligible will complete outcome measures at pre- and post-intervention, and follow-up (Fig. 1, Table 1).Fig. 1 Flow chart of participants through the study.

Fig. 1

Table 1 SPIRIT checklist.

Table 1	Pre-Intervention	Post-allocation	
Timepoint	Enrolment and baseline data (Time 1)	Allocation	Intervention	Post-intervention (Time 2)	Follow-up (Time 3)	
Enrolment	
Informed Consent	x					
Demographic and Contact Data	x					
Eligibility Screening	x					
Allocation		x				


	
Interventions	
Online Unguided 4 Week Intervention			x			
Waitlist Control						


	
Assessments	
Module completion questions			x	x		
Revised Children's Anxiety and Depression Scale (RCADS-11; Radez et al., 2021)	x			x	x	
WHO-5 Well Being Index (Bech et al., 2003)	x			x	x	
The Prolonged Grief Disorder scale - Revised (PG-13-R; Prigerson et al., 2021)	x			x	x	
Children's Revised Impact of Events Scale (CRIES-8 item version; Perrin et al., 2005)	x			x	x	
Feasibility				x		
Acceptability				x		

2.3 Ethical considerations

The trial has been approved by the Curtin University Human Research Ethics Committee (HREC) (HRE2024–0722). Participant identity will be protected through a unique six-digit code linking participant identity to outcome measures, with identifying information stored separately from de-identified data on the Curtin University secure server. After study completion the ability to re-identify data will be deleted. The waitlist control group will be provided the intervention after a four-week waiting period. This is significantly less time than an adolescent will wait to see a mental health professional in Australia, where waitlist times can be up to 12 months. De-identified data will be stored on the Curtin University secure server for 25 years.

2.4 Intervention development

The intervention was co-designed with a 15-member Youth Advisory Committee (YAC) (53 % were aged 15–17 years, 47 % were aged 18–24 years). YAC members were recruited through an advertisement on social media and Curtin University first year psychology online noticeboards stating, ‘Are you aged 13-17 years, have lived experience of grief and interested in helping to create an online intervention for grief in adolescents?’, which resulted in n = 7 YAC members. To supplement this group to ensure the views of a larger number of people were included, n = 5 YAC members were recruited from YACs on grief from research by Egan and Breen comprised of 18–24-year-olds. A further n = 2 people were included by Payne and Raghav, and n = 1 by Egan from a previous YAC.

The YAC were 18.9 years (SD = 2.7, range 15–24 years), 93 % (n = 14) self-identified as female and 7 %, (n = 1) as male, no-one identified as non-binary or gender diverse. We aimed to ensure the intervention was relevant to a range of cultural backgrounds. YAC members resided in Australia (n = 12), Malaysia (n = 2), and India (n = 1) and reported ethnicity as Anglo-Celtic/European (n = 6), Chinese Malay (n = 2), Indian (n = 2), and Iranian, Han Chinese, African American and Zimbabwean (all n = 1 each). Members of the YAC signed a consent form and a parent provided signed parental consent for the one YAC member who was aged under 16 years.

2.4.1 Intervention development stage 1

The first stage included the first author writing a draft of an 8-module intervention. A lived experience expert who was a co-author edited the intervention to ensure relevance to young people with grief, then it was again edited by the research team. The draft was based on published materials on CBT for prolonged grief (Boelen et al., 2021; Eisma et al., 2015; Lenferink et al., 2023), and on imagery techniques for traumatic bereavement (Duffy and Wild, 2023; Murray and El-Leithy, 2023; Wild et al., 2023). The resulting word document draft contained 53 pages with text, worksheets, and exercises. The draft was circulated to the YAC who provided two rounds of feedback via email to the questions: ‘what did you like’, ‘what did you not like’ and ‘any suggestions for change’. YAC members were reimbursed AUD$75 for each meeting/email feedback, for two hours of time.

Initial recommendations included adding further information on normalising grief and changes to text to make it easier to understand. Suggestions also included extending information on mental health support helplines to include an LGBTQI+ service and a helpline for Aboriginal and Torres Strait Islander people. The first author then revised the intervention, and the YAC provided a second round of email feedback. After we added these sections, a YAC member commented “the helpline section feels more inclusive now”. At each stage, the first author provided individual feedback via email to each member outlining how recommendations had been implemented. The draft was then converted into an interactive website.

2.4.2 Intervention development stage 2

The draft website intervention was discussed with the YAC in two meetings (each one-hour duration). YAC members were sent the website modules to view five days before each meeting. Members who could not attend provided feedback via email. YAC members were asked what they liked, did not like, and any suggestions for change. They also chose the website name: www.sailgrief.org.

The feedback and recommendations were recorded in field notes of the meetings by the first author. The feedback was synthesised by the first author following conventional content analysis (Hsieh and Shannon, 2005), involving identifying the main content areas that the YAC suggested to change and implementing these changes. The YAC provided recommendations for changes to images, text content, exercises, and interactive worksheets which were implemented (see Table 2). Feedback loops were included where the first author provided feedback in meetings and via individual emails on how their suggested changes had been implemented.Table 2 Summary of recommendations from young people on the website intervention.

Table 2Positive features of the website		
Liked aesthetics.	Liked the design, images, simplistic nature.	
Content.	Liked information on normalising grief, imagery, experiments, pleasant event scheduling.	
Interactive elements.	Liked the videos, case examples and worksheets with interactive boxes to complete tasks.	


	
Suggestions for changes to the website	
Changes to aesthetics.	Increase use of bright colours, increase font size, include more images, break the website into smaller webpages with less information.	
Changes to text and content of intervention.	Specific sentences recommended to include on:
- normalising grief.

- additional symptoms and feelings associated with grief.

- keeping people motivated to continue in the program.

- information that tasks on exposure can be confronting.

	
Changes to ‘Get Help’ tab of website.	Addition of further helplines to increase inclusivity (e.g., LGBTQI+ helpline).
Add further explanation of what the resources and helplines offer for young people.	
Reduction in the amount of text.	Reduce the number of sentences.
Use dot points in some sections instead of sentences.	

2.4.2.1 Positive features of the website

YAC members provided a range of positive views on the draft website (Table 2). They liked the plain language and style of the website i.e., “I like the simplistic vibe” and “I feel like the information in each module is wonderful and clear with simple language. At the end of the module, teenagers will understand what they are going through.” Several young people noted the appeal of images, for example “I like the photos/graphics used”. When asked to comment on the Introductory module which addresses the unguided format of the intervention, an example comment was “great that it's self-paced but also a recommendation (two modules per week).” YAC members reviewed tabs on the homepage, including the ‘about us’ tab, which contained a description of the lived experience and academic team. An example comment was “experts in the field, can provide comfort that this has been designed with people who know what they are talking about.”

YAC members commented that the focus on a range of losses, not only bereavement, was a positive feature “great to provide a range of grief/loss experiences that aren't death-related”. They talked about the relevance of examples, e.g., in relation to pleasant event scheduling, “pretty good range of activities… good to see other low maintenance ideas like downloading a new album, dancing, listening to music in the shower.” YAC members provided positive feedback on video case examples, saying it “feels genuine to hear from young people talking about real issues.” Numerous YAC members outlined the appeal of sections on normalising grief i.e., “good to dismiss ‘stages of grief’ and be upfront about not providing any ‘move on’ type of advice, this would put me at ease if I were completing the course.”

2.4.2.2 Suggestions for changes to the website

The YAC members suggested ways to simplify language. We changed specific sentences that the YAC said were difficult to understand. We also implemented the change suggested by several members to reduce sentences and increase dot points to “make the information easier to read and digest”. Several people also provided sentences for the website. Example sentences written by YAC members included “grief can be experienced in many ways, and often many people may grieve about similar losses in different ways”, and “grief is personal and is not defined by other peoples' expectations on what to grieve or not to grieve”.

Many YAC members provided specific recommendations on aesthetics. Changes implemented included bold font of headings and particular words. YAC members recommended including a warning at the start of exposure tasks so “this acknowledges that this task may be confronting, and that the feeling is acknowledged”. We asked the YAC for specific feedback about whether exposure tasks and memory rescripting was too distressing for an unguided intervention. YAC members told us “module six [exposure] allows us to face emotions and it was really impactful.” and “avoidance in module six is an extremely relevant and poignant choice of topic…extremely relevant for young people's experience of grief.”

A YAC member also commented that it was helpful that modules six (exposure) and seven (memories) were later in the program, noting it could have been overwhelming if presented earlier. Another YAC member suggested that it would be helpful to include a pictorial cartoon depicting the steps to complete memory updating and the member created a cartoon that we included. YAC members were unanimous in positive feedback on this cartoon, for example, one member commented “I love the comic strip on page two (module seven) it's a really interesting way to show information and is a creative way to display it.”

2.4.3 Final co-designed intervention

The final end-product of the intervention created consists of eight modules and is unguided (see Table 3). Each module contains six to nine webpages of text, graphics, video and audio vignettes, worksheets, and interactive components. There is also one brief ‘introductory module’ consisting of two webpages which provides an overview of the program, brief material that normalises grief and the suggestion to complete two modules per week, over four weeks. Given that each module is between six to nine webpages in length, the modules are brief and are expected to take no more than 30 min to complete. If participants complete two modules per week, they would spend one hour or less per week on treatment. The suggestion of completing two modules per week over four weeks is based on another ICBT study in adolescents where this length of treatment was found to be acceptable (Shu et al., 2019).Table 3 Intervention outline.

Table 3Module	Cognitive and behavioural strategies	Title in the online intervention	
1	Psychoeducation about normalising grief	Understanding grief and loss	
2	Cognitive formulation of maintaining processes	Drawing a diagram to understand grief and loss	
3	Behavioural activation, improving self-care (eating/sleeping routines), and cognitive strategies of responsibility pie-charts to challenge self-blame	Activities and re-considering self-blame	
4	Cognitive strategies: Behavioural experiments to challenge negative thinking and predictions	Changing thinking through experiments	
5	Cognitive strategies: Thought records and continuums to challenge all or nothing and other negative thinking styles	Changing thinking through taking a different perspective	
6	Behavioural strategies: Exposure to avoided triggers, memories, and images, including writing a letter	Working with the loss	
7	Cognitive strategies: Imagery rescripting and memory updating. Behavioural strategies: Problem-solving	Looking at memories and images and problem-solving	
8	Relapse prevention	Planning for the future	

2.5 Role of the youth advisory committee in the RCT

The YAC that co-designed the intervention will meet six times during the pilot RCT. The YAC will advise on recruitment methods, distribute recruitment materials to their peers, and co-produce dissemination outputs, including a lay summary, infographic and a two-minute video to disseminate outcomes.

2.6 Participants for the proposed RCT

The inclusion criteria for the pilot RCT are: (i) self-identified experience of grief associated with death or non-death losses, (ii) age 13 to 18 years inclusive, and (iii) residing in Australia. Exclusion criteria are: (i) high suicide risk as indicated by the Columbia Suicide Screening Questionnaire (CSS; Posner et al., 2011), and (ii) parents do not provide consent. A minimum number of 85 participants will be required for the study based on the power calculation. The estimated effect size was determined by Breen et al.'s (2023) review of 42 grief interventions for 14–24 year olds, which found a large effect size (pooled d = −0.81) in anxiety symptom reduction for CBT for grief interventions. We conducted a priori power analysis using G*Power 3.1 (Faul et al., 2007), with two timepoints, between-groups, p = .05, estimated effect size = 0.8, finding we require a minimum of 68 participants to be randomised (n = 34 intervention, n = 34 waitlist control) to meet sufficient power (0.90). We also estimated an attrition rate of 25 %, based on the largest study of face-to-face CBT for prolonged grief in children and adolescents (Boelen et al., 2021), meaning that a sample size of 85 participants would be required, accounting for the estimated attrition rate.

2.7 Recruitment

Participants will be recruited through social media, organisations that provide camps for bereaved families, and Psychology students at Curtin University (aged 18 or under). Participants will be reimbursed AUD$50 at post-treatment (both intervention and waitlist control group), and AUD$25 at follow-up for the intervention group only. Psychology students will receive course credit points instead of payment, awarded at post-treatment (both intervention and waitlist control group) and follow-up (intervention group only). Participants will receive either payment or course credits regardless of whether they completed the intervention and/or questionnaires.

2.8 Procedure

Adolescents and if aged 17 or below also their parent/caregiver will register online on the study website www.sailgrief.org. Adolescents and parents will be asked to read digital participant information sheets, click on a consent button, and provide contact details, age, and country of residence. Eligible participants will complete pre-intervention measures and brief questions on treatment status (see supplementary materials). If an adolescent scores four or higher on the CSS screener, the complete measure will be conducted. Adolescents at high risk who are aged 17 or below will be contacted via telephone, and if unavailable after repeated telephone contact, via email including to their parent, to inform they do not meet criteria and information provided on referral sources. Participants aged 18 years will not be asked to provide parental consent and their parents will not be notified via telephone contact if they are at high risk. Rather, the participant will be directly contacted and offered referral suggestions. These adolescents will be able to access the intervention so as not to exclude them from potential benefit, but they will not be included in the trial.

After completion of baseline measures, eligible participants will be randomly assigned by the research assistant via an online random number generator to the intervention or a waitlist control group. This will include simple rather than block randomisation and due to budget and time constraints of the number of staff involved, the research assistant will not be blind to treatment allocation. Participants in the intervention group will be emailed the password to gain access to the intervention at www.sailgrief.org. Participants will be informed to complete two modules per week, over a four-week period. Generic email reminders will also be sent each week during the four-week period, to remind participants to complete two modules per week. Reminders will include a link to the website, and a prompt to complete a weekly CSS. If participants do not complete the weekly CSS, they will be sent three email/text reminders. In the waitlist control group, once the participants have completed the post-waitlist measures, they will be emailed the intervention and not followed up beyond this point.

The intervention is unlocked so that participants can move between modules, and each module is followed by a module completion questionnaire. To protect participants' identities, no data is stored on the website. At the start of the intervention, participants are instructed to ensure they save completed worksheets on their device as once they have left the website, the information will not be saved.

Participants will be emailed reminders with links provided to complete questionnaires at each timepoint (Table 1); Time 1 (baseline), Time 2 (four-weeks), and for intervention participants only, a follow-up assessment at Time 3 (eight-weeks after baseline assessment). Following a similar protocol (O'Brien et al., 2022), adverse effects of the intervention will be assessed by examining clinical significance and the Reliable Change Index (RCI; Jacobson and Truax, 1991). The RCI will provide an indication of change. Where there is evidence of an RCI demonstrating a deterioration after intervention this may indicate a potential negative effect of the intervention. If a participant has deteriorated as indicated through clinical significance analysis, they will be contacted via telephone if aged 18, or if age 17 and below their parent/caregiver will be informed via telephone and if unavailable by repeated phone contact, via email, with instructions to attend their general practitioner for referral to a mental health professional and reminded of emergency helplines while waiting for the referral. Negative outcomes of the intervention will be reported in the trial paper.

2.9 Outcome measures

2.9.1 Clinical efficacy outcomes

2.9.1.1 Revised Children's Anxiety and Depression Scale (RCADS) – 11 item version (Radez et al., 2021)

The 11-item short version of the RCADS (Chorpita et al., 2000) will be used to assess anxiety (primary outcome) and depressive symptoms (secondary outcome). The 11-item version, consisting of six items addressing anxiety and five depressive symptoms, demonstrates excellent internal consistency and comparable sensitivity and specificity to the 47-item version (Radez et al., 2021). Adolescents answer the questions on anxiety and depressive symptoms (e.g., ‘I have no energy for things’ on a 4-point Likert scale ranging from 0 = never to 3 = always. Scores are summed, and higher scores indicate greater symptoms of anxiety and depression. Clinical cut-offs for anxiety symptoms for boys are a score of 5 and for girls a score of 9. Clinical cut-offs for depressive symptoms for boys are 8 and for girls a score of fourteen (Radez et al., 2021). The additional two items to assess impact will also be included to assess distress and interference from symptoms (Radez et al., 2021) e.g., ‘how much do these difficulties get in the way of your everyday life in school?’. The two distress and interference items are answered on a 4-point Likert scale from 0 = not at all to 3 = a great deal. The distress and interference items are summed, with greater scores indicating a higher degree of interference and distress from symptoms of anxiety and depression.

2.9.1.2 The Prolonged Grief Disorder scale - revised (PG-13-R; Prigerson et al., 2021)

This 13-item scale will assess symptoms of prolonged grief and has good internal consistency (Prigerson et al., 2021). Based on examination of items reflecting particular prolonged grief symptoms in the United Kingdon, United States and the Netherlands, the PG-13-R was developed. The scale is applicable to bereavement, hence will only be answered and reported for participants who identify bereavement of a person. A question to address type of loss will be added to the outcome measures following the initial question of the PG-13-R, ‘Have you lost someone significant to you? If no, please describe the nature of your loss (e.g., death of a pet, parents’ divorce, relationship break-up) with an open text response box. The PG-13-R version described by Prigerson et al. (2021) with adults will be used. Participants answer questions (e.g., ‘do you feel yourself longing or yearning for the person who died?’) on a 5-point Likert scale from 1 = not at all to 5 = overwhelmingly, with higher scores indicating a greater degree of prolonged grief.

2.9.1.3 Children's Revised Impact of Events Scale (CRIES; Perrin et al., 2005)

The eight-item version of the CRIES will be used to measure symptoms of PTSD. The scale has good internal consistency, construct and concurrent validity in children and adolescents aged 8–18 years (Giannopoulou et al., 2006). At the start of the CRIES an extra sentence will be added ‘regarding the loss which you just described’ before the instructions to ensure that participants are answering the items relating to their bereavement or loss rather than another trauma. Participants answer questions (e.g., do you try to remove it from your memory?’) about the event indicating the degree of PTSD symptoms they experienced in the past 7 days on a Likert scale of 0 = not at all, 1 = rarely, 3 = sometimes and 5 = often. Scores are summed and total scores range from 0 to 40, with higher scores indicating higher PTSD symptoms. A score of 17 and above has been defined in one large study of Dutch children and adolescents as indicating a clinical cut-off for PTSD (Verlinden et al., 2014).

2.9.1.4 WHO-5 well-being index (Bech et al., 2003)

This five-item scale will be used to measure subjective well-being. The measure has good reliability and validity, including as a screening measure of depressive symptoms in adolescents (Blom et al., 2012; Topp et al., 2015). The five items are (1) being in good spirits, (2) feeling relaxed, (3) having energy, (4) waking up fresh and rested and (5) being interested in things. The items are rated on a 6-point Likert scale from 0 = not present to 5 = constantly present. Scores are summed with total scores ranging from 0 to 25. Higher scores indicate better well-being. A cut-off score of below 13 has been defined as indicating poor well-being (Blom et al., 2012).

2.9.2 Measures of feasibility and acceptability

2.9.2.1 Feasibility

The feasibility of the intervention will be assessed by examining the participant flow-chart (Fig. 1).

2.9.2.2 Acceptability

After each module, four items will be administered relating to adherence to the module, acceptability via usefulness and ease of use, and time spent on the module. The items (see supplementary materials) were adjusted from Thiels et al. (1998). Participants will also answer 10 questions adapted from Egan et al. (2021) to examine acceptability, and nine online open-ended questions about intervention feedback (see supplementary materials). At post-treatment participants will also be invited to engage in a semi-structured interview on MS Teams about their experiences and perceptions of the intervention (see supplementary materials).

2.10 Data analysis

To examine preliminary evidence for clinical efficacy, Generalised Linear Mixed Models (GLMM) will be conducted in SPSS via intent-to-treat analysis. GLMM will be used to examine outcomes of anxiety, depression, well-being, and PTSD symptoms through between groups effect sizes to assess the interaction between the intervention group versus waitlist control and time, and group and time main effects between pre-and post-intervention in the intervention versus waitlist-control group. There are three time points: pre-intervention, post-intervention, and one-month post-intervention follow-up (for the intervention group only). Following another RCT (Hoiles et al., 2022) with the same design of a waitlist-control group at post-treatment but no control group at follow-up, another GLMM will also be conducted comparing pre-treatment, post-treatment, and follow-up scores for the intervention group only. The main effect for time will be examined to determine if there is a maintenance of any intervention effects at post-treatment observed at follow-up.

As noted, negative or adverse outcomes of the intervention will be assessed through clinically significant deterioration on measures of anxiety, depression, PTSD and prolonged grief. There are no planned interim or subgroup analyses.

To evaluate acceptability, conventional content analysis (Hsieh and Shannon, 2005), will be used to examine online open-ended questions and qualitative semi-structured interview questions. Acceptability will also be analysed by reporting descriptive statistics for module completion questions. Feasibility will be examined through whether there is sufficient recruitment of participants within a 14-week period, and study attrition rates (25 % or less = acceptable attrition). Following the recommendations of Pearson et al. (2020), we will interpret feasibility using the ‘traffic light system’ to indicate whether to proceed with a larger RCT (i.e., red – stop, not feasible; yellow – continue, feasible with modifications; green – continue, feasible without modifications). Specifically, the decision of whether to continue with a further RCT after the pilot study will be operationalised as green (feasible, continue) = attrition 25 % or less and recruitment of 85 participants or more, amber (feasible, continue with modifications) = attrition between 26 %–50 % and recruitment of between 40 and 84 participants, and red (not feasible, stop) = attrition 51 % and above, and recruitment of 39 or less participants.

3 Discussion

The aim of this paper was to describe the development of co-designed unguided ICBT-G-A. We have also outlined our plans to trial the intervention in a pilot RCT. As this is the first internet-based CBT intervention for grief in adolescents, this study offers a unique contribution to the literature, addressing a critical gap in the literature on evidence-based internet interventions for grief among adolescents. Should this intervention prove effective, it may improve access to and scalability of evidence-based interventions for adolescents who have experienced grief. An important goal of the proposed study is to co-produce the research with consumers who have lived experience of grief to increase relevance of the intervention. Outcomes for adolescents will be improved by a youth advisory committee (1) co-designing an intervention that is relevant and appealing, and (2) advising on ways to recruit adolescents to the study and disseminate the intervention if proven effective. In addition, through qualitative feedback we will develop an understanding of acceptability and ways to further improve the intervention.

A potential risk to the planned RCT is insufficient recruitment of participants. This will be addressed through partnerships with adolescent mental health and grief support providers to improve feasibility of recruitment. Another potential problem is participant attrition, which has been noted in internet interventions (Andersson, 2016). We co-designed the intervention extensively with consumers to improve the relevance, appeal, and relatability of the intervention to reduce attrition. However, while we co-designed the intervention with 15 YAC members, a limitation is that gender diversity was restricted among members, with the majority being female (93 %), only one male, and no people who identified as non-binary or gender diverse. Therefore, it is unclear whether the co-designed intervention is relevant across a range of genders.

Examination of feasibility is an important aim of the study. We will use pre-defined criteria via sufficient participants recruited and study attrition to determine whether it is feasible to proceed to a fully powered RCT. Data on feasibility and acceptability will inform the next steps in refinement of the intervention and whether to progress with a larger RCT. However, a limitation of the design is that because acceptability will be assessed via qualitative open-ended questions, there are no cut-offs or norms that can be applied to determine what is sufficient, or insufficient, acceptability.

There are several potential ethical considerations. Given that the intervention is unguided and delivered via the internet, we have a protocol that includes screening out of participants at high suicide risk, and weekly monitoring of participants in the intervention group on their scores on a suicide risk measure. It is possible that participants in the control group may have an increase in distress or suicide risk that is not detected due to not measuring risk in this group, however we will provide detailed lists and suggestions of emergency and mental health services to this group at both pre and post waitlist assessment points. Another ethical consideration is that the control group need to wait to access the intervention. To mitigate this, we have chosen to make the intervention available to the control group after the post-treatment assessment, so that they only wait four weeks to access it, although we acknowledge that this adds a limitation to the design of our RCT as there will be no control group at the follow-up assessment.

If the intervention is demonstrated to be feasible, acceptable, and initial data demonstrates preliminary evidence for efficacy, a larger RCT will be conducted. Internet interventions for grief hold promise in providing an accessible way for adolescents to access help for the highly prevalent experience of grief.

Role of the funding source

This work was supported by a grant from Channel 7 Telethon Trust, Western Australia. The funder had no role in the collection, analysis, and interpretation of data, in writing of the report or the decision to submit the paper for publication. KS is supported by the 10.13039/501100000265 Medical Research Council [MR/V001841/1 ] and the National Institute for Health and Care Research (10.13039/501100000272 NIHR ) Oxford Health Biomedical Research Centre (KS: NIHR203316 ). The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A Supplementary data

Supplementary material

Image 1

Acknowledgements

We thank our Youth Advisory Committee for co-designing the intervention and co-producing this research.

☆ Trial Registration: This trial was registered in March 2024 with the Australian and New Zealand Clinical Trials Registry (ACTRN12624000337572).

Appendix A Supplementary data to this article can be found online at https://doi.org/10.1016/j.invent.2024.100771.
==== Refs
References

Andersson G. Internet-delivered psychological treatments Annu. Rev. Clin. Psychol. 12 2016 157 179 10.1146/annurev-clinpsy-021815-093006 26652054
Bech P. Olsen L.R. Kjoller M. Rasmussen N.K. Measuring well-being rather than the absence of distress symptoms: a comparison of the SF-36 mental health subscale and the WHO-five well-being scale Int. J. Methods Psychiatr. Res. 12 2 2003 65 115 10.1002/mpr.145 12830300
Blom E.H. Bech P. Hogberg G. Larsson J.O. Serlachius E. Screening for depressed mood in an adolescent psychiatric context by brief self-assessment scales: testing psychometric validity of WHO-5 and BDI-6 indices by latent trait analyses Health Qual. Life Outcomes 10 2012 149 10.1186/1477-7525-10-149 23227908
Boelen P.A. Lenferink L.I.M. Spuij M. CBT for prolonged grief in children and adolescents: a randomized clinical trial Am. J. Psychiatry 178 4 2021 294 304 10.1176/appi.ajp.2020.20050548 33472391
Breen L.J. Greene D. Rees C.S. Black A. Cawthorne M. Egan S.J. A co-designed systematic review and meta-analysis of the efficacy of grief interventions for anxiety and depression in young people J. Affect. Disord. 335 2023 289 297 10.1016/j.jad.2023.05.032 37196936
Chorpita B.F. Yim L. Moffitt C. Umemoto L.A. Francis S.E. Assessment of symptoms of DSM-IV anxiety and depression in children: a revised child anxiety and depression scale Behav. Res. Ther. 38 2000 835 855 10.1016/S0005-7967(99)00130-8 10937431
delCarpio L. Rasmussen S. Paul S. A theory-based longitudinal investigation examining predictors of self-harm in adolescents with and without bereavement experiences Front. Psychol. 11 2020 1153 10.3389/fpsyg.2020.01153 32581958
Duffy M. Wild J. Living with a loss: a cognitive approach to prolonged grief disorder – incorporating complicated, enduring and traumatic grief Behav. Cogn. Psychother. 51 2023 645 658 10.1017/S1352465822000674 37170765
Egan, S.J., McEvoy, P.M., Wade, T.D., Ure, S., Johnson, A.R., Gill, C., Greene, D., Anderson, R.A., Mazzucchelli, T.G., Wilker, L., Brown, S., & Shafran, R. (2021). Unguided low intensity psychological intervention for anxiety and depression during the COVID-19 pandemic: a randomised trial. Behav. Res. Ther., 144, 103902. doi:1016/j.brat.2021.103902.
Egan S.J. Neal J. Ure S. Callaghan T. Ho P. Shafran R. Wade T.D. The development of co-designed parent-supported cognitive behaviour therapy for perfectionism in adolescents with eating disorders: initial feasibility and acceptability J. Eat. Disord. 11 2023 138 10.1186/s40337-023-00860-6 37592307
Ehlers A. Clark D.M. A cognitive model of posttraumatic stress disorder Behav. Res. Ther. 38 4 2000 319 345 10.1016/S0005-7967(99)00123-0 10761279
Eisma M.C. Boelen P.A. van den Bout J. Stroebe W. Schut H.A.W. Lancee J. Stroebe M.S. Internet-based exposure and behavioral activation for complicated grief and rumination: a randomized controlled trial Behav. Ther. 46 2015 729 748 10.1016/j.beth.2015.05.007 26520217
Faul, F., Erdfelder, E., Lang, AG. et al. G*power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav. Res. Methods 39, 175–191 (2007). doi:10.3758/BF03193146.
Field T. Romantic breakups, heartbreak and bereavement Psychology 2 4 2011 382 387 10.4236/psych.2011.24060
Giannopoulou I. Smith P. Ecker C. Strouthos M. Dikaiakou A. Yule W. Factor structure of the Children’s revised impact of event scale (CRIES) with children exposed to earthquake Personal. Individ. Differ. 40 5 2006 1027 1037 10.1016/j.paid.2005.11.002
Harrison L. Harrington R. Adolescents’ bereavement experiences: prevalence, association with depressive symptoms and use of services J. Adolesc. 24 2001 159 169 10.1066/jado.2001.0379 11437477
Hoiles K.J. Rees C.S. Kane R.T. Howell J. Egan S.J. A pilot randomised controlled trial of guided self-help cognitive behaviour therapy for perfectionism: impact on diagnostic status and comorbidity J. Behav. Ther. Exp. Psychiatry 76 2022 101739 10.1016/j.jbtep.2022.101739
Hsieh H.F. Shannon S.E. Three approaches to qualitative content analysis Qual. Health Res. 15 9 2005 1277 1288 10.1177/1049732305276687 16204405
Jacobson N.S. Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research J. Consult. Clin. Psychol. 59 1991 12 19 10.1037//0022-006x.59.1.12 2002127
Lenferink L.I.M. Eisma M.C. Buiter M.Y. de Keijser J. Boelen P.A. Online cognitive behavioral therapy for prolonged grief after traumatic loss: a randomized waitlist-controlled trial Cogn. Behav. Ther. 52 5 2023 508 522 10.1080/16506073.2023.225744 37341432
Morris A.T. Gabert-Quillen C. Friebert S. Carst N. Dalahunty D.L. The indirect effect of positive parenting on the relationship between parent and sibling bereavement outcomes after the death of a child J. Pain Symptom Manage. 51 2016 60 70 10.1016/j.jpainsymman.2015.08.011 26387829
Murray H. El-Leithy S. Loss Working with Complexity in PTSD: A Cognitive Therapy Approach 2023 Routledge Oxford, UK 167 178
Norton M.J. Co-production within child and adolescent mental health: a systematic review Int. J. Environ. Res. Public Health 18 22 2021 10.3390/ijerph182211897
O’Brien A. Anderson R. Mazzucchelli T. Egan S.J. A protocol for unguided internet self-help cognitive behaviour therapy for perfectionism in adolescents at-risk of eating disorders Internet Interv. 29 2022 10556 10.1016/j.invent.2022.100565
Paul S. Vaswani N. The prevalence of childhood bereavement in Scotland and its relationship with disadvantage: the significance of a public health approach to death, dying and bereavement Palliative Care and Social Practice 14 2020 1 12 10.1177/2632352420975043
Pearson N. Naylor P.J. Ashe M.C. Fernandez M. Yoong S.L. Wolfenden L. Guidance for conducting feasibility and pilot studies for implementation trials Pilot and Feasibility Studies 6 2020 167 10.1186/s40814-020-00634-w 33292770
Perrin S. Meiser-Stedman R. Smith P. The Children’s revised impact of events scale (CRIES): validity as a screening instrument for PTSD Behav. Cogn. Psychother. 33 4 2005 487 493 10.1017/S1352465805002419
Posner K. Brown G.K. Stanley B. Brent D.A. Yershova K.V. Oquendo M.A. Currier G. Melvin G. Greenhill, L…Mann, J. The Columbia-suicide severity rating scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults Am. J. Psychiatry 168 12 2011 1266 1277 10.1176/appi.ajp.2011.10111704 22193671
Prigerson H.G. Boelen P.A. Xu J. Smith K.V. Maciejewski P.K. Validation of the new DSM-5-TR criteria for prolonged grief disorder and the PG-13-revised (PG-13-R) scale World Psychiatry 20 2021 96 106 10.1002/wps.20823 33432758
Radez J. Waite P. Chorpita B. Creswell C. Orchard F. Percy R. Spence S.H. Reardon T. Using the 11-item version of the RCADS to identify anxiety and depressive disorders in adolescents Research on Child and Adolescent Psychopathology 49 2021 1241 1257 10.1007/S10802-021-00817-w 33792821
Salandino V. Verrastro V. Calaresi D. Barberis N. The effectiveness of cognitive behavioral therapy for prolonged grief in children and adolescents: a systematic review Int. J. Stress. Manag. 31 1 2024 66 85 10.1037/str0000301
Schleider J.L. The fundamental need for lived experience perspectives in developing and evaluating psychotherapies J. Consult. Clin. Psychol. 91 3 2023 119 121 10.1037/ccp0000798 36795434
Shu C. Watson H.J. Anderson R.A. Wade T.D. Kane R.T. Egan S.J. A randomized controlled trial of unguided internet cognitive behavior therapy for perfectionism: impact on risk for eating disorders Behav. Res. Ther. 120 2019 103429 10.1016/j.brat.2019.103429
Simbi C.M.C. Zhang Y. Wang Z. Early parental loss in childhood and depression in adults: a systematic review and meta-analysis of case-controlled studies J. Affect. Disord. 260 2020 272 280 10.1016/j.jad.2019.07.087 31521863
Smith K.V. Ehlers A. Cognitive predictors of grief trajetories in the first months of loss: a latent growth mixture model J. Consult. Clin. Psychol. 88 2 2020 93 105 10.1037/CCP0000438 31556649
Smith K.V. Ehlers A. Coping strategies as a causal mediator of the effect of loss-related memory characteristics and negative loss-related appraisals on symptoms of PGD PTSD and depression. Psychological Medicine 53 4 2023 1542 1551 10.1017/S0033291721003123 37010218
Spuij M. Reitz E. Prinzie P. Stikkelbroek Y. deRoose C. Boelen P.A. Distinctiveness of symptoms of prolonged grief, depression, and post-traumatic stress in bereaved children and adolescents Eur. Child Adolesc. Psychiatry 12 2012 673 679 10.1007/S00787-012-0307-4
Thiels C. Schmidt U. Treasure J. Garthe R. Troop N. Guided self-change for bulimia nervosa incorporating use of a self-care manual Am. J. Psychiatry 155 1998 947 953 10.1176/ajp.155.7.947 9659862
Topp C.W. Ostergaard S.D. Sondergaard S. Bech P. The WHO-5 well being index: a systematic review of the literature Psychother. Psychosom. 84 3 2015 167 176 10.1159/000376585 25831962
Uttley C.M. Adolescence, Pet Loss, Grief, and Therapeutic Interventions 2019 Routledge NY
Verlinden E. van Meijel E.P.M. Opmeer B.C. Beer R. de Roos C. Bicanic I.A.E. Lamers-Winkelman F. Olff M. Boer F. Lindauer R.J.L. Characteristics of the revised impact of event scale in a clinically referred Dutch sample J. Trauma. Stress 27 2014 10.1002/jts.21910 338-334
Wild J. Duffy M. Ehlers A. Moving forward with the loss of a loved one: treating PTSD following traumatic bereavement with cognitive therapy The Cognitive Behaviour Therapist 16 2023 e12 10.1017/s1754470X23000041
