
==== Front
BMJ Open
BMJ Open
bmjopen
bmjopen
BMJ Open
2044-6055
BMJ Publishing Group BMA House, Tavistock Square, London, WC1H 9JR

39122385
10.1136/bmjopen-2024-084620
bmjopen-2024-084620
Communication
Public Health
1724
1506
Evidence-informed, experience-based co-design: a novel framework integrating research evidence and lived experience in priority-setting and co-design of health services
http://orcid.org/0000-0001-9721-9225
Morley Claire 1claire.morley@utas.edu.au

Jose Kim 1Kim.Jose@utas.edu.au

Hall Sonj E 2sonj@sonjhall.com

Shaw Kelly 13kelly.shaw@kphealth.com.au

McGowan Deirdre 1deirdre.mcgowan@utas.edu.au

Wyss Martina 4MWyss@primaryhealthtas.com.au

http://orcid.org/0000-0002-4112-3491
Winzenberg Tania 1Tania.Winzenberg@utas.edu.au

1 Menzies Institute for Medical Research, University of Tasmania, Hobart, Tasmania, Australia
2 Bellberry Ltd, Eastwood, South Australia, Australia
3 KPHealth, Hobart, Tasmania, Australia
4 Primary Health Tasmania, Launceston, Tasmania, Australia
DrClaireMorley; claire.morley@utas.edu.au
None declared.

2024
8 8 2024
14 8 e08462024 1 2024
22 7 2024
Copyright © Author(s) (or their employer(s)) 2024. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.
2024
https://creativecommons.org/licenses/by-nc/4.0/ This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

ABSTRACT

Objective

To describe a new co-design framework termed Evidence-informed, Experience-based Co-design (E2CD).

Background

Involving consumers and clinicians in planning, designing and implementing services results in the end-product being more likely to meet the needs of end-users and increases the likelihood of their uptake and sustainability. Different forms and definitions of co-design have been described in the literature and have had varying levels of success in health service redesign. However, many fall short of including people with lived experience in all aspects of the process, particularly in setting priorities for service (re)design. In addition, health services need to deliver evidence-based care as well as care that meets the needs of users, yet few ways of integrating research evidence into co-design processes are described. This paper describes a new framework to approach co-design which addresses these issues. We believe that it offers a roadmap to address some of healthcare’s most wicked problems and potentially improve outcomes for some of the most vulnerable people in our society. We use improving services for people with high healthcare service utilisation as a working example of the Framework’s application.

Conclusion

Evidence-informed experience-based co-design has the potential to be used as a framework for co-design that integrates research evidence with lived experience and provides people with lived experience a central role in decision-making about prioritising and designing services to meet their needs.

Health policy
Organisation of health services
Patient Participation
Primary Health Care
PUBLIC HEALTH
Australian Government Research Training Program N/A Primary Health Tasmania N/A
==== Body
pmcBackground

Involving consumers and clinicians in the planning, design and implementation of services results in the end-product being more likely to meet the needs of end-users,1 2 thereby increasing the livelihood of uptake and sustainability of new services.3 Co-production is an umbrella term covering an array of evolving methods of including consumers in the design of health and social care services.4 While definitions and uses of different ‘co’ methods have caused some confusion and much discussion in the academic literature, a recent review has called for a shift in focus away from different definitions and towards advocating for the often similar principles of these methods to be operationalised and translated in practice.4

Experience-based co-design (EBCD) is one approach to health system (re)design that is founded on the principles of action research and design thinking.5 The co-design principles of equal partnership, openness, respect, empathy and design together have been suggested by one Australian group as critical to ensure that consumers are considered as equals at all stages of the design process.6 EBCD has been defined as an approach that enables staff and patients (or other service users) to co-design services and/or care pathways, together in partnership.7 Developed in the UK, EBCD is described as being about more than simply promoting greater patient participation, but about placing the patients’ experiences at the centre of the design process.8 The Point of Care Foundation (UK) proposes eight stages in EBCD, namely, observation, interviews of service users, development of a trigger film, service provider feedback, service user feedback, joint service user and provider feedback, co-design groups and celebration event.7 Bate and Robert state that the storytelling, or identification of ‘touch points’ from the trigger film, is the very basis of experience-based design.8 However, EBCD in practice has undergone many adaptations, often due to costs and time constraints.9 10 In many instances, this has resulted in a process commonly called accelerated EBCD, where adaptations include using stock ‘trigger films’, or absence of a celebration event.11 A recent systematic review of EBCD use highlighted its frequent adaptation due to costs and time constraints.10 Two phases, each with consumer involvement, were considered crucial, namely, experience gathering and co-design.10 Additionally, Green et al recommend supplementing the process with service provider experiences, due to the potential differences in priorities and beliefs of users and providers.10 This review noted that EBCD’s predominant use has been for service improvement in local settings but notes that its use could be extended beyond this, for example, for developing new services or redesign of health policy.10 However, it is less clear how EBCD can be integrated earlier into setting priorities for health service and policy design.

While health services need to meet the needs of its users, it is imperative that they also deliver evidence-based care. However, researchers commonly drive design processes in healthcare research, for example, undertaking literature reviews prior to engaging with stakeholders.12 This may narrow the range of potential solutions from the outset or leave teams conflicted as to whether published literature or stakeholder views should be given priority.12 Indeed, one qualitative evaluation of participants’ experiences of co-design identified the need to limit researcher domination of the co-design process to enable the establishment and maintenance of genuine partnerships.13 Approaches to using research evidence to supplement user experience in co-design are also infrequently described. One proposed approach to improving commissioning of health services integrates research evidence into a co-creation process in a limited way but does not emphasise the role of people with lived experience in decision-making or the importance of incorporating knowledge from lived experience in co-design.14 This is a critical gap that needs to be addressed.

People with high healthcare service use (HSU) are the small proportion of people who use a disproportionately high amount of healthcare services, determined by frequency of healthcare use such as hospital admissions, emergency department (ED) presentations, primary care visits or by incurring high healthcare costs.15 Indeed, there are calls for HSU to be considered a red flag of patients’ physical, mental, spiritual and social deprivation.16 Healthcare service redesign is a recognised priority for improving services for people with HSU,17 though not a panacea given how factors outside the health system, such as unemployment and homelessness, contribute to their issues.18 Interventions to reduce health service use by people with HSU have often been ineffective. Failure to recognise patient needs in intervention design may contribute to this. Reviews highlight the importance of considering patients’ unique needs19 and contexts.20 Despite this, the diverse needs of people with HSU are poorly understood. Systematic reviews of interventions are silent on whether and how lived experience has been incorporated into the design of interventions.2022 Failing to do this may lead to services being designed that do not address patient needs or barriers to access.23

This paper aims to describe a new framework building on EBCD to address the limitations of co-design approaches identified above, using the complex problem of HSU as an example to its potential application.

The framework: evidence-informed, experience-based co-design

We have developed a new framework called Evidence-informed, Experience-based Co-design (E2CD) that applies mixed methods research methodologies to integrate research evidence with lived experience in a co-design process in which people with lived experience have a central, decision-making role. Components of the framework are shown in figure 1, which also shows that it builds on EBCD,7 retaining its strong emphasis on the primacy of views of people with lived experience and of their involvement in co-design. The process is cyclic and iterative and can begin at any phase depending on the context in which (re)design is needed. Key features that together distinguish it from other approaches are that:

people with lived experience are kept central to the process by their lived experience informing each step and by them being involved in all phases including explicit decision-making roles for priority setting and co-design.

data analysis, lived experience and evidence synthesis are integrated and incorporated throughout the process, without a priori decisions made by the researchers.

researchers are responsive to participants’ needs and generate evidence as requested, thereby acting as equal partners in rather than drivers of the process. Throughout, participants are encouraged to voice questions that can be addressed by rapid, focused syntheses of existing evidence.

it has a priority-setting phase that gives people with lived experience a role in deciding co-design priorities.

services and their evaluations are co-designed in one contiguous process.

it is applicable to both the design of new services and redesign of existing services.

it is iterative. Phases can be revisited when more information emerges.

Figure 1 Graphic depiction of the Evidence-informed, Experience-based Co-design (E2CD) Framework. The process moves iteratively through 5 phases, with a focus on people with lived experience at all times as shown by their central placement in the Figure. Participants at any time can call on rapid responsive focused evidence synthesis to support the process as shown by this surrounding the process (outer circle of figure). In the understanding and refining phases, holistic understanding of the issue being examined from consumer, clinician, and policy-maker perspectives is sought. A formal process is then undertaken to develop a person-centred, stakeholder-informed, and evidence-based list of priority areas for new and/or improved models of care and service (Prioritising phase). The highest priority area is then addressed using an EBCD-based approach to co-design a new, or re-design an existing model of service delivery and its evaluation (Service and evaluation co-design phase). The improvements are implemented and evaluated with results from the Implementation and Evaluation phase informing service development for the next priority area.

Evidence-informed, experience-based co-design in action

We are piloting the application of the E2CD framework to the issue of HSU in our local context in Tasmania. Tasmania is Australia’s only island state and has a population that is older, more geographically dispersed and more socioeconomically disadvantaged than the rest of Australia,24 25 which magnifies the effects of HSU on access and availability of healthcare services. We use the issue of HSU as an example to illustrate how the E2CD framework could be applied (illustrated in figure 2).

Figure 2 Application of Evidence-informed, experience-based co-design (E2CD) to high healthcare service utilisation (HSU).

Understanding

The understanding phase has three components: understanding lived experience, using available data to understand HSU and including lessons from evaluations of initiatives already in place. To do this, we are:

evaluating a new, co-designed, nurse-led service that offers wrap-around care for people who have experienced ≥4 hospital admissions in a 12-month period26;

undertaking a data linkage study that links Tasmanian public hospital admission, ED presentation and cause of death data to enable a comprehensive understanding of the characteristics and patterns of different manifestations of HSU locally, and;

conducting a qualitative study interviewing Tasmanians with lived experience with different types of HSU to gain an in-depth understanding of the causes, consequences and needs of people with HSU in Tasmania (n=60 approx.). We will use purposive sampling to ensure a diverse range of people are included (eg, by age, sex, rurality and region).

Refining

In the refining phase, approximately 20 policymakers, clinicians (acute and primary/community care), commissioners of services, funders and other stakeholders will be interviewed to gain their perspectives on aspects of HSU such as its causes, consequences, priority areas and potential solutions. This will augment information from the understanding phase. Although using focus groups to gather participant experience is recommended in EBCD,7 we have chosen to undertake interviews as EBCD facilitators have reported that individually interviewing service providers has the added benefit of engaging and enhancing their commitment to the process.27 Service mapping will also be undertaken to identify services already available to people with HSU.

Prioritising

In the prioritising phase, a facilitated workshop with membership from all stakeholder groups previously involved including people with lived experience will be held to determine local priorities. This may be a service use pattern in one area (eg, ED presentations in a particular region) or a particular condition (eg, diabetes). We plan to use Nominal Group Technique28 as this is a consensus method used to avoid domination of decision-making by individuals who have a vested interest in the outcome29 and so aligns with the goals of E2CD.

Service evaluation and co-design

During this phase, a group, including members from all stakeholder groups involved thus far, will be convened to co-design/redesign a service to address the agreed highest priority. If needed, interviews with people who are potential end-users of that specific service will be undertaken, ensuring that potential end-users are central to the process. Evidence from all preceding phases will be communicated to the design group. EBCD principles will be used7 to ensure that the voice of people with lived experience is central to all decisions (figure 2). The evaluation framework for the new service will be co-designed simultaneously.

Implementation and evaluation

Involving all key stakeholders from the outset, including funders and commissioners of services, is intended to facilitate early implementation of the co-designed service.9 The evaluation of the new service will run concurrently with its implementation, with lessons learnt informing ongoing improvements within the service. This evaluation is important, as there is a reported lack of robust evaluation of interventions that are co-produced, leading to a paucity of evidence that co-designed services lead to improved health outcomes.30 Once this is implemented, the co-design group can revisit the priority setting phase and select the next priority for another co-design phase.

Rapid, responsive, focused evidence synthesis

To reduce the risk that researchers drive co-design outcomes rather than evidence synthesis being a tool used by people participating in co-design, at any phase of E2CD, stakeholders, including people with lived experience, can identify questions they would like the research team to address. The research team is equipped to rapidly respond to these requests, ensuring that the relevant information is summarised and provided to the stakeholders in a timely manner to support decision-making. Methods such as rapid reviews will be used for this.

Discussion

With current expenditure on health growing faster than GDP in most countries,31 it is critical to ensure that expenditure on new initiatives and services is channelled to those that are both evidence-based and acceptable to end-users. Many governments, including Australia, are moving towards ensuring that healthcare is value-based and that the voice of consumers is included at all stages of healthcare planning.32 Involving all potential end-users in prioritising and co-designing services may provide some assurance of acceptability and thereby sustainability of new initiatives. The E2CD approach we propose provides a framework that systematically involves all stakeholders in service (re)design from priority setting right through to implementation and evaluation, integrates research evidence with lived experience and gives people with lived experience a central, decision-making role. It offers a roadmap to address some of healthcare’s most wicked problems and potentially improve outcomes for some of the most vulnerable people in our society, such as people with HSU. In doing so, it has the potential to be a guide for researchers seeking to improve integration of lived experience into designing and testing complex interventions, and it has potential application in the commissioning of health services. In addition, it has the potential to address the researcher/consumer power imbalance often identified as an issue in health service redesign.

The potential for power imbalances between co-design participants is widely acknowledged.2 4 5 11 33 34 This is not always considered a negative, with one study finding that co-design can challenge traditional relationships between patients and clinicians and may blur the boundaries between practice and academia.2 Nonetheless, co-design leaders need to be aware that real and perceived power imbalances can derail the process and ensure no one group dominates. This can be achieved by setting up power-sharing structures and promoting collective ownership.2 13 33 34 The E2CD framework, in which researchers are equal partners rather than drivers of co-design, aims to avoid ‘researcher dominance’.13 Having a research team work in partnership to provide evidence synthesis as requested by the co-design participants could help prevent researchers’ a priori judgements of evidence from outweighing the evidence of lived experience and unduly influencing co-design outcomes. It may also lessen the power disparity that is so often an impediment to true partnerships in co-design.13

The E2CD framework can be applied to healthcare research and could be particularly useful when undertaking research aiming to improve health services or implement complex interventions. Many research funding bodies now strongly encourage or even mandate that researchers engage with consumers when designing research studies.35 36 They believe that the inclusion of consumer priorities, values and experiences is a means of ensuring research delivers fit-for-purpose outcomes that will be adopted by end-users.36 The Framework provides a way for researchers to work with people with lived experience and integrate evidence into choosing research topics to address (prioritisation) and the co-design of any interventions/models of care they choose to study. This is likely to result in a greater alignment of research with stakeholder needs and improve the impact of research performed. It is important, however, to ensure that all stakeholders are cognisant of the time, costs and risks associated with using co-design in research.37 These need to be acknowledged by funding bodies in terms of available funding and timelines and researchers in terms of deciding when to best use this approach.

The commissioning of health services is another area in which the E2CD framework could be useful. Commissioning is defined as a continual and iterative cycle involving the development and implementation of services based on needs assessment, planning, co-design, procurement, monitoring and evaluation38 and occurs in countries including the UK, New Zealand and Australia.39 Undertaking a needs analysis with end-user involvement is the basis for sound commissioning decisions.40 However, consumer involvement in the commissioning process to date has been mainly limited to making changes to minor aspects of service provision rather than any engagement in strategic planning or priority setting.41 The lack of flexibility in how funds can be expended has been identified as another constraint, as it limits the ability to be innovative with services at a local level.39 41 42 We believe that the E2CD framework could help address these limitations. First, if adhered to, it provides a blueprint for the incorporation of lived experience and the involvement of end-users, including consumers, at every stage of the commissioning process, ensuring that the expressed needs of local communities are understood by providers and incorporated into service (re)design. Second, it provides a structure for Primary Heath Networks to engage proactively within their communities to identify local needs and generate evidence to lobby for funds to address local priorities.

This article has some limitations. First, E2CD describes an overarching approach rather than a detailed methodology, so when applying it, users will need to choose methods appropriate for each phase. These choices will depend on factors including the context in which E2CD occurs, the health issue it aims to address and any specific barriers to engagement for people with lived experience. Choosing sensitive engagement processes and tailoring methods for priority-setting and co-design to meet the needs of specific groups of people with lived experience will be critical to its successful application. Second, we use HSU as an example to illustrate the potential utility of the E2CD framework. Therefore, a detailed explanation of the specific methods to be used in its application is beyond this article’s scope. Finally, as consumers were not involved in developing the framework, evaluating its use in practice with people with lived experience is essential.

Conclusion

As the cost of healthcare continues to rise, it is crucial that services deliver evidence-based care that meets the needs of its users. Involving all stakeholders in decisions regarding prioritising areas for service provision as well as designing services increases the likelihood that they will be embraced by end-users and outcomes will be sustained. While not a panacea, evidence-informed, experience-based co-design provides a framework to potentially address some of healthcare’s most wicked problems.

Review Process File
8 8 2024

Funding: This work is supported by Primary Health Tasmania under the Australian Government’s Primary Health Networks Program and by an Australian Government Research Training Program (RTP) Scholarship - DM, a doctoral candidate, is a recipient of scholarship funding from both entities.

Prepublication history for this paper is available online. To view these files, please visit the journal online (https://doi.org/10.1136/bmjopen-2024-084620).

Ethics approval: Not applicable.

Provenance and peer review: Not commissioned; externally peer reviewed.
==== Refs
References

1 Boivin A Lehoux P Lacombe R et al Involving patients in setting priorities for healthcare improvement: a cluster randomized trial Implementation Sci 2014 9 24 10.1186/1748-5908-9-24
2 Grindell C Coates E Croot L et al The use of co-production, co-design and co-creation to mobilise knowledge in the management of health conditions: a systematic review BMC Health Serv Res 2022 22 877 10.1186/s12913-022-08079-y 35799251
3 Wiles LK Kay D Luker JA et al Consumer engagement in health care policy, research and services: a systematic review and meta-analysis of methods and effects PLoS ONE 2022 17 e0261808 10.1371/journal.pone.0261808 35085276
4 Masterson D Areskoug Josefsson K Robert G et al Mapping definitions of co‐production and co‐design in health and social care: a systematic scoping review providing lessons for the future Health Expect 2022 25 902 13 10.1111/hex.13470 35322510
5 Moll S, D Miatello A Murray-Leung L et al Co-designing services for youth with mental health issues: novel elicitation approaches Int J Qual Methods 2019 1 1 13
6 Agency for Clinical Innovation A Guide to Build Co-Design Capability In: ACI Agency for Clinical Innovation 2019
7 The Point of Care Foundation EBCD: experience-based co-design toolkit london, UK: the point of care foundation n.d. Available https://www.pointofcarefoundation.org.uk/resource/experience-based-co-design-ebcd-toolkit/
8 Bate P Robert G Experience-based design: from redesigning the system around the patient to co-designing services with the patient Qual Saf Health Care 2006 15 307 10 10.1136/qshc.2005.016527 17074863
9 Carr VL Sangiorgi D Büscher M et al Integrating evidence-based design and experience-based approaches in healthcare service Design HERD 2011 4 12 33 10.1177/193758671100400403 21960190
10 Green T Bonner A Teleni L et al Use and reporting of experience-based codesign studies in the healthcare setting: a systematic review BMJ Qual Saf 2020 29 64 76 10.1136/bmjqs-2019-009570
11 Robert G Cornwell J Locock L et al Patients and staff as codesigners of healthcare services BMJ 2015 350 g7714 10.1136/bmj.g7714 25670179
12 Rousseau N Turner KM Duncan E et al Attending to design when developing complex health interventions: a qualitative interview study with intervention developers and associated stakeholders PLoS ONE 2019 14 e0223615 10.1371/journal.pone.0223615 31613913
13 Pallesen KS Rogers L Anjara S et al A qualitative evaluation of participants’ experiences of using co-design to develop a collective leadership educational intervention for health-care teams Health Expect 2020 23 358 67 10.1111/hex.13002 31999883
14 Karnon K Lo K In-DEPtH framework: evidence-informed, co-creation framework for the design, evaluation and procurement of health services BMJ Open 2019 9 e026482 10.1136/bmjopen-2018-026482
15 Shukla D Faber E Sick B Defining and characterizing frequent attenders: systematic literature review and recommendations J Patient Cent Res Rev 2020 7 255 64 10.17294/2330-0698.1747 32760757
16 Huang M van der Borght C Leithaus M et al Patients’ perceptions of frequent hospital admissions: a qualitative interview study with older people above 65 years of age BMC Geriatr 2020 20 332 10.1186/s12877-020-01748-9 32894056
17 Knox K Greysen SR Patient perspective is critical in developing interventions for frequently admitted patients J Hosp Med 2019 14 573 4 10.12788/jhm.3250 31532360
18 Canadian Institute for Health Information Pan-Canadian Forum on High Users of Health Care Toronto, Canada Canadian Institute for Health Information 2014
19 Goodwin A Henschen BL O’Dwyer LC et al Interventions for frequently hospitalized patients and their effect on outcomes: a systematic review J Hosp Med 2018 13 853 9 10.12788/jhm.3090 30379144
20 Berkman ND Chang E Seibert J et al Management of high-need, high-cost patients: a “best fit” framework synthesis, realist review, and systematic review Agency for Healthcare Research and Quality (AHRQ) 2021
21 Delaney RK Sisco-Taylor B Fagerlin A et al A systematic review of intensive outpatient care programs for high-need, high-cost patients Transl Behav Med 2020 10 1187 99 10.1093/tbm/ibaa017 33044534
22 Hudon C Chouinard M-C Lambert M et al Key factors of case management interventions for frequent users of healthcare services: a thematic analysis review BMJ Open 2017 7 e017762 10.1136/bmjopen-2017-017762
23 Schumacher JR Lutz BJ Hall AG et al Impact of an emergency department-to-home transitional care intervention on health service use in medicare beneficiaries Med Care 2021 59 29 37 10.1097/MLR.0000000000001452 33298706
24 Australian Bureau of Statistics 3101.0 - australian demographic statistics Canberra:ABS 2017 Available http://www.abs.gov.au/ausstats/abs@.nsf/Latestproducts/3101.0Feature%20Article1Jun%202017?opendocument&tabname=Summary&prodno=3101.0&issue=Jun%202017&num=&view=
25 Tasmanian Medicare Local Primary health indicators tasmania report hobart: tasmanian medicare local limited 2015
26 Tasmanian Department of Health Healthcare Connect North Hobart, Tasmania Tasmanian Government 2023 Available https://www.health.tas.gov.au/health-topics/community-health/healthcare-connect-north
27 Tollyfield R Facilitating an accelerated experience-based co-design project Br J Nurs 2014 23 136 41 10.12968/bjon.2014.23.3.136 24526020
28 Gordon S Potter M Hamer P The nominal group technique: a useful consensus methodology in physiotherapy research NZ J Physiother 2004 32 126 30
29 Rankin NM McGregor D Butow PN et al Adapting the nominal group technique for priority setting of evidence-practice gaps in implementation science BMC Med Res Methodol 2016 16 110 10.1186/s12874-016-0210-7 27566679
30 Slattery P Saeri AK Bragge P Research co-design in health: a rapid overview of reviews Health Res Policy Syst 2020 18 17 10.1186/s12961-020-0528-9 32046728
31 World Health Organisation New Perspectives on Global Health Spending for Universal Health Coverage Geneva World health Organisation 2018
32 Australian Health Ministers National Health Reform Agreement (NHRA) Lomg-term Health Reforms Roadmap 2021
33 Bird M McGillion M Chambers EM et al A generative co-design framework for healthcare innovation: development and application of an end-user engagement framework Res Involv Engagem 2021 7 12 10.1186/s40900-021-00252-7 33648588
34 Dimopoulos-Bick TL O’Connor C Montgomery J et al “Anyone can co-design?”: a case study synthesis of six experience-based co-design (EBCD) projects for healthcare systems improvement in New South Wales, Australia Pat Exp J 2019 6 93 104 10.35680/2372-0247.1365
35 National Institute for Health and Care Research Involve patients UK: HIHR 2023 Available https://www.nihr.ac.uk/health-and-care-professionals/engagement-and-participation-in-research/involve-patients.htm
36 Medical Research Future Fund Australian medical research and innovation priorities 2022-2024 Department of Health and Aged Care Canberra Australian Government 2021
37 Oliver K Kothari A Mays N The dark side of coproduction: do the costs outweigh the benefits for health research? Health Res Policy Syst 2019 17 33 10.1186/s12961-019-0432-3 30922339
38 Commonwealth Government of Australia A Commissioning Overview in the PHN Context Department of health 2018
39 Bates S Wright M Harris-Roxas B Strengths and risks of the primary health network commissioning model Aust Health Rev 2022 46 586 94 10.1071/AH21356 35437143
40 Sexton S Schehrer S Involving Users in Commissioning Local Services York, UK Joseph Rowntree Foundation 2010
41 Gardner K Dickinson H Moon K Re-orienting health systems through a commissioning approach: finding solutions for improved consumer engagement Health Res Policy Sys 2019 17 1 6 10.1186/s12961-019-0471-9
42 Tembo D Morrow E Worswick L et al Is Co-production just a pipe dream for applied health research commissioning? an exploratory literature review Front Sociol 2019 4 50 10.3389/fsoc.2019.00050 33869373
