
==== Front
Open Res Eur
Open Res Eur
Open Research Europe
2732-5121
F1000 Research Limited London, UK

10.12688/openreseurope.17311.1
Essay
Articles
Evolution of informed consent in research: From the Hippocratic Oath to the tailored consent
[version 1; peer review: 2 approved, 1 approved with reservations]

Fons-Martinez Jaime Conceptualization Investigation Project Administration Writing – Original Draft Preparation https://orcid.org/0000-0002-8319-3658
a12
Murciano-Gamborino Carlos Validation Writing – Review & Editing https://orcid.org/0009-0003-7794-8748
1
Diez-Domingo Javier Conceptualization Funding Acquisition Supervision 123
1 Vaccine Research Area, Foundation for the Promotion of Health and Biomedical Research of Valencia Region, FISABIO, Valencia, Spain
2 CIBER de Epidemiología y Salud Pública, Instituto de Salud Carlos III, Madrid, Spain
3 Universidad Católica de Valencia San Vicente Mártir. Facultad de Medicina y Ciencias de la Salud, Valencia, Spain
a jaime.fons@fisabio.es
No competing interests were disclosed.

17 4 2024
2024
4 7221 3 2024
Copyright: © 2024 Fons-Martinez J et al.
2024
https://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Background

Informed consent (IC) is essential in defending the autonomy of potential participants in clinical research. Despite the advances in research ethics, particularly in IC, the different guidelines and codes have not been fully implemented. Several studies have presented consent deficiencies that have resulted in unethical practices or poor understanding of the IC.

Main body

This article reviews the evolution of IC, from its philosophical origins and initial use in the Ottoman Empire (16th century) to its use in clinical research today. It also presents the vision of the European project i-CONSENT (Grant Agreement number: 741856), whose main purpose is to improve the understanding of ICs in research and identifies the key components of a new paradigm to develop patient-centred ICs.

Conclusions

In many cases, the IC has served to protect the investigator or sponsor from complaints. Different ethical guidelines have sought to make the IC a more useful tool, with little success. Today’s IC is mainly a bureaucratic and legal process that fails to consider the patient’s point of view. In this context, the Guidelines for Tailoring the Informed Consent Process in Clinical Studies provide alternatives to the current IC process, focusing on the patient’s opinions and making them part of the process, thereby improving clinical research quality.

Informed consent
research ethics
The Nuremberg Code
Declaration of Helsinki
Belmont Report
i-CONSENT
eConsent
Horizon 2020 Framework Programme741856 This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No [741856](Improving the guidelines for Informed Consent, including vulnerable populations, under a gender perspective [I-CONSENT]). This paper only reflects the personal views of the stated authors. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
==== Body
pmcBackground

The informed consent (IC) process is an essential procedure that ensures the fundamental rights and freedoms of the research subject, allowing them to voluntarily decide whether to participate in a study, and to withdraw without consequences at any time.

The changes in recent decades have consolidated the requirements for IC in research, making it an essential document to comply with ethical and legal standards. This has led to a highly regulated process, albeit with several gaps, as the different guidelines and codes have had limited success in practice.

These documents are often prepared solely by one of the stakeholders (usually the sponsor) without including other perspectives or interests. As a result, consent documents are long and complex, and are seen more as a bureaucratic act (a legal requirement and a 'contract' between the sponsor, the researcher, and the participant) than as an informational document. These complex documents mean that, in many cases, potential participants decide on their participation in a research study based exclusively on the oral information provided by the researcher, without even reading the consent form. This violates ethical standards: first, because the written information provided is difficult to understand by the patient; and second, the oral information provided by the researcher is beyond the scrutiny of the Ethics Committees or Good Clinical Practice inspections, contradicting efforts to trace all information within clinical trials. Oral information provided for consent represents the only gap in this traceability 1 .

In the systematic review and meta-analysis conducted by Tam, Huy, Thoa et al. 2 , no significant advances were found in the understanding of any element of IC over recent decades.

Origin of informed consent

The idea of IC had its origin in Modernity, specifically in the political philosophy of the 16th century. It is built on the idea that human relationships cannot be based on a vertical relationship, in which one party has all the power and the other passively obeys. This idea, which eventually reached medicine four centuries later (20th century), represented a paradigm shift. It went from paternalistic Hippocratic medicine, where the doctor held all the knowledge (and made the decisions) and the patient was unable to decide for himself (and passively obeyed), to a relationship where the principle of autonomy governs; the patient makes the decisions after being informed by the doctor, making his status as a citizen prevail, with the right to be informed and make decisions about his health 3, 4 . In the case of medicine, this shift has been slow and, in many cases, has had to be driven through legislative impositions or judicial decisions, leading in some cases to defensive practices 3, 4 . Leclercq et al. 5 and Murray 6 compiled and analysed several relevant judgments regarding IC in the field of surgery, its procedures, or the information that it must include. Despite early use of the concept of IC in the judicial realm, the term “informed consent” was only introduced into the judicial lexicon in 1957, when an appellate judge in California issued his written opinion in the case of Salgo vs. Leland Stanford Junior University Board of Trustees 7 .

In other cases, members of the scientific and/or medical society itself have demanded changes. In 1902, the psychiatrist Albert Moll emphasized the need for IC after collecting 600 cases of unethical non-therapeutic research on humans 8 . The physiologist Walter Cannon, in 1916, proposed to the American Medical Association (AMA) that they mandate IC for research. However, the AMA rejected his proposal, arguing that they considered that misconduct was not a problem of the research itself, but of dishonest researchers, and believed that better research and clinical care would come through trust, and not through regulation 9 . Basing the consent on a relationship of trust precludes traceability of the whole process, especially the oral explanation of the investigator. Despite this paradigm shift and the fact that IC focuses on autonomy, its ethical foundation requires correct articulation of the four classic principles of modern bioethics: non-maleficence, justice, beneficence, and autonomy. As Simón indicates: “ what ‘must be done’, therefore, will be the result of the prudent weighing of the obligations derived from respect for autonomy and beneficence, in a framework pre-established by non-maleficence and justice” 3 .

As we have already mentioned they are two different fields where the IC is used in medicine: treatment and research. Even both of them are founded on the principle of autonomy, consent for treatment is mainly based on case law while consent for research is based on ethical codes, statutes and administrative regulations 10 .

First informed consent documents

The use of consent documents (in the form of contracts and with obvious differences with current consents) dates back to the 16th century, in the former Ottoman Empire, where contracts were signed before carrying out surgical interventions. In this regard, Sayligil and Ozden 11 present a private consent form registered in Sharia Court Records (Ser’iyye Sicilleri) found in Bursa, dated 1524; Selek 12 identifies a contract from 1539, signed in Gaziantep, Turkey (formerly known as Aintab); and Ajlouni 13 shows a 1677 contract signed in the Ottoman Tripoli (now Libyan territory). These documents identify the contractor (patient or his/her father), the doctor, the court before which the contract is presented, the patient's illness, the procedure (including in some cases postoperative treatment), fees, the absence of responsibility of the doctor in the event the patient is harmed or dies, and are signed by the patient (or his/her father) and the doctor in the presence of witnesses (identifying them).

In the Western world, the first mentions of IC date back to the 18th century, also in the field of surgery. The orthopaedists Baker and Stapleton, without informing or obtaining prior approval from the patient (Slater), refractured and used an experimental medical device to align and straighten his non-union fractured leg, which was being treated with bandages. In the judgment of the trial “ Slater vs Baker and Stapleton” (1767), the court ruled in favour of the patient, saying that a patient should be informed of the procedure, so that he/she takes courage and charge of the situation, and can face the operation 14 .

Altschuler 15 notes that one of the first descriptions of the concept of IC appears in Edgar Allen Poe's novel " The facts of the case of M. Valdemar" (1845), in which the protagonist asks Mr. Valdemar for permission to practice a hypnosis experiment on him before his death.

Suárez-Obando and Ordoñez 16 and Vollmann and Winau 8 identify the First and Second Prussian Directives on Research as the predecessors of today’s IC in research. The First Prussian Directives on Research (1891) regulated research with prisoners, and included, as a requirement, the consent of the prisoner to be treated with the experimental treatment. The Second Prussian Directive on Research (1900) indicated that in medical interventions that did not have the purpose of diagnosis, treatment or immunization, neither minors nor the mentally incompetent could participate. In addition, it indicated that the subject must give his/her “unambiguous consent” after a “proper explanation of the possible negative consequences" of the intervention. This directive also indicates that the medical doctor is the only one who can perform or authorize research interventions, and that the fulfilment of all requirements and other circumstances of the case had to be documented in the medical record 8 .

Suárez-Obando and Ordoñez also highlight the pioneering role of the Reed Commission in its investigations on yellow fever in Cuba (1900). It included a document similar to a contract that explained the risks of participating in the study, the lack of an effective treatment against the disease, the will of the subject to participate in the research, and the conditions of participation 16 .

Informed consent in research after the Second World War: From the Nuremberg Code to the present day

The Nuremberg Code (1947)

Even though the Nuremberg Code has never been officially adopted as law by any nation or as ethical guidelines by any major medical association, due to its profound influence on global human rights law and medical ethics, some authors identify it as the most important document in the history of medical research ethics 17 .

During what is commonly known as the Nuremberg Trials, the International Military Tribunal prosecuted the leading Nazi authorities, the Trial of the Major War Criminals being the best known. A second group of trials of minor war criminals, under Control Council law no. 10 in the Military Courts of Nuremberg of the United States, included the so-called “Doctors' Trial”. Twenty of the 23 defendants were physicians, accused of torture and murder in the conduct of medical experiments on concentration camp inmates. These physicians were charged with war crimes and crimes against humanity; they were accused of having violated the Hippocratic Oath and having behaved in a way incompatible with their profession. The Nuremberg Code (1947) grew out of this trial, being drafted by the International Tribunal of Nuremberg and advised by Leo Alexander and Andrew Ivy, the American doctors who participated in it. The participation of Alexander and Ivy was fundamental. Before and during the trial, these doctors provided the chief prosecutor, Telford Taylor, with different documents with ethical, legal and scientific principles and requirements for human research, which already included free and informed consent. The memorandum entitled “ Ethical and Non-Ethical Experimentation on Human Beings” and the document “ Principles of Ethics Concerning Experimentation with Human Beings,” adopted by the AMA House of Delegates in December 1946, were among these documents. The Nuremberg Code identified the 10 basic principles that medical experiments should meet to satisfy moral, ethical and legal concepts. These included the specification that researchers must always recruit competent research subjects who understand the nature of the research and who voluntarily consent to participate. For this reason, the subjects must have sufficient knowledge and understanding to be able to make a conscious decision, which derives from a prior explanation (principle 1) and the subject’s right to withdraw their participation from the research (principle 9) 8, 9, 17, 18 .

While some American researchers understood the need for these principles (although they sought to moderate the strict language of the Code), many others considered that the Nuremberg Code did not apply to them, as it had no legal authority and they considered it a response to the experimental work done by Nazi researchers 9 . As De Abajo points out, doctors never felt any connection to the Code, which was largely alien to the profession, drawn up "only" to judge criminals.

It should be noted that Ivy acknowledged during the Trial that there was no specific code for medical research prior to 1946, and that the principles adopted by the AMA were expressly formulated for the Doctors’ Trial 17 . On this point, Ghooi criticizes the Nuremberg Code’s lack of originality, and points out that six of its 10 principles (including the need for IC) derive directly from the Guidelines for Human Experimentation issued during the Weimar Republic by the German minister of the Interior in 1931, with no mention of them in the Code 18 .

Declaration of Helsinki (1964-last revision 2013)

In June 1964, the World Medical Association (WMA) adopted the Declaration of Helsinki, which has been amended nine times to date, the last time in October 2013.

The Declaration of Helsinki was based on the Nuremberg Code and mainly on the document “ Principles for those in Research and Experimentation” (WMA, 1954) 19 . It was developed “ as a statement of ethical principles for medical research involving human subjects, including research on identifiable human material and data. The Declaration is intended to be read as a whole and each of its constituent paragraphs should be applied with consideration of all other relevant paragraphs.” 20

This Declaration gives great importance to IC, a term that appears to have been employed for the first time in a medical research ethics document in the first revision of the Declaration, adopted in Tokyo (1975). The current version (2013) contains a section with eight articles, from 25 to 32, dedicated exclusively to IC. This section includes, among other aspects: 1) the need for IC and willingness to participate; 2) that potential participants must receive adequate information, including the right to refuse to participate in the study or to withdraw consent to participate at any time without reprisal; 3) the importance of ensuring that the potential subject has understood the information; 4) the need for consent by a legally authorized representative when the person is incapable of giving informed consent; 5) cases in which the individual should not be included in the research; 6) circumstances when consent does not need to be obtained; 7) the importance of avoiding undue influence; and 8) how to proceed with identifiable human material or data.

Another important contribution of the Declaration of Helsinki is the introduction of the need for an additional external guarantee, provided by the supervision of an independent committee, which is responsible for examining the study protocol, providing comments and suggestions to the investigator, and approving it, before the study begins. This figure was introduced in the second version of the Declaration, in Tokyo (1975).

Belmont Report (1979)

Despite the advances in research ethics achieved by the Nuremberg Code and the Declaration of Helsinki, some investigators and studies continued disregarding the basic ethical standards, such as IC. In 1966, Henry Beecher published the article " Ethics and Clinical Research", highlighting the lack of ethics of many research studies published in scientific journals, and emphasized the importance of obtaining IC, which he identifies as the most important component of ethics in research. He also advocated the non-publication of results obtained from unethically obtained data 21 .

In 1972, Jean Heller published an article in the Washington Star and the New York Times, in which she denounced a study sponsored by the US Public Health Service on the effect of untreated syphilis in poor rural African-American subjects. The study was entitled “USPHS Untreated Syphilis Study at Tuskegee”. The study that began in 1932, when the treatment was ineffective and quite toxic, lasted 40 years (although its initial expected duration was six months). Participation in the study provided participants with free medical examinations, free meals, and burial insurance, but not the benefit of providing IC. The participants were not informed about medical advances in the treatment of syphilis, despite the fact that the efficacy of penicillin treatments was demonstrated in 1945. They were kept without treatment with terrible consequences for their health 22 .

The impact of both articles led to legislative changes in the United States and the creation in 1974 of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. This Commission published the Belmont Report in 1979 23 . The Report summarizes the basic ethical principles identified by the Commission during the 4-day discussion that took place in February 1976 at the Smithsonian Institution's Belmont Conference Center, and during the Commission's monthly deliberations held over a period of nearly four years. The Report is a statement of basic ethical principles and guidelines on research involving human subjects 23 .

The Belmont Report marks a before and after for IC. It is divided into three parts:

Boundaries Between Practice and Research: distinguishes the practice of an accepted therapy from biomedical and behavioural research. While practice is aimed at enhancing the well-being of an individual patient, research seeks to test a hypothesis, draw conclusions, and obtain generalizable knowledge.

Basic Ethical Principles: defines the three basic ethical principles: Respect for Persons, Beneficence, and Justice. Respect for Persons, which is the one that interests us most when talking about IC, suggests that individuals should be treated as autonomous entities and that those with diminished autonomy should be protected, and indicates that, “in most cases of research involving human subjects, respect for persons demands that subjects enter into the research voluntarily and with adequate information”.

Applications: identifies the requirements involved in applying the general principles, which are IC, Assessment of Risks and Benefits, and Selection of Subjects.

According to the Report, IC is based on the triad Information, Comprehension and Voluntariness:

Information: several specific codes contain recommendations to ensure that the subject has sufficient information, but Information still has several issues in terms of the criteria to judge the amount and sort of information that should be provided. However, problems may arise, when giving information on a relevant issue is likely to impair the validity of the research.

Comprehension: a person's ability to understand is a function that depends on intelligence, rationality, maturity, and language. The information must be adapted to the capabilities of the subject. Moreover, the manner and context in which the information is presented is as important as the information itself.

Unlike the Declaration of Helsinki, which indicates that the participation of subjects who cannot give their consent must be authorized by their legal representative, the Belmont Report indicates that participation must be authorized by a third party. These parties should be those “most likely to understand the incompetent subject's situation and to act in that person's best interest”. Subjects with limited comprehension should be informed and their objections should be honored, except when the research provides them with a therapy otherwise unavailable.

Investigators are responsible for ensuring that the subject has understood the information. A verbal or written comprehension test may be useful in this respect.

Voluntariness: valid consent must be given voluntarily, free of coercion (threat of harm is presented intentionally to obtain compliance), undue influence (offering an excessive, unwarranted, inappropriate or improper reward or other overture for participating) or unjustified pressure (when people in a position of authority or with influence urge the subject to participate).

Although supervision of the principle of autonomy by independent committees is not required in the Report, it recognizes the important role of these committees in assessing beneficence and any potential risks and benefits associated with the research 1 .

ICH Guidelines for Good Clinical Practice (1996 – last revision 2017)

The International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH) was created in 1990 with the mission of achieving “greater harmonisation worldwide to ensure that safe, effective, and high quality medicines are developed and registered in the most resource-efficient manner” 24 . This harmonisation is achieved through the development of the ICH Guidelines.

Frías 25 indicates that these standards were driven by the desire to combine criteria and that they represent a " set of obligations of sponsors, monitors and researchers who participate in conducting clinical trials".

Guideline E6 of the ICH Guidelines refers to Good Clinical Practice (GCP). The first version was published in 1996, and was used from 1997 to 2017, when it was replaced by Revision 2 (R2). A new revision of the guideline (R3) began at the end of 2019 - early 2020.

Since its first version, the Guidelines have dedicated section 4.8 to “Informed Consent of Trial Subjects”. This section includes relevant information, such as the obligation to have a favourable opinion by the Institutional Review Board or Independent Ethics Committee; aspects of autonomy and voluntariness (this includes aspects such as the need for consent, the importance of informing, the non-acceptance of undue coercion or influence, or the importance of using a language understandable to the potential participant or their legally accepteable representative .); aspects related to IC when the research includes minors, people unable to provide consent or in emergency situations; minimum information that must be given to the subject or the need to use simple and practical language, among others. These guidelines also give importance to oral information in the relationship between the research team and the subject, and it is considered that both oral and written information must be understandable.

CIOMS Guidelines (1982- last version 2016)

The Council for International Organizations of Medical Sciences (CIOMS) was founded in 1949 under the auspices of the World Health Organization (WHO). In 1982, CIOMS and the United Nations Educational, Scientific and Cultural Organization (UNESCO), in association with the WHO, drew up the “ Proposed International Ethical Guidelines for Biomedical Research Involving Human Subjects”, with the aim of providing “ internationally vetted ethical principles and detailed commentary on how universal ethical principles should be applied, with particular attention to conducting research in low-resource settings”.

The fourth version of the guidelines (2016) 26 establishes 25 guidelines and four appendices, with an introduction and a brief description of previous instruments and guidelines. These guidelines are a very valuable text in research ethics, especially with regard to IC, since several of them address related aspects that are of great interest. For example, guideline 7, “ Community engagement”, incorporates the idea of including potential participants and communities in the design of the IC process. In addition, it dedicates Guideline 9, “ Individuals capable of giving informed consent”, exclusively to IC and Guideline 10 to the “ Modifications and waivers of informed consent”. The IC dilemma in biobanks and in future research is addressed in Guidelines 11, “ Collection, storage and use of biological materials and related data”, and 12, “ Collection, storage and use of data in health-related research”. Guidelines 15 to 21 include aspects related to consent in specific groups and situations, such as research with vulnerable persons and groups, adults incapable of giving IC, children and adolescents, or women (with particular attention to research during pregnancy and breastfeeding). Research in disaster and disease outbreaks are also included in these guideline situations, as are cluster randomized trials. CIOMS guidelines should be taken into account when discussing research ethics, and especially IC.

What are the main challenges of informed consent in research nowadays?

Despite major advances in research ethics and the increasing regulation of informed consent, there are a number of challenges that have persisted over the past 50 years and are difficult to overcome 27 . These include:

Length and difficulty of comprehension

In an attempt to increase transparency and ensure that the potential participant has the necessary information to make an informed decision on whether or not to participate in research, informed consent has become increasingly regulated and standardised, resulting in lengthy documents 28 . This increase in the content of informed consent forms does not guarantee that they contain all important information for potential participants or that they properly understand it, in fact, too much information can impair understanding 29 .

In addition, even the use of plain language has been widely recommended, most consent materials still include complex language 30 . This is an important issue, as health information materials (including informed consent) are often too complex for a significant proportion of the population who lack the necessary skills and health literacy levels to adequately understand them 31 .

Focus

Often, both the sponsor and the researchers have considered informed consent primarily as a bureaucratic act and a legal requirement, centred on the document and the act of signing.

The informed consent documents have generally consisted of lengthy texts that are difficult to understand, written in complex and technical language 2, 32 . Moreover, in most cases, they are written by the sponsor, without taking into account the perspectives of other stakeholders, and are sometimes constructed more as defensive documents designed to serve the interests of institutions and sponsors than for informational purposes 27 . Documents should be understood as the basis of a meaningful exchange between researcher and subject rather than as an end 33 .

Rather than focusing on the signature, informed consent should be understood as a process rather than a punctual act 34 . Indeed, the emphasis on the signature and the idea that it entails the participant's adequate understanding of the information is increasingly being questioned 35 .

Respecting diversity of values and backgrounds

The inclusion of people with different characteristics and backgrounds in research is extremely important, to the extent that some authors consider that improving diversity in clinical trials is a moral and scientific imperative 36 . Informed consent forms often adopt standardised approaches that do not take into account the characteristics and are not tailored to the needs, preferences and interests of potential participants. Nevertheless, the characteristics of the population must be taken into account when designing informed consents, for example, whether the person is illiterate or the culture to which he or she belongs. It is important to bear in mind that norms vary across cultures 37 and these may affect informed consent and concepts such as autonomy, voluntariness, altruism or even illness.

Technological advances

Research methodologies and ways of communicating have changed over the years. These advances have also brought new challenges and opportunities for informed consent, enabling new approaches to both how informed consent is presented and how it is obtained. Grady et al. 27 identify several challenges and areas of research in the use of electronic and digital informed consent, in terms of disclosure, comprehension, voluntariness and authorisation.

The systematic review done by Gesualdo et al. on the use of digital tools in the informed consent process 38 highlights their potential positive impact on improving participants' understanding and satisfaction, and indicates that at least it seems clear that they do not affect them negatively. The authors also highlight the lack of standardised methods for assessing the impact of digitally supported informed consent processes.

The use of social media to recruit participants for clinical trials is increasingly common, but it also brings with it ethical challenges for which there is no clear, agreed-upon answer. Social media recruitment requires legal and ethical standards to be applied in a context that may be unfamiliar to researchers and ethics committees 39 .

Caplan and Friesen 40 explain the potential benefits of using social media in recruiting participants for clinical trials. They highlight the control it allows and its advantages to reach more diverse populations, by being able to deliver different messages to different audiences or to select the profile of the person who will receive them, based on age, location, language, education, marital status or occupation.

Investigator’s communication skills

Several authors have highlighted the importance of communication between the physician and the patient/participant in creating a favourable and trusting environment for the discussion of different health issues. This contributes to patient satisfaction. In many cases physicians overestimate their communication skills 41 .

During the informed consent process, the conversation between the investigator and the potential participant is key. In this regards, Hayman, Taylor, Peart et al. 42 identified researcher’s verbal explanation as the most useful information for the parents who were invited to enrol their infant in a research project, over written information; Nishimura, Carey, Erwin et al. 43 included the extended conversation between researcher and participant as one of the elements to improve comprehension of the IC; Stevens and Pletsch 30 indicate that participants trust health professionals more than what is written in the IC and give much weight in their decision to participate in the study to the assessment of the health professional recommending the study, rather than weighing the information received itself; and Tam et al. 2 identify the explanatory ability of researchers as a factor influencing participant understanding.

Biobanks

There are different types of informed consent applicable to biobanks: specific/close consent; broad consent; blanket/open consent; dynamic consent. The use of one type of consent or another has different implications for the donor's autonomy and control over the use of the samples he/she has donated in different research studies. Sometimes these differences are not clear to the donor or even to some researchers who include the collection of biological samples in their studies.

Pandemics

The COVID-19 situation we have been experiencing since the first quarter of 2020 has highlighted the importance of having guidelines on how to act in health pandemic situations from all fields, including medical research and informed consent.

The global COVID-19 pandemic created a new scenario in medical research, both in terms of how to combine efforts and share information internationally and how to deal with legal and ethical dilemmas, including those related to informed consent. In Europe, the European Medicines Agency (EMA), as well as national medicines agencies, responded to this new challenge by issuing guidances on the management of clinical trials during the pandemic. These guidances included proposals to make clinical trials more flexible during this period so that they could continue. Sixteen agencies (15 national agencies and the EMA) have included guidance on obtaining the IC, in order to prevent the spread of the virus and to comply with restrictions imposed by governments during the pandemic. These guidances included alternative forms of consent or even exceptions to consent to avoid overcrowding of health care facilities and the need for participants to travel, and proposed alternatives to the traditional paper-based IC forms 44 .

i-CONSENT (2017–2021)

The European Commission echoed most of these difficulties and challenges, and in October 2015 published a call for proposals (reference number SwafS-17-2016) under the title “The Ethics of informed consent in novel treatment including a gender perspective”, which was resolved in favour of the project entitled “Improving the guidelines of informed consent, including vulnerable populations, under a gender perspective (i-CONSENT)” (Grant Agreement 741856).

This four years project (2017–2021) aims to improve the informed consent processes in clinical research, making them easier to understand and tailored to the potential participant’s needs and preferences. The final product of the project are a series of guidelines to improve Informed Consent process, including vulnerable populations, under a gender perspective and relying on ICT tools 45 .

These guidelines, entitled “Guidelines for Tailoring the Informed Consent Process in Clinical Studies”, have been written in accordance with other existing guidelines and legal documents, and should be read in conjunction with them 46 . They aim to “ help researchers utilise bidirectional and continuous communication during the process of informed consent, without losing sight of vulnerable populations, multiculturalism and gender perspectives” 1 .

i-CONSENT defines the IC process as a “bidirectional communication process that begins with the first contact with the potential participant and continues throughout the study until its end. During this process, continuous feedback and communication between the participant and the research team are essential” 46 .

Furthermore, the Guidelines for Tailoring the Informed Consent Process in Clinical Studies aim to help improve IC in the future, for which they propose a paradigm shift in which the research participant is central to the IC process. i-CONSENT envisions an IC that:

Is considered a process that adds value, rather than a bureaucratic procedure or legal requirement.

Ceases to be a defensive document and focuses on the participant.

Does not focus on the signature and is seen as a process that extends from the first contact with the potential participant to the end of his/her participation in the study.

Takes into account the point of view of the most involved stakeholders, such as potential participants and investigators, and not just the sponsor.

Incorporates potential participants in the process of preparing informative materials from the beginning (design, co-creation, revision) through participatory methodologies, such as design thinking or other qualitative or quantitative methodologies to determine their opinion, such as surveys, interviews or focus groups.

Improve participants' health literacy throughout the consent process.

Is inclusive and tailored to the participant.

The methodology employed to develop the guidelines used a multidisciplinary approach. Several reviews of legal, ethical and scientific documents have been conducted on different aspects of IC. Qualitative techniques (such as focus groups, nominal groups, and design thinking) have been used with potential participants, representatives of patient associations, investigators, representatives of the pharmaceutical industry, ethicists, members of ethics committees and regulators to obtain information and explore their points of view, needs and interests. Social media analysis has also been carried out.

Following the recommendations included in the guidelines, IC materials were prepared for three study cases (mock scenarios/studies) 47 . These materials were tested in three different populations and countries with very positive results. Additionally, a panel of experts assessed the appropriateness of some of the recommendations included in the guidelines 48 .

The recommendations included in the guidelines have been already used in a real clinical trial: VIGIRA 49 .

The pillars of informed consent adapted to the participant

It is crucial that IC enables a person to:

Make an informed and autonomous decision about their participation in a study.

Re-evaluate their participation throughout the study, and understand their freedom to withdraw at any time.

The acronym i-CONSENT contains the core elements of a comprehensive consent process ( Table 1).

Table 1. The i-CONSENT acronym and core elements of the consent process.

I	Information	Complete and clear information is essential for the potential participant to be able to make an
autonomous decision.	
C	Co-creation	The inclusion of potential participants during the design and review of study information materials is
key to ensure that they are understandable and address the target population’s needs and preferences.	
O	Ongoing process	Consent should be a two-way continuous communication process that begins at first contact with the
potential participant, and continues until the end of the study.	
N	New technologies, methods, and processes	The consent process should include technical and methodological innovations to facilitate the
participant’s experience. Their appropriateness from a social, methodological, legal and ethical point of
view should always be taken into consideration.	
S	Self-determination (autonomy)	Autonomy is a fundamental principle. The purpose of the informed consent is to ensure that the
potential participant is able to make an autonomous and free decision.	
E	Empowerment	Participants should be empowered to make their own decisions.	
N	Nonstandard (inclusive and tailored)	Research should be inclusive to meet the needs of the potential participants and respect the basic
bioethics principle of justice. There is no single best way to conduct the consent process. The ‘ideal’
solution will differ in every setting and therefore needs careful design. Where possible, the consent
process should be tailored to the needs of the target population.	
T	Trusted	Good practices are essential to build trust between investigators and potential participants, and to
increase society’s trust in research.	
Source: Guidelines for Tailoring the Informed Consent Process in Clinical Studies 46 .

The Guidelines for Tailoring the Informed Consent Process in Clinical Studies recommend that the IC process incorporate key interventions designed to enhance autonomy and inclusion. As noted in the expert assessment of the recommendations included in the guidelines, the four keys to improving understanding of IC in clinical trials are: “(1) consider consent a two-way continuous interaction that begins at the first contact with the potential participant and continues until the end of the study; (2) improve investigators’ communication skills; (3) co-create the information; and (4) use a layered approach, including information to compensate for the potential participant’s possible lack of health literacy and a glossary of terms” 48 .

These key recommendations, together with those contained in the rest of the guidelines, have the potential to result in higher quality, lower cost and ethically justified research. The potential impact of better information and communication is high at the ethical level (safeguarding people's autonomy) and in research in general.

Conclusions

The principle of autonomy has reached the field of medicine later than other disciplines, due to the strong Hippocratic tradition and because of the sensitivity of health (individual and collective) itself. From the 16th century to the present day, consent in many cases has had a defensive purpose (to safeguard the doctor or sponsor from possible complaints) rather than an informative one, becoming a perversion of the consent itself. Legislation and court rulings have been placing emphasis on the need for consent, and the contents that it must include have been expanded.

The different ethical guidelines that have emerged have sought to make consent a more useful tool and closer to its objective (to defend the autonomy of the participant), but their non-mandatory nature has meant they have less practical impact.

The problems presented by IC today — such as to be focused on the signature; considered mainly a bureaucratic act and a legal requirement by the sponsor and research staff; their excessive length; use of very complicated language and medical jargon; being drawn up by only one of the interested parties, usually the sponsor, without taking other points of view into account; or that a large part of the potential participants deciding whether or not to participate based on the discussion with the researcher, without even reading the informational materials — drastically diminish their success as a guarantor of the participant's autonomy.

Despite the availability of several guidelines, sponsors and researchers do not use them properly. Thus, involving all stakeholders, including regulators, is key to moving from theorizing to practice.

The Guidelines for Tailoring the Informed Consent Process in Clinical Studies, prepared within the framework of the i-CONSENT project, promote a change in the way of conceptualizing IC. Thus, it becomes a process that provides added value and prevents it from becoming a bureaucratic act focused on the participant’s signature on the IC form, having as a central concept the implementation of a participant-centred design and the improvement of effective communication.

The guidelines provide high level recommendations to improve the IC process from the drafting of consent materials until the end of the study. They also include a step-by-step description of the IC process and a checklist to implement comprehensive and inclusive IC, as well as 14 fact sheets and six tools with recommendations and examples to put ethical considerations into practice.

Ethics and consent

Ethical approval and consent were not required.

Acknowledgements

The authors would like to thank the partners of the i-CONSENT project for the work carried out during the duration of this project.

Data availability

No data are associated with this article.

10.21956/openreseurope.18711.r40698
Reviewer response for version 1
Bellver Capella Vicente 1Referee https://orcid.org/0000-0002-8776-397X

Romero-Wenz Lukas 2Co-referee
1 Universidad de Valencia, València, Spain
2 Legal and Political Philosophy, Universitat de Valencia, Valencia, Valencian Community, Spain
13 9 2024 Copyright: © 2024 Bellver Capella V and Romero-Wenz L
2024
https://creativecommons.org/licenses/by/4.0/ This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Version 1recommendationapprove
The document is directly related to a research project funded by the European Union that examined the challenges facing informed consent today and the proposals to ensure that informed consent documents are truly a guarantee of the fundamental rights of the subject of the investigation and not a bureaucratic procedure with a primarily defensive objective.

The paper consists of three parts. The first presents a summary of the milestones in the history of informed consent, with some especially relevant information little known to the general public. The second refers to the main challenges that informed consent presents today. And, finally, and echoing the results of the i-Consent project, the principles that should inform the guidelines that are approved for the preparation of informed consent documents are presented.

Is the argument persuasive and supported by appropriate evidence?

Yes

Is the work clearly and cogently presented?

Yes

Is the topic of the essay discussed accurately in the context of the current literature?

Yes

Does the essay contribute to the cultural, historical, social understanding of the field?

Yes

Reviewer Expertise:

Bioethics and Biolaw

We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

10.21956/openreseurope.18711.r40697
Reviewer response for version 1
Wade Julia 1Referee https://orcid.org/0000-0001-6486-6477

1 Bristol Medical School (PHS), University of Bristol, Bristol, England, UK
14 8 2024 Copyright: © 2024 Wade J
2024
https://creativecommons.org/licenses/by/4.0/ This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Version 1recommendationapprove-with-reservations
Is the topic of the essay discussed accurately in the context of the current literature?

The authors refer to past and current literature in a way that is helpful to the reader in pulling relevant literature together.

One useful paper that currently isn't referenced is Dixon Woods et al (2007). This paper argues that it is not possible to 'fix' information provision such that parallel discussion alongside written information becomes imperative.

Great to see mention of layered consent; other work in this area could be referenced, e.g. Symons et al (2022). Is the work clearly and cogently presented?

The authors present a valid argument, namely that research informed consent processes meet the requirements of ethics committees but at the expense of providing information that is relevant and easily understood by patients. Is the argument persuasive and supported by appropriate evidence?

Yes, the argument is valid and is well supported by the literature referenced.

It would be very helpful if the authors could clearly distinguish between the principle of informed consent vs the process of ensuring consent is fully informed vs different ways to operationalise the process of ensuring informed consent is given to take part in research.

The principle is very much that consent should be voluntary and based on understanding what the research involves. History has shown us that we need guidelines to ensure this principle is followed where possible and reasonable alternatives are in place where not, and this is clearly and helpfully documented here. Where these guidelines have not been in place, researchers/healthcare professionals may abuse the power they have over patients/potential participants.

How that is currently operationalised can be critiqued.  How do we best inform people and how do we best ensure that they understand to what they are consenting? There are 3 key principles underpinning informed consent 1) voluntariness; 2) understanding; 3) capacity. Voluntariness is probably of higher priority than the principle of understanding; writing as someone who has repeatedly consented to take part in research without reading all the participant facing information. Ideally we want people to be fully informed, but as a participant, I can waive my right to read the entire information sheet.

Three sets of stakeholders have subtly different agendas as their top priority: potential participants have their own current interests as priority, but also frequently altruism to help others in future; Ethics committees set out to protect patients from harm during the research process; Researchers set out to recruit to their research in order to benefit patients in future and with this in mind, try to be transparent and clear in what the research involves.  Inevitable tensions arise from these different priorities: insistence by ethics committees on communicating potential risk of harm results in the current very long participant information sheets which function as a barrier to patient understanding and giving truly informed consent.

In the abstract and introduction, it would be helpful if the authors could distinguish between the principle of informed consent, the range of processes by which to ensure informed consent is given and be clear to distinguish these from the participant information sheet/consent form (currently referred to as 'the document', second paragraph of p3).  Currently both the abstract and text refer to 'poor understanding of the IC' and I believe this refers to the informed consent form or participant information sheet (a 'document') rather than the whole process of ensuring information provision, understanding and voluntariness.

In fact it is the IC process that should be made more patient-centred, not just the IC form and it was the IC process that was the focus of i-CONSENT. So helpful to be very clear in specifying whether a reference is being made to the process vs the IC form throughout to avoid confusion. Does the essay contribute to the cultural, historical, social understanding of the field?

The essay provides helpful historical perspective to account for processes currently adopted when seeking to ensure fully informed consent is given for participation in research.

found it helpful to have this historical context as it accounts for many of the processes in place today, some of which, as the authors argue, actually function as a barrier to patient understanding of what they consenting.

Thank you for the opportunity to review this essay, I found it informative and extremely helpful in mapping the historical and current context.

Is the argument persuasive and supported by appropriate evidence?

Partly

Is the work clearly and cogently presented?

Partly

Is the topic of the essay discussed accurately in the context of the current literature?

Partly

Does the essay contribute to the cultural, historical, social understanding of the field?

Yes

Reviewer Expertise:

Optimising informed consent within Trials research.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

10.21956/openreseurope.18711.r40693
Reviewer response for version 1
Lanzerath Dirk 1Referee
1 University of Bonn, Bonn, Germany
25 5 2024 Copyright: © 2024 Lanzerath D
2024
https://creativecommons.org/licenses/by/4.0/ This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Version 1recommendationapprove
The article provides an excellent overview of the development of informed consent. It is particularly interesting that it is not only based on European developments and documents.  The authors take a critical look at the now excessive flood of information in informed consent and the question of whether the purely administrative process of obtaining a signature really fulfils the actual aim of freely and informed consent to participate in a research project. Digitalisation of processes and harmonisation of procedures are also addressed

I would like to mention the following suggestions for revision or additions:

- Individual consent is not common in all countries, but the group/tribe, partner or other persons play an important role. This should be noted and also changes questions about e-consent, for example.

- The title is somewhat misleading because it is not about research but exclusively about medical research. Therefore, "medical" for research should be added to the title.  In psychology and the social sciences, for example, there are other problems and lines of development than in medicine.

Is the argument persuasive and supported by appropriate evidence?

Yes

Is the work clearly and cogently presented?

Yes

Is the topic of the essay discussed accurately in the context of the current literature?

Yes

Does the essay contribute to the cultural, historical, social understanding of the field?

Yes

Reviewer Expertise:

research ethics, bioethics

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Competing interests: No competing interests were disclosed.

Competing interests: I took part in the I-CONSENT consultation process (consultation with key stakeholders about the proposed I-CONSENT framework). I have no other conflicts of interest.

Competing interests: No competing interests were disclosed.
==== Refs
1 Fons-Martínez J Ferrer-Albero C Russell R : i-CONSENT: Presentation of the Project and the Importance of Participants’ Perspectives in the Informed Consent Process. BioLaw Journal. 2019; (Special Issue ):3–10. 10.15168/2284-4503-396
2 Tam NT Huy NT Thoa LTB : Participants' understanding of informed consent in clinical trials over three decades: systematic review and meta-analysis. Bull World Health Organ. 2015;93 (3 ):186–98H. 10.2471/BLT.14.141390 25883410
3 Simón P : Diez mitos en torno al consentimiento informado. An Sist Sanit Navar. 2006;29 (2 ):29–40. 10.4321/S1137-66272006000600004 17308537
4 De Siqueira JE : Los orígenes del consentimiento informado en clínica. Revista de Bioética Latinoamericana. 2009;3 (1 ).
5 Leclercq WKG Keulers BJ Scheltinga MRM : A review of surgical informed consent: past, present, and future. A quest to help patients make better decisions. World J Surg. 2010;34 (7 ):1406–15. 10.1007/s00268-010-0542-0 20372902
6 Murray PM : The History of Informed Consent. Iowa Orthop J. 1990;10 :104–9.
7 Mazur DJ : Influence of the law on risk and informed consent. BMJ. 2003;327 (7417 ):731–4. 10.1136/bmj.327.7417.731 14512484
8 Vollmann J Winau R : Informed consent in human experimentation before the Nuremberg code. BMJ. 1996;313 (7070 ):1445–9. 10.1136/bmj.313.7070.1445 8973233
9 Jones DS Grady C Lederer SE : "Ethics and Clinical Research"--The 50th Anniversary of Beecher's Bombshell. N Engl J Med. 2016;374 (24 ):2393–8. 10.1056/NEJMms1603756 27305197
10 del Carmen MG Joffe S : Informed consent for medical treatment and research: a review. Oncologist. 2005;10 (8 ):636–41. 10.1634/theoncologist.10-8-636 16177288
11 Sayligil O Ozden H : A written consent form dating back to 1524 in Bursa Ser'iye (Sharia Court) records and a proposal of a new start date for consent forms. Ann Saudi Med. 2014;34 (5 ):433–6. 10.5144/0256-4947.2014.433 25827701
12 Selek S : A written consent five centuries ago. J Med Ethics. 2010;36 (10 ):639. 10.1136/jme.2010.037713 20817814
13 Ajlouni KM : History of informed medical consent. Lancet. 1995;346 (8980 ):980. 10.1016/S0140-6736(95)91608-3 7564776
14 Ortiz-Martinez JG : Consentimiento informado en cirugía ortopédica. Pers bioét. 2009;13 (1 ):59–84. Reference Source
15 Altschuler EL : Informed consent in an Edgar Allen Poe tale. Lancet. 2003;362 (9394 ):1504. 10.1016/S0140-6736(03)14710-1 14602460
16 Suárez-Obando F Ordoñez A : Ética de la Investigación científica: la fiebre amarilla, la Comisión Reed y el origen del consentimiento informado. Infectio. 2010;14 (3 ):206–16. 10.1016/S0123-9392(10)70112-0
17 Shuster E : Fifty years later: the significance of the Nuremberg Code. N Engl J Med. 1997;337 (20 ):1436–40. 10.1056/NEJM199711133372006 9358142
18 Ghooi RB : The Nuremberg Code-A critique. Perspect Clin Res. 2011;2 (2 ):72–6. 10.4103/2229-3485.80371 21731859
19 De Abajo FJ : La Declaración de Helsinki VI: Una revisión necesaria, pero ¿suficiente? Rev Esp Salud Pública. 2001;75 :407–20. Reference Source 11764559
20 World Medical Association: WMA Declaration of Helsinki - Ethical principles for medical research involving human subjects.64th WMA General Assembly, Fortaleza, Brazil, October 2013,2018; Accessed 30 Nov 2021. Reference Source
21 Beecher HK : Ethics and clinical research. N Engl J Med. 1966;274 (24 ):1354–60. 10.1056/NEJM196606162742405 5327352
22 White MG : Why Human Subjects Research Protection Is Important. Ochsner J. 2020;20 (1 ):16–33. 10.31486/toj.20.5012 32284679
23 The National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. The Belmont Report.Ethical Principles and Guidelines for the Protection of Human Subjects of Research. Belmont, 1979,2018; Accessed 30 Nov 2021. Reference Source
24 Mission: International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use.Accessed 30 Nov 2021. Reference Source
25 Frías Iniesta J : Buenas Prácticas Clínicas (BPC) y Normas ICH (Conferencia Internacional de Armonización).García AG, editor: El Ensayo Clínico en España. España: Farmaindustria,2001;265–70.
26 Council for International Organizations of Medical Sciences (CIOMS): International Ethical Guidelines for Health-related Research Involving Humans.Switzerland: CIOMS;2016. Reference Source
27 Grady C Cummings SR Rowbotham MC : Informed Consent. N Engl J Med. 2017;376 (9 ):856–867. 10.1056/NEJMra1603773 28249147
28 AND Consulting Group: Ethical, privacy, and data protection issues associated with the use of ICT for obtaining informed consent. Scene-Setting Report. 2020. 10.5281/zenodo.4729580
29 Beardsley E Jefford M Mileshkin L : Longer consent forms for clinical trials compromise patient understanding: so why are they lengthening? J Clin Oncol. 2007;25 (9 ):e13–4. 10.1200/JCO.2006.10.3341 17369564
30 Stevens PE Pletsch PK : Informed consent and the history of inclusion of women in clinical research. Health Care Women Int. 2002;23 (8 ):809–19. 10.1080/07399330290112335 12487696
31 Rowlands G Protheroe J Winkley J : A mismatch between population health literacy and the complexity of health information: an observational study. Br J Gen Pract. 2015;65 (635 ):e379–86. 10.3399/bjgp15X685285 26009533
32 Falagas ME Korbila IP Giannopoulou KP : Informed consent: how much and what do patients understand? Am J Surg. 2009;198 (3 ):420–35. 10.1016/j.amjsurg.2009.02.010 19716887
33 Office of the Commissioner, Office of Clinical Policy and Programs, Office of Clinical Policy : A Guide to Informed Consent Guidance for Institutional Review Boards and Clinical Investigators. USA: FDA,1998. Reference Source
34 Bento SF Hardy E Osis MJD : Process for obtaining informed consent: women's opinions. Dev World Bioeth. 2008;8 (3 ):197–206. 10.1111/j.1471-8847.2006.00183.x 19046257
35 Shah P Thornton I Turrin D : Informed Consent.[Updated 2021 Jun 14]. In: StatPearls.Treasure Island (FL): StatPearls Publishing;2021.
36 Gray DM 2nd Nolan TS Gregory J : Diversity in clinical trials: an opportunity and imperative for community engagement. Lancet Gastroenterol Hepatol. 2021;6 (8 ):605–7. 10.1016/S2468-1253(21)00228-4 34246352
37 Ruiz-Casares M : Mental health: Tailor informed-consent processes. Nature. 2014;513 (7518 ):304. 10.1038/513304a 25230633
38 Gesualdo F Daverio M Palazzani L : Digital tools in the informed consent process: a systematic review. BMC Med Ethics. 2021;22 (1 ): 18. 10.1186/s12910-021-00585-8 33639926
39 Gelinas L Pierce R Winkler S : Using Social Media as a Research Recruitment Tool: Ethical Issues and Recommendations. Am J Bioeth. 2017;17 (3 ):3–14. 10.1080/15265161.2016.1276644 28207365
40 Caplan A Friesen P : Health disparities and clinical trial recruitment: Is there a duty to tweet? PLoS Biol. 2017;15 (3 ): e2002040. 10.1371/journal.pbio.2002040 28249024
41 Ha JF Longnecker N : Doctor-Patient Communication: A Review. Ochsner J. 2010;10 (1 ):38–43. 21603354
42 Hayman RM Taylor BJ Peart NS : Participation in research: Informed consent, motivation and influence. J Paediatr Child Health. 2001;37 (1 ):51–4. 10.1046/j.1440-1754.2001.00612.x 11168870
43 Nishimura A Carey J Erwin PJ : Improving understanding in the research informed consent process: a systematic review of 54 interventions tested in randomized control trials. BMC Med Ethics. 2013;14 : 28. 10.1186/1472-6939-14-28 23879694
44 de Jong AJ Santa-Ana-Tellez Y van Thiel GJMW : COVID-19 and the Emerging Regulatory Guidance for Ongoing Clinical Trials in the European Union. Clin Pharmacol Ther. 2021;109 (6 ):1517–1527. 10.1002/cpt.2225 33666223
45 Improving the guidelines for Informed Consent: i-CONSENT, 2021; Accessed 30 Nov 2021. Reference Source
46 i-CONSENT Consortium: Guidelines for Tailoring the Informed Consent Process in Clinical Studies.Spain: FISABIO, Generalitat Valenciana;2021. 10.5281/zenodo.4563938
47 i-CONSENT Pilot Validation. 2021; Accessed 30 Nov 2021.
48 Fons-Martinez J Ferrer-Albero C Diez-Domingo J : Assessment of the appropriateness of the i-CONSENT guidelines recommendations for improving understanding of the informed consent process in clinical studies. BMC Med Ethics. 2021;22 (1 ): 138. 10.1186/s12910-021-00708-1 34645425
49 Ethics Committee gives green light to using an i-CONSENT informed consent. 2020; Accessed 30 Nov 2021. Reference Source
