
==== Front
J Antimicrob Chemother
J Antimicrob Chemother
jac
Journal of Antimicrobial Chemotherapy
0305-7453
1460-2091
Oxford University Press UK

39011836
10.1093/jac/dkae226
dkae226
Original Research
AcademicSubjects/MED00740
AcademicSubjects/MED00290
AcademicSubjects/MED00230
Cultivating global antimicrobial stewardship: linguistic and cultural validation of the Australian National Antimicrobial Prescribing Survey appropriateness assessment definitions for Portugal
https://orcid.org/0000-0002-9375-4623
Palos C NOVA National School of Public Health, Public Health Research Centre; Comprehensive Health Research Center, CHRC, REAL, CCAL, NOVA University Lisbon, Lisbon, Portugal
Grupo Luz Saúde, Lisbon, Portugal

Ierano C National Centre for Antimicrobial Stewardship, Peter Doherty Institute for Infection and Immunity, University of Melbourne, Melbourne, Australia
Royal Melbourne Hospital Guidance Group, Melbourne Health, Melbourne, Australia

Santos M J D ISCTE-Instituto Universitário de Lisboa, Centro de Investigação e Estudos de Sociologia, Lisbon, Portugal

Leitão I Unidade Local PPCIRA, Hospital Beatriz Ângelo, Clínica Universitária de Doenças Infeciosas e Parasitárias, Faculdade de Medicina, Universidade de Lisboa, Lisbon, Portugal

Nogueira P NOVA National School of Public Health, Public Health Research Centre; Comprehensive Health Research Center, CHRC, REAL, CCAL, NOVA University Lisbon, Lisbon, Portugal

https://orcid.org/0000-0003-1425-029X
James R National Centre for Antimicrobial Stewardship, Peter Doherty Institute for Infection and Immunity, University of Melbourne, Melbourne, Australia
Royal Melbourne Hospital Guidance Group, Melbourne Health, Melbourne, Australia

Paiva J A Intensive Care Department, Centro Hospitalar Universitário de São João, Medicine Department, Faculty of Medicine, University of Porto, Porto, Portugal

Thursky K National Centre for Antimicrobial Stewardship, Peter Doherty Institute for Infection and Immunity, University of Melbourne, Melbourne, Australia
Royal Melbourne Hospital Guidance Group, Melbourne Health, Melbourne, Australia

Sousa P NOVA National School of Public Health, Public Health Research Centre; Comprehensive Health Research Center, CHRC, REAL, CCAL, NOVA University Lisbon, Lisbon, Portugal

Corresponding author. E-mail: ca.palos@ensp.unl.pt
9 2024
16 7 2024
16 7 2024
79 9 22812291
23 11 2023
14 6 2024
© The Author(s) 2024. Published by Oxford University Press on behalf of British Society for Antimicrobial Chemotherapy.
2024
https://creativecommons.org/licenses/by-nc/4.0/ This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial License (https://creativecommons.org/licenses/by-nc/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact reprints@oup.com for reprints and translation rights for reprints. All other permissions can be obtained through our RightsLink service via the Permissions link on the article page on our site—for further information please contact journals.permissions@oup.com.

Abstract

Background

Antimicrobial resistance is driven by inappropriate antimicrobial prescribing. The National Antimicrobial Prescribing Survey (NAPS) is an Australian-developed auditing platform to assist in the assessments of antimicrobial quality by antimicrobial stewardship programmes using consensus-based definitions. The NAPS has demonstrated to be transferable to other countries. Its adaptation to Portugal could improve knowledge about the quality of antimicrobial prescribing in the country.

Objectives

To translate, culturally adapt, and validate the Australian Hospital NAPS appropriateness assessment definitions of antimicrobial prescribing for Portugal.

Methods

International recommendations on translation and adaptation of instruments were followed. Two panels of experts participated in the process, using Zoom® for discussions and interviews, and Google Forms® for assessing vignettes. A native English-speaking person proficient in Portuguese conducted the back-translation. SPSS v.28 and Excel® were used for validity calculation.

Results

The Portuguese version was well accepted, its implementation being perceived as desirable and feasible by the experts. Validation process showed a Fleiss’ κ score of 0.483 (95% CI, 0.415–0.551, P < 0.005) for appropriateness, and an average agreement with the Australian NAPS team of 0.8 and 0.9, respectively, for appropriateness and reasons for inappropriateness.

Conclusions

The Portuguese version of the Australian Hospital NAPS appropriateness assessment definitions of antimicrobial prescribing, the first to be translated from English, was deemed non-inferior to the original, was well accepted, considered to be desirable and feasible, and could inspire other countries, particularly other Portuguese-speaking countries, to adapt and validate them in their own contexts, reinforcing the possibility of transferring NAPS use beyond Australia.

NOVA-Comprehensive Health Research Centre
==== Body
pmcIntroduction

Infections caused by antimicrobial-resistant bacteria, a ‘silent’ and ‘overlooked’ pandemic, are associated with increased morbidity, mortality and costs.1–4 Exposure to antimicrobials is the major drive for the emergence of bacterial resistance, a natural adaptative process that is at least 30 000 years old.5,6 There is a clear need to improve the use of antimicrobials to preserve their effectiveness and reduce associated side effects, in line with the implementation of the WHO One Health strategy, combining interventions in humans, animals and environment.7 The careful analysis of appropriateness of individual antimicrobial prescriptions, followed by feedback to the prescribers (audit and feedback strategy), is a major component of antimicrobial stewardship (AMS) programmes and the most effective intervention to optimize prescribing, and is central to gathering qualitative information about antimicrobial prescribing.8,9

To address the limitations resulting from the absence of a universal definition of appropriateness of antimicrobial prescribing, consensus-based definitions and a platform were developed by the Royal Melbourne Hospital Guidance Group and the National Centre for Antimicrobial Stewardship for the Australian National Antimicrobial Prescribing Survey (NAPS). The NAPS is a web-based qualitative auditing platform that provides a standardized and validated tool to assist hospitals in assessing the appropriateness of antimicrobial prescribing practices.10 After two pilot studies to evaluate feasibility and acceptance, and identify targets for quality improvement, the NAPS programme is now formally recognized as a key AMS activity to support national hospital accreditation in Australia and receives government funding to help sustain the programme’s clinical and technical support and delivery of annual aggregated reports for benchmarking. The NAPS is successfully performed in different hospital contexts, including public and private, across high- and low- to middle-income countries (Canada, New Zealand, the UK, Bhutan, Fiji, Malaysia, Papua New Guinea, Timor-Leste and Vietnam) and by various assessor types (physicians, pharmacists, nurses).11,12

A novel feature of the NAPS is the national and now international uptake of the defined metric of appropriateness. Appropriateness differs to guideline compliance, as not all antimicrobial indications have a guideline to assess against. Thus, the concept of appropriateness allows for an assessment of these clinical caveats, and for wider scalability to other settings and countries. The metric was developed by a multidisciplinary team of AMS experts and refined over the initial years of the implementation of the NAPS based on the plan–do–study–act framework for quality improvement.13 An inter-rater study was conducted in 2014 with a high level of agreement observed overall but was higher for compliance than appropriateness, as expected.14

Despite the implementation of a national programme with regulations, guidelines and indicators regarding antimicrobial therapy, Portugal has no standardized definitions of appropriateness for assessing antimicrobial prescription quality, resulting in heterogeneity of assessments and scarcity of qualitative data in this field.15,16

The translation, cross-cultural adaptation, and validation of the Hospital NAPS appropriateness assessment definitions for Portugal is an important step to prepare for piloting the NAPS auditing tool and methodology in the country. Piloting the NAPS in Portugal aims to improve knowledge on the magnitude of and the reasons for inappropriate antimicrobial prescribing. Implementing the NAPS will complement the existing quantitative information and, therefore, optimize AMS activities and antimicrobial prescribing.

This study aimed to translate, culturally adapt, and validate the Australian Hospital NAPS appropriateness assessment definitions for Portugal. Our objective extended beyond mere translation, to ensure that the adapted definitions resonate with the cultural and clinical context of Portuguese hospitals, thereby maintaining the robustness and reliability of the NAPS tool in evaluating and improving AMS activities.

Materials and methods

NAPS appropriateness assessment definitions

The NAPS is an online platform that contains the NAPS auditing tool. In the NAPS auditing tool, antimicrobial prescriptions are assessed as appropriate (optimal or adequate), inappropriate (suboptimal or inadequate) or not assessable, according to consensus-based appropriateness definitions.10 The Hospital NAPS appropriateness assessment definitions are presented in a coloured matrix that works to minimize subjectivity and standardize assessments. Importantly, the appropriateness assessment definitions are applicable in the absence of guidelines and allow for flexibility when guidelines and/or microbiological results are unavailable. The NAPS appropriateness assessment definitions are suitable for both therapeutic and surgical prophylaxis and consider the rationale for inappropriateness such as excessive or overlapping spectrum of activity, severity of patient allergies and the risk of drug toxicity.

Translation, cross-cultural adaptation, and validation of the NAPS appropriateness assessment definitions

Based on international recommendations on translation and adaptation of instruments, a step-by-step approach, summarized in Figure 1, was followed.17,18

Figure 1. Steps in the process of translation, cross-cultural adaptation, and validation of the Hospital NAPS appropriateness assessment definitions. This figure appears in colour in the online version of JAC and in black and white in the print version of JAC.

Step 1: Preparation

As a part of the PhD research of the first author, the Australian NAPS programme was contacted. Contractual arrangements between respective universities and Melbourne Health were signed for approval of a Pilot Portugal Hospital NAPS.

Portuguese physicians and pharmacists with expertise in AMS were invited purposively and distributed to participate in two separate panels, detailed in Table 1, aiming for translation and adaptation to Portuguese (panel 1) and validation of the translated contents (panel 2). To carry out the translation from English into Portuguese, the members of panel 1 were selected based on their proficiency in English.

Table 1. Panels of members involved in the translation and validation process of the NAPS appropriateness definitions

Panel members (initials)	Specialty	Number of years of AMS	
Panel 1			
 I.L.	Infectious Diseases	2	
 I.N.	Infectious Diseases	12	
 L.P.	Infectious Diseases	5	
 L.F.	Hospital Pharmacy	8	
 L.M.	Cardiothoracic Surgery	6	
Panel 2			
 C.N.	Infectious Diseases	7	
 D.P.	Internal Medicine and Intensive Care Medicine	8	
 E.P.	Infectious Diseases	3	
 J.B.	Infectious Diseases	4	
 N.P.	Infectious Diseases	6	
 M.C.	Internal Medicine and Intensive Care Medicine	8	
 M.J.R.	Hospital Pharmacist	4	
 P.M.	Internal Medicine and Intensive Care Medicine	3	
 P.R.	Internal Medicine	5	
 S.L.	Infectious Diseases	6	

Invitations were made by e-mail and phone calls. Written informed consent was obtained for the members of panel 2.

Step 2: Forward translation

I.L., a member of panel, 1 carried out the first translation of the Hospital NAPS appropriateness assessment definitions (Portugal Hospital NAPS definitions, version 1.0).

Step 3: Review and reconciliation of the translation

Version 1.0 was e-mailed to and independently reviewed by the other members of panel 1, followed by a Zoom® meeting (Zoom Video Communications, Inc., USA), conducted by C.P. and M.J.D.S., resulting in a reconciliated translation (version 1.1).

Step 4: Back-translation to English

Version 1.1 was sent by e-mail and back translated by J.Y., an native English-speaking person proficient in Portuguese, who was not privy to the original NAPS definitions.

Step 5: Back-translation review

The back-translation was reviewed and discussed with J.Y. (see Acknowledgements section) by C.P. for adjustments before being sent to the Australian NAPS team, which gave its approval. The result was version 2.0. Changes in the process of translation are summarized in Table 2.

Table 2. Summary of changes in the translation process of the Hospital NAPS appropriateness assessment definitions

English original (NAPS)	Version 1.0 (initial translation)	Version 1.1 (reconciliated translation)	Version 2.0 (after back-translation)	Version 3.0 (after proofreading)	
If endorsed guidelines present/absent	Na presença/ausência de recomendações validadas	Recomendações terapêuticas disponíveis/não disponíveis	Recomendações disponíveis/não disponíveis		
The antimicrobial prescription was reviewed and endorsed by an infectious diseases clinician or a clinical microbiologist	A prescrição antimicrobiana foi revista e validada por um infeciologista ou microbiologista clínico	A prescrição antimicrobiana foi revista e validada por um membro da equipa do PAPA ou por um infeciologista ou microbiologista clínico			
…Antimicrobial choice…	…Escolha do fármaco…	…Escolha do antimicrobiano…			
Documented or presumed indication…	A infeção documentada ou presumida…	A indicação documentada ou presumida…			
The prescribed antimicrobial will cover the likely or cultured pathogens…	O antimicrobiano prescrito cobre os agentes etiológicos mais prováveis ou os identificados…		O antimicrobiano prescrito atua sobre os agentes etiológicos mais prováveis ou os identificados…		
Antimicrobial prescription does not optimally follow…however is a reasonable choice for the likely causative or cultured pathogens	A prescrição antimicrobiana não está totalmente de acordo… mas constitui uma alternativa aceitável para cobertura dos agentes etiológicos mais prováveis ou identificados nos exames culturais		A prescrição antimicrobiana não está totalmente de acordo… mas constitui uma alternativa aceitável para cobertura dos agentes etiológicos mais prováveis ou identificados nos exames culturais e/ou o contexto clínico do doente		
For surgical prophylaxis, as above and duration is less than 24 hours	Em profilaxia cirúrgica, como acima e com duração inferior a 24 horas		Em profilaxia cirúrgica, como acima, administrada no timing preconizado e com duração não excedendo as 24 horas	Em profilaxia cirúrgica, como acima, administrada na janela temporal preconizada e com duração não excedendo as 24 horas	
…Failure to appropriately de-escalate with microbiological results	…Falha na apropriada descalação, de acordo com os resultados microbiológicos	…Falha na descalação, de acordo com os resultados microbiológicos	…Falha na descalação, de acordo com os resultados microbiológicos e a evolução favorável do doente		

Step 6: Pre-testing and cognitive interviewing

Based on version 2.0, 10 clinical vignettes requiring antimicrobial appropriateness assessments (Table 3), developed by C.P., were discussed within panel 1. Members of panel 2 applied version 2.0 of the Portugal Hospital NAPS appropriateness assessment definitions to assess the appropriateness of the antimicrobials. Assessments were completed via Google Forms® (Google Corp., USA). In the case of antimicrobials assessed as inappropriate, panel 2 members were required to select one or more reasons for inappropriateness, as defined in the NAPS: the indication did not require any antimicrobials; surgical prophylaxis lasting more than 24 h; incorrect route of administration, dose or frequency, or duration; spectrum too broad or too narrow; allergy or microbiology mismatch.

Table 3. Agreement rate of assessments of the clinical vignettes (appropriateness and reasons for inappropriateness) with the Australian team

Clinical vignette (V)	Classification of appropriateness	Reasons for inappropriateness (all reasons)	Reasons for inappropriateness (at least one reason)	
V1: Pneumococcal pneumonia treated with amoxicillin/clavulanate	0.8	0.5	1	
V2: Acute exacerbation of COPD treated with levofloxacin	0.8	0.3	0.9	
V3: Acute pyelonephritis treated with meropenem	1	0.1	0.8	
V4: Non-complicated influenza treated with azithromycin	0.9	0.9	1	
V5: Asymptomatic bacteriuria treated with trimethoprim/sulfamethoxazole	1	1	1	
V6: Skin and soft tissue infection treated with piperacillin/tazobactama	0.6	0	0.9	
V7: Skin and soft tissue infection treated with vancomycina	0.4	0.6	0.6	
V8: Skin and soft tissue infection treated with linezolid	0.6	0.5	0.5	
V9: Surgical prophylaxis with cefazolin	0.9	0.2	0.9	
V10: Viral diarrhoea treated with ciprofloxacin	1	0.8	0.9	
V1–V10				
 Average	0.8 (0.9)b	0.5 (0.5)b	0.9 (0.9)b	
 Median	0.7 (0.9)b	0.5 (0.5)b	0.9 (0.9)b	
 Mode	— (1)b	0.5 (0.5)b	0.9 (0.9)b	
aThe Portuguese auditors and the Australian NAPS team noticed poor information regarding clinical evolution and laboratory results.

bRevised agreement score once V6 and V7 were excluded.

Internal validity for the overall assessment of appropriateness was measured using the inter-rater context statistic of Fleiss’ Kappa coefficient (κ) on SPSS v.28 (IBM corp., USA).19,20

External validity was conducted via comparison of the reference assessments provided by the Australian team (C.I., R.J., R.C., C.C. and Z.R.; see Acknowledgements section) with the English translated vignettes and was calculated using Excel® (Microsoft Corp., USA).

Subsequently, the members of panel 2 participated in individual, semi-structured interviews, based on a script that was developed by C.P. and M.J.D.S. aiming to explore the following dimensions (Table 4): the layout (D1) and the content (D2) of the Portugal Hospital NAPS appropriateness assessment definitions; the experience of using the definitions in the clinical vignettes (D3); and the potential for widespread use in the Portuguese context (D4). Interviews were conducted by C.P. (and M.J.D.S. for the first one), allowing for interviewees to use their own words to express their views and experience, with attention to reducing interviewer bias. Interviews took place (and were recorded) using Zoom®, mainly after working hours, and outside the working place to minimize interferences. During the interviews, notes were taken by C.P., contributing to the further analysis.

Table 4. Interviewing debriefing results

Dimensions (D)	Questions	Strengths/opportunities	Weaknesses/difficulties	Challenges/suggestions	
D1
Layout of the Portugal Hospital NAPS appropriateness assessment definitions.	What do you think of the layout of the document you used with the definitions?	Pleasant, clear and easy to read.
Coloured boxes and bold words.
‘It makes a lot of sense and is easy to find the box that fits the best.’ a	Centred text.
Poor contrast of colours between the boxes.
Restricted number of words in bold.	Increase the contrast of colours between boxes.
Increase the number of words in bold.
Convert centred text to justified.	
D2
Content of the Portugal Hospital NAPS appropriateness assessment definitions.	What do you think of the definitions used to classify the appropriateness of prescriptions?	The ones mentioned in D1.
Clear, objective and easy to understand concepts.
Well-balanced definitions between depth and applicability.
Flexibility, not restricted to compliance with guidelines or specific situations.
Comprehensiveness, taking into account with several factors to be taken into account in the prescription.
‘Quite clear’.a	Absence of ‘optimal’ category for surgical prophylaxis.
Exclusion of information regarding the timing of the first administration of antibiotics and possible intra-operative boosts.
Little focus on clinical evolution when compared with microbiological results for evaluation and de-escalation.
Allergies and microbiology results have the same weight for classifying as inappropriate.	Need for adaptation and transition to these definitions.
Need for learning and training on these definitions.
To create an ‘optimal’ category for surgical prophylaxis.
To include information regarding the timing of the first administration of antibiotics and possible intra-operative boosts.
To include the clinical evolution beside microbiological results for evaluation and de-escalation.
In surgical prophylaxis, replace ‘duration less than 24 hours’ by ‘duration no more than 24 hours’.	
D3
Ease of use of the Portugal Hospital NAPS appropriateness assessment definitions in the pre-test (clinical vignettes).	How easy do you think it is to use the definitions to classify the appropriateness of prescriptions?	Fast and easy to use, with minimal training.
Suitable for the classification of hospital appropriateness of antimicrobial prescribing in hospitals.
Usable by physicians and pharmacists.
‘Super easy’. a	Difficulties in classifying some prescriptions due to insufficient information in the clinical vignettes that could be overcame in real situations.
Possible future difficulties in distinguishing between adequate and suboptimal prescriptions in some situations, especially by non-prescribers (pharmacists).
Time-consuming	Need for familiarization with the concepts and training, especially by non-prescribers (pharmacists).	
D4
Potential for widespread use of the of the Portugal Hospital NAPS appropriateness assessment definitions.	What is your opinion on the applicability of the definitions in assessing the quality of antibiotic prescriptions by the antimicrobial stewardship teams in Portuguese hospitals?	Feasible for both daily evaluations and prevalence studies.
Opportunity to obtain information on the quality of antimicrobial prescribing in Portuguese hospitals, based on standardized definitions, allowing their internal and external comparability and improving assessment by the AMS teams and their interaction with prescribers.
‘Very Useful’. a
‘Great opportunity’. a	Expected difficulties resulting from scarcity of national guidelines for antimicrobial use.
Time-consuming.	Need for familiarization with the concepts and training, especially by non-prescribers (pharmacists).
Need for complement with informatic tools for register and computation.	
aExpert quotes.

Step 7: Review of cognitive interviewing results

Based on the information gathered in the written notes and the recorded interviews, a synopsis was constructed through a debriefing process involving C.P. and M.J.D.S., in which the content of all interviews was organized and coded into strengths/opportunities, weaknesses/difficulties and challenges/suggestions. Non-verbal language was not analysed. Data were further condensed and are summarized in Table 4. Further modifications to the Portugal Hospital NAPS appropriateness assessment definitions were made to consider the interview analyses and suggestions from panel 2. Further discussions and approval from panel 1 and the Australian NAPS team led to version 2.1.

Step 8: Proofreading and final version

Subsequent proofreading by a physician (M.L.d.S.) and a hospital manager (D.T.) (see Acknowledgements section) ensured typographical and grammatical accuracy, leading to the final version of the Portugal Hospital NAPS appropriateness assessment definitions (version 3.0).

Results

Translation and back-translation

The process up to version 2.0 resulted from a consensus between the members of panel 1. Minor word corrections in the initial back-translated version were completed before they were shared with the Australian NAPS team, who accepted the version. Table 2 summarizes the main changes, taking as reference the first translated version.

Validation

The validation process was based on the assessment of 100 evaluations of 10 clinical vignettes by the 10 members of panel 2, using version 2.0.

Internal validity was assessed by calculating inter-rater reliability, using Fleiss’ κ coefficient for appropriateness assessments.

Two vignettes were recognized as having insufficient information, a problem also potentially faced in real-world situations and difficult evaluations. When all vignettes were considered, the calculated κ was 0.483 (95% CI, 0.415–0.551, P < 0.005), meaning a moderate strength of agreement (degrees of agreement: 0.01–0.2, slight; 0.21–0.4, fair; 0.41–0.6, moderate; 0.61–0.8, substantial; 0.81–1.0, almost perfect).13 With the exclusion of two vignettes that were noted to have less information regarding follow-up and microbiology results, κ increased to 0.586.

Regarding external validity (Table 3), the agreement with the classification of overall appropriateness assessments between the Portuguese auditors and the Australian NAPS team ranged from 0.4 to 1 (average 0.8; median 0.7). The agreement for all the selected reasons for inappropriateness ranged from 0 to 1 (average, median and mode 0.5). When the analysis was restricted to the agreement of at least one reason, these results had an increase and ranged from 0.5 to 1 (average, median and mode 0.9). As for the Fleiss’ κ, with the exclusion of the two vignettes, the results also had a slight improvement.

Interviews and debriefing

The duration of each interview ranged between 21 and 32 min (average and median 24.5). The participants were very empathetic, enthusiastic and proactive, which, as experts, contributed to the richness of the analysis. Regarding the adequacy of the definitions of this tool, four dimensions were explored. Results are summarized in Table 4, organized as: strengths/opportunities; weaknesses/difficulties; and challenges/suggestions for each of the discussed dimensions.

After discussion within panel 1 and the Australian NAPS team, some of the suggestions made in the interviews were accepted, resulting in version 2.1. According to the interviewing debriefing, improvements were made to the definitions. Suggestions related to changes in surgical prophylaxis (inclusion of information about intra-operative re-dosing) were not accepted by the Australian NAPS team, as they would diverge significantly from the current NAPS appropriateness assessment definitions. The Australian NAPS team noted that the category ‘optimal’ is already applicable to surgical prophylaxis and didn’t require duplication of text to reiterate this. Corrections were made to the clinical vignettes for future utilization. Finally, the proofreading process suggested a single change, which was approved by panel 1 and the Australian NAPS team, leading to the final version of the Pilot Portugal Hospital NAPS appropriateness assessment definitions (version 3.0, Figure 2) that will be used in the Pilot Portugal Hospital NAPS.

Figure 2. Portugal Hospital NAPS appropriateness assessment definitions, version 3.0.

Discussion

There is a clear need to improve the appropriateness of antimicrobial use towards its ‘prudent’ or ‘responsible’ utilization.21,22 Appropriateness is an important quality measure, although defining it is a challenge in itself.23 The implementation of initiatives aimed at gathering the quality of antimicrobial prescribing requires the adoption of standardized definitions of antimicrobial appropriateness. This aims to reduce the subjectivity associated with the ‘eye of the interpreter’ phenomenon.24,25 In comparison with ‘guideline compliance’, which is commonly referred to in the current literature, appropriateness can be applied in scenarios where guidelines are not available. Not all indications for antimicrobials have guidelines, and not all countries or facilities have contextually relevant guidelines. Thus, appropriateness allows for greater scalability across multiple facilities and countries. The Australian Hospital NAPS appropriateness assessment definitions have been successfully implemented since 2013, with NAPS audits performed across different hospital contexts, by various assessor types and in different cultural settings.10–12 The translation, cross-cultural adaptation and validation of the Australian Hospital NAPS appropriateness assessment definitions for Portugal aims to support the implementation of the Hospital NAPS methodology in Portugal through the realization of a pilot study (Pilot Portugal Hospital NAPS) to address the current knowledge gaps regarding antimicrobial prescribing quality, and associated reasons for inappropriateness. This novel application of translation, back-translation and validation of the Hospital NAPS appropriateness assessment definitions beyond the English language will support the implementation of a high-quality Pilot Portugal Hospital NAPS programme.

Translation and back-translation

International recommendations on translation and adaptation of instruments, including the need for conceptual and cultural equivalence, were reviewed and were central to this process.17,18 The iterative discussion of the independent reviews by the members of the panel of experienced physicians and pharmacists in AMS allowed the achievement of a high-quality version, as demonstrated by its consensus approval by the Australian NAPS team.

Validation

The decision to use clinical vignettes for simulation purposes is based on a well-established method to evaluate knowledge and behaviour, namely in antimicrobial prescribing.26–29

The use of the Fleiss’ κ coefficient is suitable for the calculation of agreement among several assessors (internal validity) and allows comparability with other studies in this field.19,20 The obtained score of 0.483 (0.586 if two vignettes were excluded) reflects a moderate agreement, in line with other papers comparing antimicrobials with other medications, as well as with the results obtained in the remote assessment of appropriateness in Australia, reflecting the complexity of antimicrobial use and the difficulties in its assessment.30–33

The average degree of agreement with the classification done by the Australian NAPS team (external validity) of 0.8 reflects a good ability of the translated version to measure appropriateness, pointing towards feasibility and generalizability of the Hospital NAPS appropriateness assessment definitions beyond Australia.11,12 Best results were achieved for situations where the use of antibiotics was not required (asymptomatic bacteriuria, viral gastroenteritis, influenza), as well as for surgical prophylaxis. The worst result was obtained for the treatment of skin and soft tissue infection with vancomycin, possibly resulting from insufficient information related to clinical evolution. Erroneous identification of linezolid as a broad-spectrum antibiotic and absence of de-escalation of amoxicillin/clavulanate contributed to a lower agreement with the Australian team assessments. The misinterpretation of the concept of ‘microbiology mismatch’ and the difficulties felt in the interpretation of the two vignettes could also have contributed to lower results but do also reflect real life and the scarcity of information in medical records. The obtained results for the agreement of at least one selected reason for inappropriateness (average, median and mode 0.9) is similar to the assessments of the Australian NAPS team.

Interviews and debriefing

Interviews and debriefing are well-established tools in qualitative research that allow for free expression of thoughts and feelings that otherwise would not be possible to obtain (‘think aloud’).34–36 A script was developed to provide an interview guide and structure that allowed for flexibility in participant responses and elaboration of new concepts and ideas provided by the participant. The use of remote technologies, which had its maximal expression during the COVID-19 pandemic, has been proved as a valuable method in qualitative research.37

Interviews and debriefing to a panel of experienced Portuguese physicians and pharmacists involved in AMS activities supports our conclusions that the translated, culturally adapted to Portugal and validated Hospital NAPS appropriateness assessment definitions were well accepted, and its implementation perceived as desirable and feasible, with a potential for widespread application in Portuguese hospitals. Thus, the implementation of a Pilot Portugal Hospital NAPS provides an opportunity to increase knowledge about antimicrobial prescribing in Portuguese hospitals. According to the panel, challenges are expected, related to the transition from other definitions already in use in some hospitals, the choice between adequate and suboptimal subcategories in some cases, or its utilization by non-prescribers. Contrary to Australia, Portugal has a limited number of national guidelines for antimicrobial prescribing, which further highlights the importance of translating the Hospital NAPS appropriateness assessment definitions to minimize the subjectiveness of these assessments. AMS activities are mainly allocated to physicians, while pharmacists are not widely involved in antimicrobial appropriateness assessments. This will require additional education and training, which should be seen as an opportunity for multidisciplinary collaborations. While the original Hospital NAPS appropriateness assessment definitions are applicable to any antimicrobial and to paediatric and intensive care patients, the Portugal Hospital NAPS appropriateness assessment definitions were pre-tested only for antibiotics prescribed for non-critically ill adult patients.

Conclusions

Despite the cultural differences between Australia and Portugal in antimicrobial prescribing and AMS activities, the translation presented herein of the Australian Hospital NAPS appropriateness assessment definitions was deemed non-inferior to the original. It was well accepted and perceived as desirable and feasible by a panel of Portuguese experts on AMS to improve knowledge on the quality of prescribing, allow for benchmarking and, ultimately, optimize AMS activities and the quality of prescribing in Portuguese hospitals. These definitions and the NAPS methodology will be incorporated in the Pilot Portugal Hospital NAPS. This is the first non-English version of the Hospital NAPS appropriateness assessment definitions, reinforcing the already demonstrated feasibility, generalizability and transferability of the original Hospital NAPS definitions and methodology beyond Australia. This work could pave the way and inspire other countries, particularly Portuguese-speaking countries, to adapt and validate them in their own linguistic, cultural and organizational contexts.

Acknowledgements

We wish to express our gratitude to the Australian NAPS programme and to all the participants in the processes of translation, back-translation, adaptation, validation and proofreading: Caroline Chen, Cláudia Nazareth, Duarte Tavares, Dulce Pascoalinho, Eduarda Pena, Isabel Neves, James Yates, Joana Batista, Leonor Pássaro, Luís França, Luís Miranda, Margarida Câmara, Maria José Rei, Marta Leal dos Santos, Nuno Pereira, Paulo Mergulhão, Pedro Raimundo, Ron Cheah, Sara Lino and Zohal Rashidzada.

Funding

The Open Access publication of this paper is supported by a fee from Fundação Ciência e Tecnologia, IP national support through CHRC (UIDP/04923/2020).

Transparency declarations

None to declare.
==== Refs
References

1 Laxminarayan R . The overlooked pandemic of antimicrobial resistance. Lancet 2022; 399 : 606–7. 10.1016/S0140-6736(22)00087-3 35065701
2 Antimicrobial Resistance Collaborators . Global burden of bacterial antimicrobial resistance in 2019: a systematic analysis. Lancet 2022; 399 : 629–55. Published correction in: Lancet 2022; 400: 1102. 10.1016/S0140-6736(21)02724-0 35065702
3 OECD . Stemming the Superbug Tide: Just A Few Dollars More. OECD Publishing, 2018.
4 O’Neill J. Tackling Drug-Resistant Infections Globally: Final Report and Recommendations. 2016. https://amr-review.org/sites/default/files/160518_Final%20paper_with%20cover.pdf.
5 D’Costa VM , KingCE, KalanLet al Antibiotic resistance is ancient. Nature 2011; 477 : 457–61. 10.1038/nature10388 21881561
6 Courvalin P . Why is antibiotic resistance a deadly emerging disease? Clin Microbiol Infect 2016; 22 : 405–7. 10.1016/j.cmi.2016.01.012 26806259
7 WHO . One Health. https://www.who.int/health-topics/one-health#tab=tab_1health
8 Davey P , MarwickCA, ScottCLet al Interventions to improve antibiotic prescribing practices for hospital inpatients. Cochrane Database Syst Rev 2017; 2 : CD003543. 10.1002/14651858.CD003543.pub4 28178770
9 Palos C , SousaP. Leveraging antimicrobial stewardship: focus on individual prescriptions appropriateness. In: CotrimT, SerranheiraF, SousaP, HignettS, AlbolinoS, TartagliaR, eds. Health and Social Care Systems of the Future: Demographic Changes, Digital Age and Human Factors. HEPS 2019. Advances in Intelligent Systems and Computing, Volume 1012. Springer, 2019; 127–34. 10.1007/978-3-030-24067-7_15
10 James R , UpjohnL, CottaMet al Measuring antimicrobial prescribing quality in Australian hospitals: development and evaluation of a national antimicrobial prescribing survey tool. J Antimicrob Chemother 2015; 70 : 1912–8. 10.1093/jac/dkv047 25724986
11 James R , NakamachiY, MorrisAet al The feasibility and generalizability of assessing the appropriateness of antimicrobial prescribing in hospitals: a review of the Australian National Antimicrobial Prescribing Survey. JAC Antimicrob Resist 2022; 4 : dlac012. 10.1093/jacamr/dlac012 35156035
12 Chuki P , DorjiT, JamesRet al Antibiotic use and quality indicators of antibiotic prescription in Bhutan: a point prevalence survey using the Australian National Antimicrobial Prescribing Survey tool. JAC Antimicrob Resist 2023; 5 : dlad100. 10.1093/jacamr/dlad100 37614960
13 Langley GJ Provost LP , MoenRDet al The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. John Wiley & Sons, 2009.
14 National Centre for Antimicrobial Stewardship and Australian Commission on Safety and Quality in Health Care . Antimicrobial prescribing practice in Australia: results of the 2014 national antimicrobial prescribing survey. 2015. https://www.safetyandquality.gov.au/sites/default/files/migrated/Antimicrobial-prescribing-practice-in-Aust-hospitals-NAPS-2014-Results.pdf
15 Portuguese Government . Despacho n.° 10901/2022. Diário da República n.° 174/2022, Série II de 2022-09-08. Atualiza o Programa de Prevenção e Controlo de Infeções e de Resistência aos Antimicrobianos (PPCIRA). 2022. https://diariodarepublica.pt/dr/detalhe/despacho/10901-2022-200789503
16 Portuguese Government . Despacho n.° 9390/2021. Diário da República n.° 187/2021, Série II de 2021-09-24. Plano Nacional para a Segurança dos Doentes 2021-2026. 2021. https://diariodarepublica.pt/dr/detalhe/despacho/9390-2021-171891094
17 WHO. WHO Guidelines on Translation and Adaptation of Instruments. https://www.scribd.com/document/533869240/WHO-Guidelines-on-Translation-and-Adaptation-of-Instruments
18 Wild D , GroveA, MartinMet al Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: report of the ISPOR task force for translation and cultural adaptation. Value Health 2005; 8 : 94–104. 10.1111/j.1524-4733.2005.04054.x 15804318
19 Landis JR , KochGG. The measurement of observer agreement for categorical data. Biometrics 1977; 33 : 159–74. 10.2307/2529310 843571
20 Cole R . Inter-rater reliability methods in qualitative case study research. Sociol Methods Res 2023; 10.1177/00491241231156971
21 ECDC . Proposals for EU guidelines on the prudent use of antimicrobials in humans. 2017. https://www.ecdc.europa.eu/en/publications-data/proposals-eu-guidelines-prudent-use-antimicrobials-humans
22 Monnier AA , SchoutenJ, Le MaréchalMet al Quality indicators for responsible antibiotic use in the inpatient setting: a systematic review followed by an international multidisciplinary consensus procedure. J Antimicrob Chemother 2018; 73 Suppl 6 : vi30–9. 10.1093/jac/dky116 29878221
23 Buetow SA , SibbaldB, CantrillJAet al Appropriateness in health care: application to prescribing. Soc Sci Med 1997; 45 : 261–71. 10.1016/s0277-9536(96)00342-5 9225413
24 DePestel DD , EilandEHIII, LusardiKet al Assessing appropriateness of antimicrobial therapy: in the eye of the interpreter. Clin Infect Dis 2014; 59 Suppl 3 : S154–61. 10.1093/cid/ciu548 25261542
25 Spivak ES , CosgroveSE, SrinivasanA. Measuring appropriate antimicrobial use: attempts at opening the black box. Clin Infect Dis 2016; 63 : 1639–44. 10.1093/cid/ciw658 27682070
26 Peabody JW , LuckJ, GlassmanPet al Measuring the quality of physician practice by using clinical vignettes: a prospective validation study. Ann Intern Med 2004; 141 : 771–80. 10.7326/0003-4819-141-10-200411160-00008 15545677
27 van Hauwermeiren E , IosifidisE, KärkiTet al Development of case vignettes for assessment of the inter-rater variability of national validation teams for the point prevalence survey of healthcare-associated infections and antimicrobial use in European acute care hospitals. J Hosp Infect 2019; 101 : 455–60. 10.1016/j.jhin.2019.01.018 30682398
28 Lucet JC , Nicolas-ChanoineMH, RoyCet al Antibiotic use: knowledge and perceptions in two university hospitals. J Antimicrob Chemother 2011; 66 : 936–40. 10.1093/jac/dkq541 21393217
29 van der Voort T , BrinkmanDJ, BenemeiSet al Appropriate antibiotic prescribing among final-year medical students in Europe. Int J Antimicrob Agents 2019; 54 : 375–9. 10.1016/j.ijantimicag.2019.05.008 31075400
30 Sikkens JJ , van AgtmaelMA, PetersEJet al Assessment of appropriate antimicrobial prescribing: do experts agree? J Antimicrob Chemother 2016; 71 : 2980–7. 10.1093/jac/dkw207 27287237
31 Mol PG , GansRO, PandayPVet al Reliability of assessment of adherence to an antimicrobial treatment guideline. J Hosp Infect 2005; 60 : 321–8. 10.1016/j.jhin.2004.11.022 16002017
32 Taylor CT , StewartLM, ByrdDCet al Reliability of an instrument for evaluating antimicrobial appropriateness in hospitalized patients. Am J Health Syst Pharm 2001; 58 : 242–6. 10.1093/ajhp/58.3.242 11217180
33 Cotta MO , SpelmanT, ChenCet al Evaluating antimicrobial therapy: how reliable are remote assessors? Infect Dis Health 2016; 21 : 3–10. 10.1016/j.idh.2016.01.002
34 Wills GB . Cognitive Interviewing: A Tool for Improving Questionnaire Design. 2005. https://us.sagepub.com/en-us/nam/cognitive-interviewing/book225856
35 Staunton H , WillgossT, NelsenLet al An overview of using qualitative techniques to explore and define estimates of clinically important change on clinical outcome assessments. J Patient Rep Outcomes 2019; 3 : 16. 10.1186/s41687-019-0100-y 30830492
36 McMahon SA , WinchPJ. Systematic debriefing after qualitative encounters: an essential analysis step in applied qualitative research. BMJ Glob Health 2018; 3 : e000837. 10.1136/bmjgh-2018-000837
37 Gill P , BaillieJ. Interviews and focus groups in qualitative research: an update for the digital age. Br Dent J 2018; 225 : 668–72. 10.1038/sj.bdj.2018.815
