
==== Front
PLoS One
PLoS One
plos
PLOS ONE
1932-6203
Public Library of Science San Francisco, CA USA

10.1371/journal.pone.0310125
PONE-D-24-05416
Registered Report Protocol
Computer and Information Sciences
Artificial Intelligence
Research and Analysis Methods
Research Assessment
Systematic Reviews
Biology and Life Sciences
Neuroscience
Cognitive Science
Cognitive Psychology
Intelligence
Human Intelligence
Biology and Life Sciences
Psychology
Cognitive Psychology
Intelligence
Human Intelligence
Social Sciences
Psychology
Cognitive Psychology
Intelligence
Human Intelligence
Biology and Life Sciences
Neuroscience
Cognitive Science
Cognition
Memory
Face Recognition
Biology and Life Sciences
Neuroscience
Learning and Memory
Memory
Face Recognition
Biology and Life Sciences
Neuroscience
Cognitive Science
Cognitive Psychology
Perception
Face Recognition
Biology and Life Sciences
Psychology
Cognitive Psychology
Perception
Face Recognition
Social Sciences
Psychology
Cognitive Psychology
Perception
Face Recognition
Research and Analysis Methods
Database and Informatics Methods
Database Searching
Computer and Information Sciences
Artificial Intelligence
Machine Learning
Support Vector Machines
Computer and Information Sciences
Artificial Intelligence
Machine Learning
Biology and Life Sciences
Neuroscience
Cognitive Science
Cognitive Psychology
Learning
Human Learning
Biology and Life Sciences
Psychology
Cognitive Psychology
Learning
Human Learning
Social Sciences
Psychology
Cognitive Psychology
Learning
Human Learning
Biology and Life Sciences
Neuroscience
Learning and Memory
Learning
Human Learning
Artificial intelligence in human resource development: An umbrella review protocol
Artificial intelligence in human resource development: An umbrella review protocol
https://orcid.org/0000-0003-4338-3614
Yoo Sangok Conceptualization Data curation Methodology Project administration Supervision Validation Writing – original draft Writing – review & editing *
Nimon Kim Conceptualization Data curation Methodology Software Supervision Validation Writing – review & editing
Patole Sanket Ramchandra Conceptualization Data curation Validation Writing – review & editing
Human Resource Development, The University of Texas at Tyler, Tyler, Texas, United States of America
Correa Juan Editor
Critical Centrality Institute, MEXICO
Competing Interests: The authors have declared that no competing interests exist.

* E-mail: syoo@uttyler.edu
9 9 2024
2024
19 9 e03101258 2 2024
23 8 2024
© 2024 Yoo et al
2024
Yoo et al
https://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

The recent surge in artificial intelligence (AI) has significantly transformed work dynamics, particularly in human resource development (HRD) and related domains. Scholars, recognizing the significant potential of AI in HRD functions and processes, have contributed to the growing body of literature reviews on AI in HRD and related domains. Despite the valuable insights provided by these individual reviews, the challenge of collectively interpreting them within the HRD domain remains unresolved. This protocol outlines the methodology for an umbrella review aiming to systematically synthesize existing reviews on AI in HRD. The review seeks to address key research questions regarding AI’s contributions to HRD functions and processes, as well as the opportunities and threats associated with its implementation by employing a technology-aided systematic approach. The coding framework will be used to synthesize the contents of the selected systematic reviews such as their search strategies, data synthesis approaches, and HRD-related findings. The results of this umbrella review are expected to provide insights for HRD scholars and practitioners, promoting continuous improvement in AI-driven HRD initiatives. This protocol is preregistered on the Open Science Framework (https://doi.org/10.17605/OSF.IO/Z8NM6) on May 27, 2024.

The author(s) received no specific funding for this work. Data AvailabilityAll supplementary files are available in an open-access repository: https://osf.io/af6d7/.
Data Availability

All supplementary files are available in an open-access repository: https://osf.io/af6d7/.
==== Body
pmcIntroduction

Artificial intelligence (AI) refers to the ability of machines to perform near or human-like functions, such as learning, interaction, and problem-solving, encompassing the culmination of computers, computer-related technologies, machines, and information communication technology innovations and developments, giving computers the ability to perform [1, 2]. The AI market is anticipated to reach a $407 billion by 2027, indicating substantial growth from its estimated revenue of $86.9 billion in 2022. This surge is projected to make a 21% net contribution to the United States GDP by 2030, highlighting the profound impact of AI on economic growth [3]. Reasonably, a considerable 64% of businesses believe artificial intelligence will enhance their overall productivity [3]. Furthermore, according to an annual McKinsey Global Survey conducted in mid-April 2023, generative AI (Gen AI) has captured significant attention across the business landscape. Individuals from various regions, industries, and seniority levels are incorporating Gen AI into their professional and personal activities in their workplaces [4].

The recent proliferation of AI has dramatically changed the way we work [2, 5]. In the field of human resource development (HRD) and related areas, the integration of AI presents opportunities to optimize talent acquisition, streamline learning and development initiatives, and enhance the strategic values of HRD in the workplace [5, 6]. The far-reaching impact of AI underscores the need for a nuanced understanding of its role in HRD functions.

In academia, a burgeoning interest in AI in the workplace is evident through the growing body of research, leading to a surge of literature reviews focused on AI in HRD and related areas (e.g., [2, 5, 7]). For example, [5] conducted a critical review of the literature on AI and its impact on workplace outcomes, specifically within HR functions. [6] delved into the literature on AI applications, with a particular emphasis on the learning and development function. Despite the valuable contributions of these endeavors, the question of how these individual reviews can be collectively interpreted within the field of HRD remains unanswered.

To attain a comprehensive understanding of the rapidly expanding knowledge base, there is a need to systematically synthesize existing reviews on AI in HRD and related areas. An umbrella review, representing the highest level of evidence, offers a comprehensive overview of existing systematic reviews in a specific field. It enables scholars to compare the findings of systematic reviews relevant to a specific review question [8, 9].

Hence, the proposed review outlined in this protocol aims to unveil patterns, trends, and gaps in the current understanding of AI in HRD literature. Additionally, we expect that this umbrella review will provide HRD scholars and practitioners with insights into the evolving concepts and practices associated with AI in HRD, thereby promoting continuous improvement in AI-driven HRD initiatives. The key research questions to be addressed in our umbrella review are:

RQ1: How does AI contribute to HRD functions and processes?

RQ2: What are the opportunities and threats of implementing AI in HRD?

In pursuit of the objective, this protocol proposes a technology-aided umbrella review process to synthesize systematic literature reviews on AI in the field of HRD and related areas. This systematic approach is designed to alleviate subjectivity in the review process, including the selection of search terms, thereby enhancing the rigor and objectivity of this umbrella review.

Materials and methods

Design and setting of the study

This technology-aided umbrella review protocol adheres to the guidelines of PRISMA-P (Preferred reporting items for systematic review and meta-analysis protocols), serving as a guide for planning and documenting review methods [10, 11]. The completed PRISMA-P checklist to confirm essential and minimum components of a systematic review is available in the S1 File. To achieve a comprehensive understanding of AI implementation in HRD, this protocol is designed to systematically incorporate existing systematic literature reviews on AI in HRD and related areas, mitigating subjective decision-making during review conduct [10]. This protocol is pre-registered on the Open Science Framework (OSF): https://doi.org/10.17605/OSF.IO/Z8NM6. In the main research using this protocol, we plan to incorporate guidelines from the updated PRISMA 2020 statement to ensure comprehensive reporting of our umbrella review [12].

Database and data management

A structured search will be conducted in the Scopus and Web of Science databases, selected for their relevance to the field of study and comprehensive coverage. The review process, encompassing screening, will be coordinated utilizing Rayyan to ensure a systematic and efficient workflow [13].

Search strategy

Keywords to create a comprehensive search string that will be used to search systematic reviews for this umbrella review were collected. Table 1 describes the final search sub-strings of each component. AI-, HRD-, and SLR-related strings include search terms combined using the Boolean operator OR. In the final search string, the Boolean operator AND will be used to combine the three sub-strings. As our umbrella review aims to synthesize existing systematic literature reviews, the SLR-related string includes one search term that narrows the scope of our project. The specific search term identification strategy and term matching details are illustrated in the supplementary files (S2 and S3 Files).

10.1371/journal.pone.0310125.t001 Table 1 Final search sub-strings of each component a.

AI-related string	HRD-related string	SLR-related string	
"AI" OR algorithm* OR automation OR "artificial intelligen*" OR "artificial-intelligen*" OR "augmented reality" OR "autonomous agent*" OR bayesian* OR block-chain OR blockchain OR "business intelligence" OR chat* OR "cloud computing" OR cloud-computing OR "collaborative intelligence" OR "collective intelligence" OR "competitive intelligence" OR "complex network*" OR computation* OR computer* OR "conversational agent*" OR "deep learning" OR "digital transformation" OR "digital twin*" OR "expert system*" OR "face recognition" OR "facial recognition" OR fuzzy* OR "human-agent interaction*" OR "human-computer interaction*" OR "human-robot interaction*" OR "robot-human interaction*" OR "human computer interaction*" OR "human machine interaction*" OR "human robot interaction*" OR "image recognition" OR "industry 4.0" OR "industry 5.0" OR "intelligent agent*" OR "internet of thing*" OR IoT OR "language processing" OR "large language model*" OR LLM OR "machine intelligence" OR "machine learning" OR ML OR "natural language*" OR "neural network*" OR neural-network* OR NLP OR "pattern recognition" OR "random forest*" OR "recommendation engine*" OR "remote monitoring" OR "remote sensing" OR robot* OR "smart device*" OR "society 5.0" OR "speech recognition" OR "support vector* machine*" OR SVM OR technolog* OR "text mining" OR "text processing" OR virtual* OR "wearable sensor*" OR "wireless sensor network*"	"action learning" OR "career development" OR "CD" OR "change management" OR coach* OR "corporate social responsibility" OR creativity OR "CSR" OR cultur* OR diversity OR e-hrm OR "employ* experience*" OR "employ* relation*" OR "employee analytic*" OR "employee performance" OR engagement OR "environmental, social, and corporate governance" OR "ESG" OR "future of work*" OR "HR" OR "HR analytic*" OR "HRD" OR "HRM" OR human-capital OR "human capital" OR human-resource* OR "human resource*" OR innovation OR job* OR knowledge* OR leader* OR learn OR "OC" OR "OD" OR "organization* change" OR "organization* development" OR "organization* performance" OR "people analytic*" OR "performance appraisal" OR "performance assessment" OR "professional development" OR retention OR skill OR succession OR "talent analytic*" OR "talent development" OR "talent management" OR "task performance" OR team* OR train* OR turnover OR workforce OR workplace*	“systematic literature review”	
a The Boolean operator AND will be used to combine the three components.

Screening process

We will employ a two-stage screening strategy. First, the relevance of each article will be evaluated based on its title and abstract. Articles that meet the exclusion criteria will be excluded. The second stage will evaluate the relevance of articles based on full texts using the inclusion and exclusion criteria. The screening process will be coordinated using Rayyan.

Eligibility criteria

To uphold consistency and reproducibility in the screening process among coders, the inclusion and exclusion criteria are established. First, eligible studies are systemic literature reviews specifically focused on AI in the field of HRD and related areas. This inclusion criterion aims to contribute to the synthesis of high-quality evidence and insights derived from rigorous research methodologies. The initial search will be confined to peer-reviewed journal articles and conference proceedings written in English and published from 1995 onwards, aligning with the search practices in previous literature reviews on AI (e.g., [5–7]).

Regarding the exclusion criteria, first, studies that do not explicitly explore AI-related technology will be excluded, ensuring a targeted exploration of the subject matter. Second, studies unrelated to a workplace setting will be excluded, as this umbrella review is specifically tailored to the application of AI in the workplace. Third, non-systemic literature reviews, which lack a structured and systematic approach, will also be excluded to maintain the methodological rigor of the review. Fourth, as this umbrella review specifically targets systemic literature reviews, studies employing meta-analysis as the primary research methodology will not be considered for inclusion. Finally, as explained in the inclusion criteria, book chapters and non-referred articles will be excluded to maintain the scholarly standard and reliability of the information under consideration.

Data extraction

We will use Rayyan to extract data. Extraction fields will be set up with the relevant information from the studies, and Rayyan’s tagging and coding features will be used to categorize and organize the extracted data. Disagreements will be discussed and resolved using Rayyan’s conflict resolution feature. The extraction fields for recording the finally selected systematic review studies will include:

Review article information Full study citation

The number of citations

Title, abstract, and keywords

Publication outlet (e.g., journal) and year

Details of the search strategy used in the study Database, journal types, research context, scope

Timeframe

Search terms and string(s)

Scope of AI-related technologies (e.g., AI, machine learning, large language model)

Scope of HRD-related functions (e.g., training & development, organizational development)

Details of the data analysis strategy in the study Analysis approaches (e.g., bibliometrics, contents analysis, topic modeling, clustering)

HRD-related findings in the study HRD-related areas in which AI applies to

The benefits and possibility of AI adoption in HRD functions

The enablers and obstacles of AI adoption in HRD functions

Contributing factors to the effectiveness of AI-based HRD practices

Other key contents/findings of the study (e.g., Future research directions)

Data synthesis

Thematic coding will be a crucial part of this umbrella review, focusing on discerning patterns in the implementation of AI within HRD. By employing an HRD framework, the goal of the thematic coding is to systematically categorize and analyze relevant literature to identify recurrent themes and trends in AI adoption across various HRD contexts. Furthermore, thematic coding facilitates the identification of key opportunities and challenges associated with AI implementation in HRD. The synthesis can highlight common issues faced by organizations integrating AI into HRD practices and, conversely, showcase successful strategies and innovative approaches. Ultimately, the thematic coding approach provides a comprehensive understanding of the current state of AI in HRD and sets the stage for suggesting future research directions and practical recommendations to enhance AI-driven HRD initiatives.

In addition to thematic coding, the data synthesis plan incorporates descriptive statistics. Descriptive statistics involves quantifying the occurrence of specific themes or concepts related to AI implementation in HRD across the selected systematic literature reviews. Specifically, frequency analysis helps to identify the prevalence of certain trends, challenges, or opportunities and visualization techniques can be employed to present these findings in a clear and accessible manner. R will be utilized for statistical analysis and visualization. We plan to use the base package [14] for statistical analysis and ggplot2 [15] for visualization.

Conclusions

This protocol will guide an umbrella review process to synthesize existing systematic reviews on AI in HRD. This umbrella review aims to explore the intersection of AI and HRD using existing reviews in the field of HRD and related areas. The anticipated outcomes of this umbrella review are intended to unveil patterns, opportunities, and threats of AI implementation in HRD. They will provide insights into AI-driven HRD initiatives. All data and analyses will be placed in an open-access repository, and the URL will be provided in the final manuscript.

Despite the expected contributions of this project, several limitations should be discussed. First, the protocol’s reliance on systematic literature reviews may introduce a potential bias, as certain valuable perspectives from non-systematic reviews or other types of reviews may be overlooked. Second, the scope of the review is contingent upon the availability of relevant literature published in English from 1995 onwards; this temporal and linguistic restriction may exclude valuable insights from non-English publications or earlier works that could contribute to a more nuanced understanding of the historical development of AI in HRD. Lastly, it should be mentioned that as AI-related technology is evolving rapidly future updates to this umbrella review will be necessary to ensure that it includes the most updated trends and practices.

Supporting information

S1 File PRISMA-P checklist (https://osf.io/2935t).

(DOCX)

S2 File Search term identification strategy (https://osf.io/vgck2).

(DOCX)

S3 File VosViewer keywords and search terms matching (https://osf.io/nxc7v).

*Note: All supplementary files are available in an open-access repository: https://osf.io/af6d7/.

(XLSX)

10.1371/journal.pone.0310125.r001
Decision Letter 0
Correa Juan Academic Editor
© 2024 Juan Correa
2024
Juan Correa
https://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Submission Version0
9 May 2024

PONE-D-24-05416Artificial intelligence in Human Resource Development:  An umbrella review protocolPLOS ONE

Dear Dr. Yoo,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Jun 23 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Antonio L. García-Izquierdo, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please ensure that you include a title page within your main document. We do appreciate that you have a title page document uploaded as a separate file, however, as per our author guidelines (http://journals.plos.org/plosone/s/submission-guidelines#loc-title-page) we do require this to be part of the manuscript file itself and not uploaded separately.

Could you therefore please include the title page into the beginning of your manuscript file itself, listing all authors and affiliations.

3. Please provide a complete Data Availability Statement in the submission form, ensuring you include all necessary access information or a reason for why you are unable to make your data freely accessible. If your research concerns only data provided within your submission, please write "All data are in the manuscript and/or supporting information files" as your Data Availability Statement.

Additional Editor Comments :

Dear Doctor Yoo,

I finally received the feedback from our reviewers. Their comments pinpoint critical weaknesses that should be addressed. If you can manage the following comments, you can significantly improve the quality of your manuscript. Be aware that my decision should not be assumed as a preliminary acceptance of your research. Given the topic of your work amid the emergence of artificial intelligence, the manuscript has some merits, but the document demands major revisions following suggestions provided as follows:

Reviewer #1

Here are my comments on this manuscript to improve its clarity:

• I recommend citing and reviewing the updated PRISMA document, "The PRISMA 2020 statement: an updated guideline for reporting systematic reviews" published in BMJ 2021; 372 doi: <https: 10.1136="" bmj.n71="" doi.org="">.

• Consider supplementing the database search with AI tools such as Perplexity and Consensus.

• I was unable to access the supplementary files, so I cannot provide comments on them.

• The authors mention that they will use a different program than Covidence for information extraction since it only allows two coders. Please specify which program will be used.

• It is unclear which descriptive statistics will be used and which programming language will be utilized for this purpose. Is R being used?

• Additionally, will inferential statistics be employed in the study?

• Data and all analyses should be placed in a repository and the URL provided in the manuscript.

Reviewer #2

In their work, the authors explored different ways to define research terms based on a initial search with key terms such as artificial intelligence, large language model, machine learning and human resource, from which they extracted more specific terms and also included some trendy terms related with analytics, and after that, the relevance of the resulting terms were validated using the VOSViewer software, to finally propose a search string to developing the review in databases such as Scopus or WoS.

Unfortunately, the proposed protocol describes a search process in which the authors made some decisions that made the protocol very specific, when they included, for instance, terms related with analytics, only because they are trendy.

Another aspect affecting the replicability of the protocol is that the authors do not list, in the references, the 16 studies from which extract AI-related and HDR search terms.

For these reasons, from my point of view, the protocol presented here is not replicable.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions</https:>

Comments to the Author

1. Does the manuscript provide a valid rationale for the proposed study, with clearly identified and justified research questions?

The research question outlined is expected to address a valid academic problem or topic and contribute to the base of knowledge in the field.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Is the protocol technically sound and planned in a manner that will lead to a meaningful outcome and allow testing the stated hypotheses?

The manuscript should describe the methods in sufficient detail to prevent undisclosed flexibility in the experimental procedure or analysis pipeline, including sufficient outcome-neutral conditions (e.g. necessary controls, absence of floor or ceiling effects) to test the proposed hypotheses and a statistical power analysis where applicable. As there may be aspects of the methodology and analysis which can only be refined once the work is undertaken, authors should outline potential assumptions and explicitly describe what aspects of the proposed analyses, if any, are exploratory.

Reviewer #1: Partly

Reviewer #2: Partly

**********

3. Is the methodology feasible and described in sufficient detail to allow the work to be replicable?

Reviewer #1: No

Reviewer #2: No

**********

4. Have the authors described where all data underlying the findings will be made available when the study is complete?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception, at the time of publication. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: No

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above and, if applicable, provide comments about issues authors must address before this protocol can be accepted for publication. You may also include additional comments for the author, including concerns about research or publication ethics.

You may also provide optional suggestions and comments to authors that they might find helpful in planning their study.

(Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Here are my comments on this manuscript to improve its clarity:

• I recommend citing and reviewing the updated PRISMA document, "The PRISMA 2020 statement: an updated guideline for reporting systematic reviews" published in BMJ 2021; 372 doi: <https: 10.1136="" bmj.n71="" doi.org="">.

• Consider supplementing the database search with AI tools such as Perplexity and Consensus.

• I was unable to access the supplementary files, so I cannot provide comments on them.

• The authors mention that they will use a different program than Covidence for information extraction since it only allows two coders. Please specify which program will be used.

• It is unclear which descriptive statistics will be used and which programming language will be utilized for this purpose. Is R being used?

• Additionally, will inferential statistics be employed in the study?

• Data and all analyses should be placed in a repository and the URL provided in the manuscript.

Best regards,

FMR</https:>

Reviewer #2: After reading the manuscript title “Artificial intelligence in Human Resource Development: An umbrella review protocol” in which the authors present a protocol to development a umbrella review about the use of AI in HRD focused only in review articles. In their work, the authors exploring different ways to define research terms based on a initial search with key terms such as artificial intelligence, large language model, machine learning and human resource, from which they extract more specific terms and also include some trendy terms related with analytics, and after that, the relevance of the resulting terms were validated using the VOSViewer software, to finally proposed a search string to developing the review in databases such as Scopus or WoS.

Unfortunately, the proposed protocol describes a search process in which the authors made some decisions that made the protocol very specific, when they included, for instance, terms related with analytics, only because they are trendy.

Another aspect affecting the replicability of the protocol is that the authors do not list, in the references, the 16 studies from which extract AI-related and HDR search terms.

For these reasons, from my point of view, the protocol presented here is not replicable.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Fernando Marmolejo-Ramos

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

10.1371/journal.pone.0310125.r002
Author response to Decision Letter 0
Submission Version1
28 May 2024

We greatly appreciate the valuable comments. They prompted us to revise our manuscript thoroughly, with the hope of enhancing its overall quality. Please see the response table attached. We have provided our point-by-point explanations addressing each comment by specific areas of focus.

Attachment Submitted filename: AI_SLR_Protocol_Response Table_r1_final.docx

10.1371/journal.pone.0310125.r003
Decision Letter 1
Correa Juan Academic Editor
© 2024 Juan Correa
2024
Juan Correa
https://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Submission Version1
4 Jul 2024

PONE-D-24-05416R1Artificial intelligence in Human Resource Development:  An umbrella review protocolPLOS ONE

Dear Dr. Yoo,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Aug 18 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Juan C Correa

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments:

Dear Dr. Yoo,

I have read the most recent version of the manuscript and I think it addresses the reviewers' comments. Based on reviewers' comments and my own reading of your manuscript, I think your paper have shown a reasonable review protocol. There is, however, a final minor detail regarding your manuscript that will be of great value for our readers. In the last version of the manuscript, you mentioned: "R will be utilized for statistical analysis and visualization." This statement is quite generic and provides no clear guidance. Please be specific regarding the libraries or packages you are going to use. For example, if you plan to do some data visualizations, be aware you can use standard libraries such as R base, or more specialized libraries such as "ggplot2" or "ggstatsplot." Likewise, some analyses can take advantage of using network data such as "igraph" or "statnet" (for authors co-citations analysis) and/or textual data by using packages such as "quanteda" or "tidytext." Be aware that some of these data analysis and data visualizations can be easily achieved with "bibliometrix" and its shiny app "biblioshiny." A short statement illustrating these intended resources can be achieved by including one or two sentences with some of these details. Once these inclusions are addressed, the manuscript can be accepted.

[Note: HTML markup is below. Please do not edit.]

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

10.1371/journal.pone.0310125.r004
Author response to Decision Letter 1
Submission Version2
9 Jul 2024

You can also find our responses to the comments in the attached response letter.

Additional Editor Comments: I have read the most recent version of the manuscript and I think it addresses the reviewers' comments. Based on reviewers' comments and my own reading of your manuscript, I think your paper have shown a reasonable review protocol. There is, however, a final minor detail regarding your manuscript that will be of great value for our readers. In the last version of the manuscript, you mentioned: "R will be utilized for statistical analysis and visualization." This statement is quite generic and provides no clear guidance. Please be specific regarding the libraries or packages you are going to use. For example, if you plan to do some data visualizations, be aware you can use standard libraries such as R base, or more specialized libraries such as "ggplot2" or "ggstatsplot." Likewise, some analyses can take advantage of using network data such as "igraph" or "statnet" (for authors co-citations analysis) and/or textual data by using packages such as "quanteda" or "tidytext." Be aware that some of these data analysis and data visualizations can be easily achieved with "bibliometrix" and its shiny app "biblioshiny." A short statement illustrating these intended resources can be achieved by including one or two sentences with some of these details. Once these inclusions are addressed, the manuscript can be accepted.

Responses: Thank you for the suggestion to specify the packages we plan to use in the analysis stage. As this study involves only descriptive statistics and visualizations, we plan to use R base and ggplot2 for this project. We’ve added a sentence to specify the packages on page 10 of the main protocol:

“R will be utilized for statistical analysis and visualization. We plan to use the base package (R Core Team, 2024) for statistical analysis and ggplot2 (Wickham, 2016) for visualization” (p.10).

It should be noted that, as stated in the data synthesis section, we plan to primarily use thematic coding by employing an HRD framework. While we will definitely consider using SNA, text analysis, and bibliometrics for future potential projects, we do not plan to use them in the current project using this protocol.

Journal Requirements: Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Responses: We have reviewed the reference lists in the main protocol and the supplementary file (S2_File.docx) and confirmed that there are no retracted articles. Therefore, no changes are needed to the reference lists.

Attachment Submitted filename: AI_SLR_Protocol_response_r2.docx

10.1371/journal.pone.0310125.r005
Decision Letter 2
Correa Juan Academic Editor
© 2024 Juan Correa
2024
Juan Correa
https://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Submission Version2
26 Aug 2024

Artificial intelligence in Human Resource Development:  An umbrella review protocol

PONE-D-24-05416R2

Dear Dr. Yoo,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Juan Correa

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

10.1371/journal.pone.0310125.r006
Acceptance letter
Correa Juan Academic Editor
© 2024 Juan Correa
2024
Juan Correa
https://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
29 Aug 2024

PONE-D-24-05416R2

PLOS ONE

Dear Dr. Yoo,

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now being handed over to our production team.

At this stage, our production department will prepare your paper for publication. This includes ensuring the following:

* All references, tables, and figures are properly cited

* All relevant supporting information is included in the manuscript submission,

* There are no issues that prevent the paper from being properly typeset

If revisions are needed, the production department will contact you directly to resolve them. If no revisions are needed, you will receive an email when the publication date has been set. At this time, we do not offer pre-publication proofs to authors during production of the accepted work. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few weeks to review your paper and let you know the next and final steps.

Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

If we can help with anything else, please email us at customercare@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Juan Correa

Academic Editor

PLOS ONE
==== Refs
References

1 Chen L. , Chen P. , & Lin Z. (2020). Artificial intelligence in education: A review. IEEE Access, 8 , 75264–75278. doi: 10.1109/ACCESS.2020.2988510
2 Raisch S. , & Krakowski S. (2021). Artificial intelligence and management: The automation—augmentation paradox. Academy of Management Review, 46 (1 ), 192–210. doi: 10.5465/amr.2018.0072
3 Haan, K. (2023, April 25). Top AI statistics and trends in 2023. Forbes Advisor. https://www.forbes.com/advisor/business/ai-statistics/.
4 Chui, M., Yee, L., Hall, B., & Singla, A. (2023, August 1). The state of AI in 2023: Generative AI’s breakout year. QuantumBlack, AI by McKinsey. https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai-in-2023-generative-ais-breakout-year#/.
5 Pereira V. , Hadjielias E. , Christofi M. , & Vrontis D. (2023). A systematic literature review on the impact of artificial intelligence on workplace outcomes: A multi-process perspective. Human Resource Management Review, 33 (1 ), 100857. doi: 10.1016/j.hrmr.2021.100857
6 Bhatt P. , & Muduli A. (2022). Artificial intelligence in learning and development: A systematic literature review. European Journal of Training and Development, 47 (7/8 ), 677–694. doi: 10.1108/EJTD-09-2021-0143
7 Glikson E. , & Woolley A. W. (2020). Human trust in artificial intelligence: Review of empirical research. Academy of Management Annals, 14 (2 ), 627–660. doi: 10.5465/annals.2018.0057
8 Aromataris E. , Fernandez R. , Godfrey C. M. , Holly C. , Khalil H. , & Tungpunkom P. (2015). Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach. JBI Evidence Implementation, 13 (3 ), 132–140. doi: 10.1097/XEB.0000000000000055 26360830
9 Faulkner G. , Fagan M. J. , & Lee J. (2022). Umbrella reviews (systematic review of reviews). International Review of Sport and Exercise Psychology, 15 (1 ), 73–90. doi: 10.1080/1750984X.2021.1934888
10 Moher D. , Shamseer L. , Clarke M. , Ghersi D. , Liberati A. , Petticrew M. , et al . (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews, 4 , 1–9. doi: 10.1186/2046-4053-4-1 25554246
11 Shamseer L. , Moher D. , Clarke M. , Ghersi D. , Liberati A. , Petticrew M. , et al . (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ, 349 , g7647. doi: 10.1136/bmj.g7647 25555855
12 Page M. J. , McKenzie J. E. , Bossuyt P. M. , Boutron I. , Hoffmann T. C. , Mulrow C. D. , et al . (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372 , n71. doi: 10.1136/bmj.n71 33782057
13 Kellermeyer L. , Harnke B. , & Knight S. (2018). Covidence and Rayyan. Journal of the Medical Library Association, 106 (4 ), 580–583. doi: 10.5195/jmla.2018.513
14 R Core Team (2024). R: A language and environment for statistical computing. R Foundation for Statistical Computing: Vienna, Austria.
15 Wickham H. (2016). ggplot2: Elegant graphics for data analysis. Springer-Verlag: New York.
