==== Front PLoS One PLoS One plos PLOS ONE 1932-6203 Public Library of Science San Francisco, CA USA 10.1371/journal.pone.0288017 PONE-D-22-28806 Research Article Medicine and Health Sciences Medical Conditions Infectious Diseases Viral Diseases Covid 19 Biology and Life Sciences Psychology Emotions Social Sciences Psychology Emotions People and Places Population Groupings Professions Medical Personnel Pharmacists Social Sciences Sociology Communications Social Communication Social Media Twitter Computer and Information Sciences Network Analysis Social Networks Social Media Twitter Social Sciences Sociology Social Networks Social Media Twitter Biology and Life Sciences Psychology Emotions Fear Social Sciences Psychology Emotions Fear Medicine and Health Sciences Epidemiology Pandemics People and Places Geographical Locations Asia Japan Biology and Life Sciences Psychology Emotions Anxiety Social Sciences Psychology Emotions Anxiety COVID-19 and its impact on the national examination for pharmacists in Japan: An SNS text analysis COVID-19 and its impact on the national examination for pharmacists in Japan https://orcid.org/0000-0002-4902-3102 Kitayama Tomoya Conceptualization Data curation Formal analysis Funding acquisition Investigation Methodology Project administration Resources Supervision Validation Visualization Writing – original draft Writing – review & editing * School of Pharmacy and Pharmaceutical Sciences, Mukogawa Women’s University, Nishinomiya, Hyogo, Japan Ptaszynski Michal Editor Kitami Institute of Technology, JAPAN Competing Interests: The authors have declared that no competing interests exist. * E-mail: tomokita@mukogawa-u.ac.jp 30 6 2023 2023 30 6 2023 18 6 e028801718 10 2022 16 6 2023 © 2023 Tomoya Kitayama 2023 Tomoya Kitayama https://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. The COVID-19 pandemic has created an extraordinary situation for undergraduate students. The aim of this study is to evaluate the impact of the COVID-19 pandemic on the national examination for pharmacists in Japan. In this study, we analyzed the content of Twitter to assess the impact of COVID-19 on the national exam, including psychological aspects. Tweets including the words "national examinations" and "pharmacists" were compiled from December 2020 to March 2021. ML-Ask, a python library, was used to evaluate the emotional register of the tweets on the basis of ten elements: Joy, Fondness, Relief, Gloom, Dislike, Anger, Fear, Shame, Excitement, and Surprise. The presence of COVID-19-related terms was clearly visible in tweets about the national examination of pharmacists between December 1st–and 15th, 2020. It was precisely during this period that the government had announced a strategy regarding national examinations, in the light of COVID-19. The analysis found that post December 16th, words associated with negative emotions were mainly related to the examination, but not to COVID-19. As a result of analyzing only infected areas, a relationship between employment and negative feeling was detected. The author(s) received no specific funding for this work. Data AvailabilityPlease obtain the data set (Remote lessons; Face-to-face lessons; National exam; prefecture) provided for the analysis from public repository (https://github.com/tomokitamukogawa/twitterdata). The data set provided is the data necessary for analysis, excluding the parts connected to individuals such as user names and IDs. OutbreaksCOVID-19 Data Availability Please obtain the data set (Remote lessons; Face-to-face lessons; National exam; prefecture) provided for the analysis from public repository (https://github.com/tomokitamukogawa/twitterdata). The data set provided is the data necessary for analysis, excluding the parts connected to individuals such as user names and IDs. ==== Body pmcIntroduction Since 2000, there have been several coronavirus epidemics in the world. For example, there was the Severe Acute Respiratory Syndrome Coronavirus (SARS-CoV) from 2002 to 2003, H1N1 influenza in 2009, and the Middle East Respiratory Syndrome Coronavirus (MERS-CoV) in 2012. The epidemics of these viruses, unlike COVID-19, did not become a pandemic in Japan. COVID-19 was a “novel Coronavirus”, and the first case of COVID-19 was reported in December 2019 from Wuhan City in China [1]. The COVID-19 pandemic went on to affect the entire world, and Japan was no exception. The pandemic not only impacted healthcare workers, but also had a major impact on undergraduate education. Regarding pedagogy, there has been a transition to remote, online education, and its impact has been analyzed. Further development of new educational programs is underway [2–5]. Healthcare undergraduate students are required to take and pass a national examination. However, no detailed study has been done on the impact of COVID-19 on national examinations. In the three years from 2018, the average number of applicants for the national examination for doctors was 9,333. The average number of applicants for the national examination for pharmacists was 15,486, approximately 1.5 times the number of applicants for the national examination for doctors [6]. Comparing the pass rates of these exams, compared to 2018 before COVID-19, in 2020 there was a slight increase in applications for the national examination for doctor (90.1% → 92.1%) and a decrease in applications for the national examination for pharmacists (70.58% → 69.58%). The number of pharmacists (321,982 as of 2020) is almost the same as the number of doctors (339,623 as of 2020), and they are important healthcare workers in Japan. To analyze the impact of COVID-19 on national examinations, this study focused on the national pharmacist examination viewed from the factors of the number of applicants and the pass rate. The purpose of this study is to analyze the impact of COVID-19 on national examinations, including psychological aspects, on a nationwide scale in Japan. It is important to study what kind of impact the global pandemic and accompanying social environmental changes have had on future pharmacists, so as to make sure things are improved in an eventual next time. Several studies reported the psychological stress caused by the pandemic in pharmacy students [7–9]. Japanese pharmacy students also feel stress, and the purpose of this study is to clarify the psychological conditions in various fields by analyzing the national examinations. This may include the suspension and/or rescheduling of planned final year class, adapting to online virtual learning, anxiety about taking national examinations, anxiety about infection, and expectations and worries about becoming a pharmacist. Previous studies analyzing the impact of COVID-19 on education have often used questionnaires. However, limitations of these studies include questionnaire content, scope of data collection, and choice of data collection subjects. In this study, text was automatically collected from Twitter and analyzed. This is because analyzing the collected text can reveal a broader range of emotional changes than using multiple choices. Methods Study sample size The tweets to be analyzed were collected for four months between December 1st, 2020 and March 31st, 2021. The national examination for pharmacists was held on February 20th and 21st, and the results were announced on March 24th, 2021. Tweet data was collected by Python 3.8.5 (https://www.python.org/). The Python-powered libraries used were tweepy, schedule, pytz and pandas. Using Python, the data was generated as a.csv file (S1 Appendix). It is important to note that not all relevant tweets were collected due to Twitter API limitations, internet environment issues, etc. Collecting tweets was done by the Python program, regardless of the intention of the author. Similarly, tweets that failed to be collected were mechanical problems and occurred randomly regardless of the author’s intention. The study database collected 27,494 tweets that included the terms "remote lessons" and "university", and 16,561 tweets containing "face-to-face lessons" and "university", to investigate the impact on university students in general. Additionally, a total of 7,326 tweets contained the terms "national examinations" and "pharmacists". Areas where a state of emergency was declared (based on the content described in the tweet_user_description) were identified from the 7,326 tweets containing "national exam" and "pharmacist". Here, only tweets in which city names or prefecture names were recognized in the tweet_user_description (Table 1) were included. 10.1371/journal.pone.0288017.t001 Table 1 Number of tweets in each prefecture. Prefecture Tweets Prefecture Tweets Prefecture Tweets Aichi* 116 Kagawa 19 Osaka* 178 Akita 22 Kagoshima 13 Saga 8 Aomori 110 Kanagawa* 344 Saitama* 484 Chiba* 111 Kochi 24 Shiga 36 Ehime 77 Kumamoto 79 Shimane 4 Fukui 27 Kyoto* 46 Shizuoka 47 Fukuoka* 85 Mie 29 Tochigi* 47 Fukushima 27 Miyagi 51 Tokushima 46 Gifu* 45 Miyazaki 15 Tokyo* 554 Gunma 37 Nagano 41 Tottori 5 Hiroshima 43 Nagasaki 18 Toyama 102 Hokkaido 108 Nara 23 Wakayama 72 Hyogo* 147 Niigata 41 Yamagata 26 Ibaraki 41 Oita 32 Yamaguchi 68 Ishikawa 38 Okayama 37 Yamanashi 16 Iwate 36 Okinawa 14 unclear 3,737 Total 7,326 *: Areas with a high rate of infection where a state of emergency was declared during the research period. Analysis of tweets The collected tweets were analyzed using KH coder (https://github.com/ko-ichi-h/khcoder) [10–12] and Python 3.8.5. The Python-powered library used was ML-Ask (https://github.com/ikegami-yukino/pymlask) [13]. KH Coder is software for quantitative text analysis or text mining, and was used for cross-tabulation, correspondence and a co-occurrence network. KH Coder is written in Perl and uses ChaSen, MeCab, TermExtract, MySQL, R language, etc., as backends. ChaSen, MeCab and TermExtract are morphological analysis engines, MySQL is a relational database management system, and R language is a programming language for statistics. ML-Ask, a python library, was used for sentiment analysis of tweets. ML-Ask performs morphological analysis of the input text and compares it with the emotion dictionary database, making it possible to classify the text into 10 axes: "Joy", "Fondness", "Relief", "Gloom", "Dislike", "Anger", "Fear", "Shame", "Excitement", and "Surprise". Based on Contextual Valence Shifters, it performs contextual emotion estimation. For example, in the case of the sentence "I can’t say I like it", since like is negated, ML-Ask infers that it is dislike, which is the opposite feeling of like [13]. Data from the analysis results were generated as.csv files using Python (S1 Appendix). Each tweet contained symbols and tag information that has no meaning in Japanese. Therefore, the data was preprocessed to remove symbols and tag information that would cause errors in the analysis. In the trial analysis by KH coder, some words were detected with inappropriate delimiters, so forced extraction words were set (S2 Appendix). The categories used in the cross-tabulation were selected from co-occurrence networks of all related words collected. For certain nouns, paraphrases were also included in the category (Covid, COVID-19, etc.). The categories used in the cross-tabulation were: Coronavirus-related: Corona, Covid, COVID-19, delta, infect, infection spread, infected person, onset, positive, cluster, close contact, fever, isolation, recuperation, spread, emergency, and state of emergency Relief measures-related: additional test, reexamination, another day, and relief measures Exam-related: exam, examination, difficulty, evaluation, score, grading, and self-scoring Friendship-related: friend, close friend and classmate. In the given time period, the number of times each of the above words occurred was counted. Differences in ratios for each period were tested for significance by residual analysis. The association coefficient (Cramer’s V) was calculated as the effect size [14]. The residual analysis was performed by the following formulas: Expectedvalue(Eij)=(∑i=1anij×∑j=1bnij)/∑i=1a∑j=1bnij Residualsvariance(Rij)=(1−∑i=1anij∑i=1a∑j=1bnij)×(1−∑j=1bnij∑i=1a∑j=1bnij) xij=(nij−Eij)/Eij×Rij pvalue=2×(1−(12πe−xij22)) Cramer’s V was calculated by the following formulas: Chisquarevalue(X2)=∑i=1a∑j=1b(nij−Eij)2Eij Cramer’sV=X2/(∑i=1a∑j=1bEij×(numberofcategories–1)) nij indicates the value of each cell. (i;columns, a is the maximum value. j;rows, b is the maximum value.) For the number of categories, we used three in the vertical direction, since the lower number should be used in the calculations. Emotional recognition of texts by ML-Ask was evaluated on the basis of ten emotional elements. Correspondence analysis was performed using emotional elements as external variables to extract words that are highly relevant to each emotion. This study also included a correspondence analysis using the software KH coder, thus obtaining a more objective classification without being biased by the researchers’ perspectives. Correspondence analysis is an analytical method that visualizes a cross-tabulation table, and displays the relationship between the number of times a word appears and a group (emotional element) as a distance. The distance is calculated by dividing the Euclidean distance by the square root of the ratio to the total number of words appearing in each group. Therefore, the whole profile was placed at the origin (0,0), and characteristics were judged by the direction in which the word appeared from that point. Ethical considerations Research using social networking service (SNS) data has increased in recent years, but uniform research ethics guidelines around it are yet to be developed [15]. According to Twitter’s terms of service, when users register, they are required to allow third parties to use the content they post. In fact, since the first post was published when the company was founded in 2006, all public posts are searchable on the Internet. In addition, Twitter revised its developer policy in 2020, clearly stating that it is possible to collect posted content for non-commercial research purposes, and clarifying the rules for using academic data. Therefore, in the case of big data analysis, the data is public information, and it is often argued that ethical review is unnecessary. In recent years, ethical guidelines for research using SNS data have been proposed [16, 17]. Most notably, George Washington University has published research ethics guidelines for using SNS data (George Washington University Libraries, online). The contents of the guidelines are as follows: the rules of the SNS platform must be followed, collection should be limited to public data, consent should be obtained when using data such as direct messages, collection of metadata such as profile information should be kept to a minimum, consent is required when quoting and posting the text of SNS, and user IDs and account names may only be provided when consent has been obtained. In this study, data was managed according to these guidelines. Therefore, the SNS text, user ID, and other identifying data is not described in the text, nor is it provided to third parties. Limitations of this study A limitation of this study was the small number of accounts (3471) in the national examination for pharmacists tweet analysis. This number was approximately 22.4% of applicants for the national examination for pharmacists, and included accounts whose applicants could not be determined. In the analysis comparing areas where the state of emergency was declared and other areas, tweets (51%) whose prefecture could not be identified were excluded from the analysis. Results and discussion Impact of COVID-19 in Japan during the study period As of March 31st, 2021, Japan reported a total of 472,947 Covid-19 infections [18]. A “third” outbreak of infectious disease occurred during the study period (Fig 1). A new variant was detected at an airport on 25 December, and an outbreak was confirmed in an urban area on 30 January [19]. In early December, a shortage of medical workers caused a medical crisis, and several prefectures applied for the dispatch of medical workers to the Japanese Self-Defense Forces. On January 23, the cumulative number of deaths due to COVID-19 exceeded 5,000. The government issued a state of emergency declaration in 11 of the 47 prefectures. A state of emergency was declared in Tokyo, Kanagawa, Saitama, and Chiba from January 7th, 2021 to March 21th, 2021. Similarly, declarations were made in other areas at different times. 10.1371/journal.pone.0288017.g001 Fig 1 Number of newly infected people and impact on the national examination for pharmacists. Regarding vaccines important for infection prevention, an application for approval of Pfizer’s vaccine was submitted to the Ministry of Health, Labor and Welfare on December 18. From February 17th, vaccination was implemented only for medical workers. On the other hand, vaccinations at universities, including students taking the national examination for pharmacists, started on June 21, three months after the examination was conducted. In November, the Ministry of Education, Culture, Sports, Science and Technology issued a notice to take remedial measures, such as conducting make-up exams, for university entrance exams, one of the major exams to be conducted during the study period. In early December 2020, the Minister of Health, Labor and Welfare announced that the government would not implement an additional examination or other relief measures for the national examination for pharmacists to be held on February 20th and 21st for people infected with COVID-19. Therefore, applicants for the national examination for pharmacist took the examination without any relief measures while unvaccinated amid medical shortages. A line graph shows the number of newly infected people. The column graph shows the percentage of tweets containing the object-related phrase in the half-month tweets. The respective numerical values are given in the table in S3 Appendix. The open column shows COVID-19-related tweet percentage, the gray column shows relief measures-related tweet percentage, the dark gray column shows exam-related tweet percentage, and the black column shows friendship-related tweet percentage. The meaning of Remote is a study tweets database that contains the terms "remote lessons" and "university", Face is a study tweets database that contains "face-to-face lessons" and "university", and Exam is a study tweets database that contains "national examinations" and "pharmacists". Impact on the national examination for pharmacists The number of people infected with COVID-19 increased during the first half of December 2020 and peaked in the first half of January (Fig 1). In response to this, tweets related to remote and face-to-face classes maintained a high percentage of new coronavirus-related words, but the decrease in the number of infected people also reduced the appearance rate of new coronavirus-related words (Fig 1, S3 Appendix). Among the tweets about the national pharmacists examination, the number of COVID-19 related words was significantly higher from December 1 to 15. This increase had been declining since late December, despite an increase in the number of infected people. Between December 1st and 15th there was a marked increase in words relating to relief measures in tweets about the national examination for pharmacists. Words associated with relief measures did not increase significantly in all collected tweets during the other time periods. This suggested that coronavirus-related and relief measures-related changes in the national examination are more affected by social conditions than by changes in the number of newly infected people. During this period, the Minister of Health, Labor and Welfare announced that in relation to the national examination for pharmacists, no relief measures such as additional examinations and alternative examination dates would be implemented for infected persons. It is highly probable that this announcement had a great impact on the examinees. For exam-related words, the peak was observed in the first half of December 2020. This suggested that the peak was related to the announcement by the Minister of Health, Labor and Welfare, but was not at a significant level. It is also speculated that the general level of interest in the national examination was high throughout the research period. Finally, changes in the number of words which were friendship-related peaked at different periods for each group of tweets. Analyzing the emotional impact of the national examination for pharmacists on examinees From December 1st to 15th, there were many words related to COVID-19 (Fig 1). "COVID-19" and "infection" were associated with fear and surprise (Fig 2A). “Every day” and “management” related to daily health care and temperature recording were similarly associated with feelings of fear and surprise. The word "exam" itself was found to be weakly associated with fear, but examination-related words were associated with various emotions such as shame, anger, gloom, and fondness. "Family" and "nice project" are related to their dislike feelings, and it was inferred that tweets containing these were in a negative context due to the influence of COVID-19. From December 16th to 31st, there was a notable drop in tweets related to COVID-19 and relief measures. As a result, no words strongly associated with fear were detected (Fig 2B). Feelings to dislike changed from words related to personal life to those related to the examination. Analysis of the tweets in January showed that there was an association between negative emotions and words related to the examination (Fig 3). COVID-19 infections peaked in January 2021, however COVID-19-related words such as "COVID-19" and "infection" were not found to be associated with any emotion since December 16. This data indicates that examinees might have been more influenced by information about the national examination than by the increase in new infections. 10.1371/journal.pone.0288017.g002 Fig 2 Analysis of the emotional impact of the national examination for pharmacists on examinees in December 2020. Correspondence analysis shows the relationship between the examinee’s emotions and the words contained in the tweet. Each figure indicates the results for the following periods: A: 1 to 15 December and B: 16 to 31 December. The annotations for conversion from Japanese are shown below. The meaning of question (1) is a sentence that is used to examination, and question (2) is a phrase used when asking someone a question. 10.1371/journal.pone.0288017.g003 Fig 3 Analysis of the emotional impact of the national examination for pharmacists on examinees in January 2021. Correspondence analysis shows the relationship between the examinee’s emotions and the words contained in the tweet. Each figure indicates the results for the following periods: A: 1 to 15 January and B:16 to 31 January. The annotations for conversion from Japanese are shown below. The meaning of question (1) is a sentence that is used to examination, and question (2) is a phrase used when asking someone a question. In tweets from February 1 to 15, phrases related to the difficulty of the exam, such as "question range" and "easy", were identified with the emotion of anger (Fig 4A). Also, "hard" and "examinee" were related to their fear feelings. Given that this period was right before the national examination, tweets were also detected that people expressed positive feelings that they solved the questions. After February 16th, "hard" and "study" were also detected in association with the emotion of fear (Fig 4B). Words detected in association with feelings of anger were related to mental states such as "anxiety" and "afford". It is suggested that this expresses feelings about one’s own impatience for the national examination. Similarly, phrases related to impatience such as "panic" and "nervousness" were detected as emotions of joy and surprise. It is speculated that many tweets were posted in the context of denying "panic" and "nervousness". On the other hand, it was observed that the examinees felt excitement and surprise when they were able to "answer". These results suggested that they had various feelings about taking the exam itself. 10.1371/journal.pone.0288017.g004 Fig 4 Analysis of the emotional impact of the national examination for pharmacists on examinees in February 2021. Correspondence analysis shows the relationship between the examinee’s emotions and the words contained in the tweet. Each figure indicates the results for the following periods: A: 1 to 15 February, B:16 to 28 February. The annotations for conversion from Japanese are shown below. The meaning of question (1) is a sentence that is used to examination, and question (2) is a phrase used when asking someone a question. The period from March 1st to 15th is the period of waiting for announcement of the results after finishing the self-assessment. "Study" and "worries" were detected in relation to negative emotions such as fear (Fig 5A). It was speculated that this was the emotion associated with the result of self-grading. "Stop" was detected in association with joy and relief, but this word was observed beginning in March. Since "study" was associated with negative emotions, "stop" was probably less relevant to "able to stop studying." The examinees may have been happy that they were able to "stop" what they had been doing other than studying before the national examination. Many words were associated with the feeling of shame. The associated words were "enterprise", "company", "finding employment", "way", "start", etc., and there were many words related to going out from university into life to society. In addition, "anxiety" was also associated with feelings of shame, suggesting that these words were related to "anxiety". After March 16, a similar tendency was observed for "stop", which was associated with feelings of joy and relief (Fig 5B). However, only "anxiety" was detected to have a strong association with feelings of shame. At the announcement of results in late March, it was observed whether the examinee passed or failed, and the various emotions that they produced. 10.1371/journal.pone.0288017.g005 Fig 5 Analysis of the emotional impact of the national examination for pharmacists on examinees in March 2021. Correspondence analysis shows the relationship between the examinee’s emotions and the words contained in the tweet. Each figure indicates the results for the following periods: A: 1 to 15 March, B:16 to 31 March. The annotations for conversion from Japanese are shown below. The meaning of question (1) is a sentence that is used to examination, and question (2) is a phrase used when asking someone a question. Analyzing the emotional impact on examinees in high infection areas Both state of emergency areas and non-state of emergency areas were analyzed throughout the period of the study (Fig 6). The main difference between state of emergency areas and other areas was the words associated with positive emotions of joy and relief, and the words associated with negative emotions of shame and fear. In the state of emergency areas, words such as "stop" and "nervousness" came up in association with positive feeling (Fig 6A). In the other areas, it was shown that "memorization" and "question" were related to feelings of relief, and that "anxiety" was strongly associated with feelings of joy. The word "nervousness" was detected between gloom and joy, and "stop" was unrelated to any emotion and was not detected (Fig 6B). "Stop" was detected in association with emotion in March after the end of the national examination (Fig 5). Further "stop" was detected in March after the end of the national examination in areas where the infection was spreading, so it is possible that they felt relief and joy in stopping the infection control behavior that they had been implementing until the examination. In areas where a state of emergency was declared, words such as "anxiety", "counseling", "employment", and "society" were associated with negative emotions (Fig 6A). "Take the exam" was detected as the word most strongly associated with fear, and "student" and "look" were associated with shame in other areas (Fig 6B). In areas with relatively low infection risk, words related to national exams were associated with negative emotions, but in areas where there was a state of emergency, words related to society were detected. Similarly, internet-related words such as "open", "chat", and "link" were associated with anger in areas where the state of emergency was declared, whereas only “answer” was associated with anger in other areas. It can be inferred that examinees in high infection areas where a state of emergency was declared were anxious not only about the examination, but also about their connection with society. 10.1371/journal.pone.0288017.g006 Fig 6 Analyzing the emotional impact on examinees in high infection areas. Correspondence analysis compares the relationship between the examinee’s emotions and the words contained in the tweets and identifies similarities and differences between high-infection areas and other areas. A) High infection areas were where a state of emergency was declared. B) Areas where no state of emergency was declared. Annotations for conversion from Japanese are shown below. The meaning of question (1) is a sentence that is used to examination, and question (2) is a phrase used when asking someone a question. Conclusions This paper analyzed the impact of COVID-19 on examinees of the national pharmacist examination in Japan. Examinees were found to be very interested in the contents of the examination system associated with the spread of COVID-19 infection. This effect was seen strongly in December 2020, when the exam rules were announced, after which there was a focus on the examination. In the first half of December, COIVD-19-related words were detected in association with fear emotions, but since then mainly exam-related words have been associated with negative emotions. On the other hand, examinees in high infection areas that had a state of emergency were found to be anxious about their connection with society, including "employment". This trend was less pronounced in other areas. The results of this study indicate that due to the impact of the COVID-19 pandemic, attention should be paid to social relevance, including the future of examinees, rather than the examination itself. Examinees experienced their final year with a curriculum that prioritized infection prevention measures. A study at an American university reported that pharmacy students felt that remote lessons hindered the communication and networking skills they needed for their post-graduation careers [7, 20]. They were very worried about not being able to take advanced classes with their classmates in the final year and the incompleteness of the classes. In addition, it is reported that the news of medical facility during the pandemic has shocked students and some students have negative feelings. Such anxieties are likely to make students feel embarrassed and hesitant about working as medical professionals because of COVID-19. Further, leaving the examinees in hesitation can lead to public health losses. Several studies have suggested that students’ anxiety and hesitation about COVID-19 could be improved with accurate knowledge. Rusgis et al. reported the relationship between the hesitation to receive the corona vaccine and the health information sources [21]. Students who were willing to receive the vaccine utilized scientific journals and school curriculum/coursework, and utilized these sources for COVID-19 information. This report suggests that high information literacy among students may reduce anxiety about pandemics. The Ministry of Internal Affairs and Communications conducted a survey of information sources on COVID-19 among six countries (Japan, America, England, France, Germany, South Korea). As a result, Japan tended to be different from other countries [22]. In Japan, the majority of respondents cited commercial broadcasting as their source of information, while in other countries public broadcasting was the most common. The rate of using specialized organizations such as WHO as sources of information was 8.8% in Japan, while the average value in other countries was 28.4%. It is believed that this background influences the anxiety that Japanese students feel about connecting with society. Teaching correct knowledge, including information literacy, is useful in alleviating students’ anxiety [23]. In the future, it will be necessary for pharmacy education in Japan to find ways to address students’ hesitation. Supporting information S1 Appendix Code used in research. (PDF) Click here for additional data file. S2 Appendix Preparation before analysis by KH coder. (PDF) Click here for additional data file. S3 Appendix Cross-tabulation between lesson format or national exams and each category. (PDF) Click here for additional data file. 10.1371/journal.pone.0288017.r001 Decision Letter 0 Ehara Akira Academic Editor © 2023 Akira Ehara 2023 Akira Ehara https://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Submission Version0 1 Feb 2023 PONE-D-22-28806COVID-19 and its impact on the national examination for pharmacists in Japan: An SNS text analysis PLOS ONE Dear Dr. Kitayama, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Mar 18 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Akira Ehara, Ph.D., M.D. Academic Editor PLOS ONE Journal Requirements: ​ When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at  https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and  https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability. "Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized. Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access. We will update your Data Availability statement to reflect the information you provide in your cover letter. Additional Editor Comments: The analysis is based on artificial intelligence, but there is no mention of the analysis method. Presumably, they are investigating the relationship between COVID-19 and emotional expressions in TWITTER's writing to find out which emotion they are close to, but there are no details. It is artificial intelligence, so it is inevitable that it is black boxed to some extent, but please mention its analysis method, even if it is only a few lines. I can read Japanese, but there is no mention of the analysis method on the website of User Local, Inc written in Japanese (https://emotion-ai.userlocal.jp/) . [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes Reviewer #3: No Reviewer #4: Partly ********** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes Reviewer #3: I Don't Know Reviewer #4: Yes ********** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes Reviewer #3: No Reviewer #4: Yes ********** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes Reviewer #3: Yes Reviewer #4: Yes ********** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: -line 67 Twitter is misspelled -double check for periods before quotation marks throughout. There are some grammatical errors. -line 327 needs revision -good figures to explain fear, happiness, anger, and sadness Reviewer #2: The study promoted by this work is really interesting. The analysis odf emotional impact connected to the pandemic lapse time is a fruitful line if research from various point of view. The authors have accurately managed the data following a correct procedure. Anyway in my opinion in getting a more robustness of the results much more must be explained for that concerns the AI technical tools which involved in this study. The tweet statement are a right "vector" by which conducts this analysis but in my opinion some Natural Language Processing methods must be exploited and deeply analyzed. Python is a good basis by which wording NLP approach with a deep sentiment analysis of the archived statements on the tweets. My request is to explain in a detailed way the methods and the related tools were involved in training these precious data in supporting the interesting results. The Journal deserves a rigorous analysis and the topics of this study too. I am ready in receiving a revised version of the paper for reconsidering my decision on Reviewer #3: Major Comments Conclusions and working hypothesis The paper sets itself to study the effect of COVID-19 on pharmacist examinations in Japan, but makes no effort in explaining why this research question is of general scientific or social relevance. It is clear that this can be of importance for the pharmacist community, but how does this inform us about other examinations in other fields? What is the underlying social or psychological process that this particular case is exemplifying? Furthermore, more information about the scale and nature of the examinations (amount of students, social importance, associated psychological or social phenomena with them, etc.) would make the question of general interest. The paper also concludes that ‘COVID-19 did not significantly influence the examinees’ attitude towards the exam’. On the assumption that the statistical analysis is correct, this conclusion needs further motivation. The reader is left without a clear sense on why this conclusion is unexpected or important. Although the paper briefly explains the circumstances (infections, etc.) of Japan at that time, it makes no effort in connecting this with the conclusion. Data Set The database contains 27949+16561+7326=51836 tweets (if my recollection is correct, there is no explicit total mentioned in the paper). Although the sample size might be sufficiently large for an experimental setup, it falls short in the context of an observational setup. In particular, most social media studies from Twitter use at least two orders of magnitude greater data sets (i.e. in the order of millions or hundreds of millions). This in isolation is not sufficient to regard the data set dissatisfying. It could well be the case that the Twitter activity for the research question is not large. But there is no reasonable effort to justify why this data set is fitting for the research question. Some notes on this: Why can we expect that Twitter conversation reflects the actual sentiment of students taking or about to take the exam? What is the total number of students taking the exam, and the total number of Twitter users posting tweets? The paper does not give the number of users, and ~50K tweets can be composed by very few users, not representative of the student population. It is not clear at all why Twitter data is better than questionnaire data, as argued in the paper. In particular, it is not clear how it avoids biases (that are present in questionnaire, or are new), as it is claimed in the paper. Methodology The methodology used is presented in pages 8 and 9. I found the explanation of the methods used very obscure. Proper formulas (rather than Excel formulas) should be used in a paper. Although I am sympathetic towards Excel, programming languages like R or Python are more reliable and powerful. At the core, although some statistical knowledge can be assumed from the reader, much more clarity in explaining the methods would be required. There are also multiple minor points on the methodology that I will list in the following section. Minor Comments This is simply a list of minor comments to the paper, in order of occurrence: Abstract: The terminology “Artificial Intelligence” is too ambiguous, and the methods used for natural language processing are not state of the art (say transformer neural networks), so there is no clear sense why that is the right terminology. The abstract is non-standardly long, and has some irrelevant information. Pg 4. Some of the phrasing is unclear, e.g.: “student hesitation for COVID-19 pandemic has been reported. Pg 4. I mentioned before that the case that Twitter data is more reliable than questionnaires needs to be made more clearly and strongly. Pg 5. Some details of the sample (say the tweet items/keys connected) should go on an appendix (if any), and not in the main body. Pg 5. Typo: ‘Titter’ Pg. 5 Unclear: “tweets were randomly collected by running the code repeatedly in Python’ Pg 6. I mentioned before that the sample size justification needs more development. Pg 6. Did the areas where state of emergency was declared exhibited more Twitter activity? Pg 7. An explanation of KH coder needs to be given. Pg 7. A detailed explanation of which symbols were deleted is not necessary for the main body of the paper. Pg 7. How were the categories for cross-tabulation determined? And why? Pg 9. The explanation of correspondence analysis has the parameter 2 as a distance, but that is somewhat confusing with the chi-squared test mentioned before. In general, as explained in the main comment, the explanation of the methodology is not clear. Pg 12. I suggest illustrating the correspondence pointed at the start of the paragraph in one single plot, and not part of it in Figure 1 and part in Table 2. Pg 13 and 14. It is unclear why to include such a large table, if most of the p-values are below statistical significance. Figures could use higher resolution for readability. Reviewer #4: 1. Introduction I share your passion for advancing a study on COVID-19 and its impact on the national examination for pharmacists in Japan: An SNS text analysis. A comprehensive argument is required to explain how COVID-19 impacted the national examination for pharmacists. The front end of the paper is fragmented and fails to locate a convincing gap in literature. The presentation of existing literature is fragmented and does not allow you to set up a compelling research problem. 2. Literature Review • The study should have a concise literature review on the study variables • A comprehensive literature support is required to justify for the selection of emotional recognition text components used in this study (anger, sadness, fear, like, and happiness) (page 15, lines 121-122). What is the uniqueness of this components? • The study requires a well-structured hypothesis. 3. Methodology • Author(s) should indicate the rationale for the selection of the respondents. Furthermore, the sample size technique used in determining the size of the study was not indicated. Author(s) should indicate the sample size technique employed in determining the sample size of the study. • Author(s) indicated using Artificial Intelligence (AI) to evaluate the emotional register of the tweets on the basis of five components. The justification for the use of Artificial Intelligence (AI) should be provided. • The techniques adopted in handling common method variance in the study was not indicated in the manuscript. Author should indicate whether the issue of method bias was significant or not significant in the study (see e.g., Conway and Lance, 2010 and Podsakoff et al.,2012). Data Analysis • Author(s) indicated applying an emotional recognition test. No justification for the application of the text was offered in the manuscript. Author(s) should justify the application of the text. Similar justification should be provided for the use of KH coder. • Values of the data analysis were not reported. Author(s) should report values obtained during the analysis with appropriate literature support. • A more robust statistical technique should be employed in testing the impact analysis by author(s). 4. Discussion and Practical Implication The result findings are not able to justify the supporting postulations. Author(s) just offered a narrative explanation which make the discussion segment of the paper not convincing. The practical implications of the study should be carefully looked at since the main thrust of the study the COVID-19 and its impact on the national examination for pharmacists was not clearly addressed in the study. ********** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: Yes: Massimiliano Ferrara Reviewer #3: No Reviewer #4: Yes: Frank Nana Kweku Otoo ********** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. 10.1371/journal.pone.0288017.r002 Author response to Decision Letter 0 Submission Version1 1 May 2023 Thank you very much for your kind the constructive comments of the reviewer. I am most grateful for the detailed review. I have made corrections and additions according to the suggestions of the reviewers. The specific key corrections and additions are described on file. Attachment Submitted filename: Reply to the reviewers.docx Click here for additional data file. 10.1371/journal.pone.0288017.r003 Decision Letter 1 Ptaszynski Michal Academic Editor © 2023 Michal Ptaszynski 2023 Michal Ptaszynski https://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Submission Version1 25 May 2023 PONE-D-22-28806R1COVID-19 and its impact on the national examination for pharmacists in Japan: An SNS text analysisPLOS ONE Dear Dr. Kitayama, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Jul 09 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript:A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Michal Ptaszynski, PhD Academic Editor PLOS ONE Journal Requirements: Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #3: (No Response) Reviewer #4: (No Response) ********** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #3: Yes Reviewer #4: Yes ********** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #3: Yes Reviewer #4: Yes ********** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #3: Yes Reviewer #4: Yes ********** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #3: Yes Reviewer #4: Yes ********** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #3: Second Review ‘COVID-19 and its impact on the national examination for pharmacists in Japan: An SNS text analysis’ Brief summary of the paper In this work the authors used a data set obtained from Twitter in order to assess the effect that COVID-19 had on the national examination for pharmacists in Japan. Using natural language processing techniques, in particular sentiment analysis, they concluded: “In the first half of December, COIVD-19 -related words were detected in association with fear emotions, but since then mainly exam-related words have been associated with negative emotions. On the other hand, examinees in high infection areas that had a state of emergency were found to be anxious about their connection with society, including "employment". This trend was less pronounced in other areas” Recommendation The paper is much improved from the first version. Provided some changes are made to it, I do think it is suitable for publication. Side Comments As a general practice, it would for the author to submit a response letter addressing each of my previous comments in particular. If they did, I do not think I received it. Comments As I mentioned, the paper greatly improved in writing clarity and in addressing the limitations that I stated in my first review. In particular, the results are explained much better and the visualizations are compelling (although some color would be appreciated). The main concern that I have is not with the main body of the text, but rather with the motivation in the introduction and the conclusions at the end. On the one hand, the study could very much profit from a better motivation. What are the research questions guiding the investigation? Why are the observations revealed by the analysis important or useful? To be clear, I think that the contribution is valuable, but its motivation is not clearly explained. For example, a natural motivation would be to argue that it is important to study how the medical community, and in particular future practitioners, reacted to a global pandemic, so as to make sure things are improved in an eventual next time. Where students deterred or invited to pursue their careers? Etc. On the other hand, the conclusion is very interesting but narrow and short. This is related with the point about the motivation. As a reader I was left wanting for a more developed account of the points made in the conclusion. For example, I found this very interesting: “The results of this study indicate that due to the impact of the COVID-19 pandemic, attention should be paid to social relevance, including the future of examinees, rather than the examination itself. Such anxieties are likely to make students feel embarrassed and hesitant about working as medical professionals because of COVID-19.” This is interesting because it suggests that the path to a better situation is not focusing so much on covid, but on the student’s emotions. Is there literature suggesting more in this direction? More generally, it would be nice to see a general recommendation about how to respond to these scenarios that builds both on present literature and the particular observations from the analysis presented. Some minor comments are the following: - Some references connecting with the effect of covid on other student populations would be useful. - I am confused why are doctors mentioned if the paper focuses on pharmacists. am not sure the contrast with questionnaires helps your case here. Also, maybe more references would be necessary. - ‘Mechanically’ (line 66) does not seem the right way. Maybe ‘automatically’? - Lines 95-115. Too much detail in the description. It is good to mention the particular libraries used, but possibly not in the main text as it makes it more convoluted and harder to read. - Formulas around line 132: This is much better than before. But what are the i,j? What are a and b? - Limitations Section: Do you know if those who posted where those getting examined? If so, how? Still, this is a very important point. The proportion of users is sufficiently large. Reviewer #4: The manuscript has seen a thorough and in-depth revision to justify publication in its current form ********** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #3: Yes: Ignacio Ojea Quintana Reviewer #4: Yes: frank nana kweku otoo ********** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. 10.1371/journal.pone.0288017.r004 Author response to Decision Letter 1 Submission Version2 8 Jun 2023 Thank you very much for your constructive comments of the reviewer. I am most grateful for the detailed review. I have gone over the comments carefully, and made corrections and additions according to the suggestions of the reviewer. Attachment Submitted filename: Response to Reviewers.docx Click here for additional data file. 10.1371/journal.pone.0288017.r005 Decision Letter 2 Ptaszynski Michal Academic Editor © 2023 Michal Ptaszynski 2023 Michal Ptaszynski https://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Submission Version2 19 Jun 2023 COVID-19 and its impact on the national examination for pharmacists in Japan: An SNS text analysis PONE-D-22-28806R2 Dear Dr. Kitayama, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Michal Ptaszynski, PhD Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #2: All comments have been addressed ********** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #2: Yes ********** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #2: Yes ********** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #2: Yes ********** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #2: Yes ********** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #2: The paper after revision can be accepted for publication. The Authors have addressed successfully all requests as well ********** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #2: Yes: Massimiliano Ferrara ********** 10.1371/journal.pone.0288017.r006 Acceptance letter Ptaszynski Michal Academic Editor © 2023 Michal Ptaszynski 2023 Michal Ptaszynski https://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. 23 Jun 2023 PONE-D-22-28806R2 COVID-19 and its impact on the national examination for pharmacists in Japan: An SNS text analysis Dear Dr. Kitayama: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Michal Ptaszynski Academic Editor PLOS ONE ==== Refs References 1 Wang C , Horby PW , Hayden FG , Gao GF . A novel coronavirus outbreak of global health concern. Lancet. 2020;395 : 470–473. doi: 10.1016/S0140-6736(20)30185-9 31986257 2 Goodlet KJ , Raney E , Buckley K , Afolabi T , Davis L , Fettkether RM , et al . Impact of the COVID-19 pandemic on the emotional intelligence of student pharmacist leaders. Am J Pharm Educ. 2022;86 : 32–36. doi: 10.5688/ajpe8519 34301541 3 Hussain A , Chau HV , Bang H , Meyer L , Islam MA . Performance of pharmacy students in a communications course delivered online during the COVID-19 pandemic. Am J Pharm Educ. 2021;85 : 1066–1074. 4 Ilankoon IMPS , Kisokanth G , Warnakulasuriya SSP . COVID-19: Impact on undergraduate nursing education in Sri Lanka. J Public Health Res 2020;9 : 1916. doi: 10.4081/jphr.2020.1916 33409248 5 Tabatabai S. COVID-19 impact and virtual medical education. J Adv Med Educ Prof. 2020;8 : 140–143. doi: 10.30476/jamp.2020.86070.1213 32802908 6 Ministry of Health, Labour and Welfare; 2023[cited 14 March 2023]. Press release materials [Internet]. Available from https://www.mhlw.go.jp/stf/houdou/index.html. 7 El-Desoky R , De La Cruz A , Thornton JD , Wanat MA , Varkey D . Identifying predictors of generalized anxiety among student pharmacists in response to the COVID-19 pandemic. Curr Pharm Teach Learm. 2023;15 : 34–42, doi: 10.1016/j.cptl.2023.02.005 36898886 8 Fuentes AV , Jacobs RJ , Ip E , Owens RE , Caballero J . Coping, resilience, and emotional well-being in pharmacy students during the COVID-19 pandemic. Ment Health Clin. 2021;11 : 274–278, doi: 10.9740/mhc.2021.09.274 34621602 9 Elbeshbeshy R , Gim S , Quattrocchi E . Impact of COVID-19 on graduating U.S. pharmacy students in the early epicenter of the pandemic in New York City. Explor Res Clin Soc Pharm. 2021;100085. doi: 10.1016/j.rcsop.2021.100085 34723239 10 Higuchi K. A two-step approach to quantitative content analysis: KH coder tutorial using Anne of Green Gables (Part 1). Ritsumeikan Social Science Review. 2016;52 : 77–91. 11 Higuchi K. A two-step approach to quantitative content analysis: KH coder tutorial using Anne of Green Gables (Part 2). Ritsumeikan Social Science Review. 2017;53 : 137–147. 12 Sugimoto A , Roman R , Hori J , Tamura N , Watari S , Makino M . How has the ‘customary nature’ of Japanese fisheries reacted to COVID-19? An interdisciplinary study examining the impacts of the pandemic in 2020. Marin Policy. 2022;138 :105005, doi: 10.1016/j.marpol.2022.105005 35185265 13 Ptaszyski M , Dybala P , RzepkaR , Araki K , Masui F . ML-Ask: Open source affect analysis software for textual input in Japanese. J Open Res Softw. 2017;5 : 16, doi: 10.5334/jors.149 14 Akoglu H. User’s guide to correlation coefficients. Turk J Emerg Med. 2018;18 : 91–93. doi: 10.1016/j.tjem.2018.08.001 30191186 15 Sinnenberg L , Buttenheim AM , Padrez K , Mancheno C , Ungar L , Merchant RM . Twitter as a tool for health research: a systematic review. Am J Public Health. 2017;107 : e1–e8. doi: 10.2105/AJPH.2016.303512 27854532 16 Ahmed W , Bath P , Demartini G . Using Twitter as a data source: An overview of ethical, legal, and methodological challenges. Advances in Research Ethics and Integrity. 2017;2 : 79–107. 17 Williams ML , Burnap P , Sloan L . Towards an ethical framework for publishing Twitter data in social research: Taking into account users’ views, online context and algorithmic estimation. Sociology. 2017;51 : 1149–1168. doi: 10.1177/0038038517708140 29276313 18 Ministry of Health, Labour and Welfare; 2021[cited 16 April 2021]. Trend in the number of newly confirmed cases (daily) [Internet]. Available from https://covid19.mhlw.go.jp/. 19 Japan Broadcasting Corporation; 2023[cited 18 March 2023]. COVID-19 special site [Internet]. Available from https://www3.nhk.or.jp/news/special/coronavirus/. 20 Stone JK , Pate AN . The impact of COVID-19 through the eyes of a fourth-year pharmacy student. Am Pharm Educ. 2020;84 : ajpe8146, doi: 10.5688/ajpe8146 32665721 21 Rusgis MM , Bays JE , Abt RM , Sperry ML . Impact of COVID-19 health information sources on student vaccine hesitancy. Curr Pharm Teach Learn. 2022;14 : 468–475, doi: 10.1016/j.cptl.2022.02.004 35483813 22 Ministry of Internal Affairs and Communications; 2022[cited 8 June 2023]. Awareness survey on false information in Japan and overseas [Internet]. Available from https://www.soumu.go.jp/main_content/000820953.pdf. 23 Kufel WD , Blaine BE , Avery LM . Pharmacy student’s knowledge and confidence of COVID-19 following an interactive didactic class. J Am Coll Clin Pharm. 2022;5 : 1082–1087, doi: 10.1002/jac5.1678 35942356