
==== Front
Saf Health Work
Saf Health Work
Safety and Health at Work
2093-7911
2093-7997
Occupational Safety and Health Research Institute

S2093-7911(24)00032-5
10.1016/j.shaw.2024.04.008
Case Report
Management Architecture With Multi-modal Ensemble AI Models for Worker Safety
Lee Dongyeop kepco.dylee@gmail.com
⁎
Lim Daesik
Park Jongseok
Woo Soojeong
Moon Youngho
Jung Aesol
Team of Occupational Safety, Convergence Technology Lab, KEPCO Research Institute, Daejeon, Republic of Korea
⁎ Corresponding author. Team of Occupational Safety, Convergence Technology Lab, KEPCO Research Institute, Daejeon, 34056, Republic of Korea kepco.dylee@gmail.com
04 5 2024
9 2024
04 5 2024
15 3 373378
4 4 2024
23 4 2024
30 4 2024
© 2024 Occupational Safety and Health Research Institute. Published by Elsevier B.V. on behalf of Institute, Occupational Safety and Health Research Institute, Korea Occupational Safety and Health Agency.
2024
Occupational Safety and Health Research Institute
https://creativecommons.org/licenses/by-nc-nd/4.0/ This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
Introduction

Following the Republic of Korea electric power industry site-specific safety management system, this paper proposes a novel safety autonomous platform (SAP) architecture that can automatically and precisely manage on-site safety through ensemble artificial intelligence (AI) models. The ensemble AI model was generated from video information and worker's biometric information as learning data and the estimation results of this model are based on standard operating procedures of the workplace and safety rules.

Methods

The ensemble AI model is designed and implemented by the Hadoop ecosystem with Kafka/NiFi, Spark/Hive, HUE, and ELK (Elasticsearch, Logstash, Kibana).

Results

The functional evaluation shows that the main function of this SAP architecture was operated successfully.

Discussion

The proposed model is confirmed to work well with safety mobility gateways to provide some safety applications.

Keywords

Autonomous platform
Ensemble AI
Power industry
Risk index
Safety management
==== Body
pmc1 Introduction

It is necessary to establish an advanced safety management system to correspond with the Korean Serious Accidents Punishment Act based on ISO45001 at various workplaces related to the Republic of Korea electric power industry [1]. In particular, it is difficult to systematize and manage safety management tasks for each type of work, with the Republic of Korea electric power industry sites having around 293 types of safety-work checklists.

Therefore, following the Republic of Korea electric power industry site-specific safety management system, this paper presents the results of designing, implementing, and testing the main functions of the safety autonomous platform that can collect video information and biometric information (BI) using GIS (geographic information system) to identify the status of safety management at the workplace and estimate the risk index (RI) of workers through ensemble artificial intelligence (AI) models.

1.1 Designing a safety autonomous platform

Fig. 1 shows the proposed Safety Autonomous Platform (SAP) architecture. This platform is largely composed of a safety data management module, an AI-based safety decision main engine, and a GIS-based worker safety management visualization. Video information and biometric information that were collected were acquired with safe data encryption.Fig. 1 Safety autonomous platform (SAP) architecture.

Fig. 1

First, the safety data management module, which is the basis of the platform, was designed as a Hadoop ecosystem. The Hadoop core project consists of HDFS (hadoop distributed file system) for distributed data storage and YARN (yet another resource negotiator) for resource management, along with Spark being used as a sub-project for high-speed data processing based on Hive and in-memory to process distributed data with SQL (structured query language) [2]. It was also grouped with other Hadoop ecosystems using HUE (hadoop user experience) as a web-based user interface to enable the execution of Hive tasks and Spark jobs [3]. NiFi is a framework for monitoring data flows that can be analyzed even if the protocol and data format are different and can control priorities when sending data. The ELK (ElasticSearch, Logstach, and Kibana) stack has the advantage of being able to quickly identify and solve problems caused by aggregating logs from multiple systems and applications, analyzing them, and creating visualizations of applications and infrastructure monitoring [4]. Kafka guarantees load balancing and fault tolerance for system safety and was used to collect real-time data generated by Republic of Korea electric power industry sites and workers.

The AI-based safety management main engine performs the role of estimating the risk index through the estimation result of this ensemble AI model, standard operating procedures (SOPs) at Republic of Korea electric power industry sites, and workers' profiles in conjunction with the safety data management module. The GIS-based worker safety management visualization comprehensively monitors the safety management and safety status of the workplace and workers based on spatial information of the power industry site.

Fig. 2 is a schematic diagram of the interworking process between the safety autonomous platform and the safety mobility gateway (SMG) that is used for safety monitoring in the Republic of Korea electric power industry site. The SMG is on-premise installed with an ensemble AI engine created and distributed through an AI-based safety-estimation main engine, which allows on-site safety managers to easily grasp the safety situation of each type of workplace and respond quickly when a danger warning message occurs.Fig. 2 Interworking with the SAP and the SMG.

Fig. 2

2 Case report

2.1 Methods

2.1.1 Procedure for constructing training dataset

In order to create an ensemble AI model tailored to the workplace and workers in the AI-based safety-estimation main engine, a systematic learning data construction method is required and, for this purpose, data purification and labeling procedures as shown in Fig. 3 were designed.Fig. 3 Data cleaning and labeling procedure for building a training dataset.

Fig. 3

While primarily refining the raw data collected through the Hadoop ecosystem, video information and biometric information, which are mainly unstructured data, are classified and duplicate data is removed. After that, meaningless data is filtered through annotation and crowdsourcing while refining the secondary data. The completed learning dataset goes through a three-step inspection process to meet the quality control criteria. Finally, when the learning dataset is completed, it is used to create ensemble AI models in the AI-based safety estimation main engine. The generated model is distributed and managed as a SMG based on a docker container.

2.1.2 Ensemble AI model generation method

Referring to Fig. 4, the ensemble AI model is created to determine the safety situation of the workplace and workers. The ensemble AI model is a model that learns several individual artificial intelligence models and infers them by combining the results of the models. This model has the advantage of performance distribution of individual models to reduce overfitting and performance improvement when individual models perform poorly.Fig. 4 Generation method of the ensemble AI models.

Fig. 4

In this paper, among the ensemble models, bagging and boosting methods suitable for determining safety situations were used. Since the bagging method has the advantage of being strong in overfit with parallel learning, and boosting has the advantage of high accuracy with sequential learning, we compared the two learning methods based on the situation of the workplace and the worker to selectively apply the better learning method. In addition, a modified soft rule is used to determine the final estimation value of the ensemble model.

The final estimation value obtained by the modified soft voting method of the ensemble AI model is used as a parameter for the safety rule shown in Table 1 along with SOPs and worker profiles at the Republic of Korea electric power industry site. Each parameter has a specific weight ratio defined according to the situation with the total being defined as 1 based on Equation (1).(1) ∑j=1nRatioj=1

Table 1 Parameters for the safety rule

Table 1Sparamj	Worker P
roficiency	Worker E
ducation level	Work E
nvironment	…	etc.	
Sratioj	Sratio1	Sratio2	Sratio3	…	Sration	

The safety rule is defined in Equation (2) and the risk index is estimated by the safety rule parameter.(2) RI=Ri−∑j=1nSparamjSratioj

Here, Ri represents the normalized risk obtained through the risk assessment of the SOPs at the workplace, Sparamj represents the number of safety management parameters that can reduce the risk, and Sratioj represents the weight ratio for the number of safety management parameters.

We generate two types of ensemble AI models as shown in Fig. 5 depending on whether the type of training data is video information or biometric information. In the case of video information, an ensemble AI model is generated by combining the YOLOv5 model [5], and in the case of biometric information, an ensemble AI model is generated by combining the TadGAN model [6], which is known to have excellent abnormality detection performance of time series data.Fig. 5 Applications of ensemble AI models in SMGs.

Fig. 5

2.2 Results

A functional evaluation was performed by implementing the proposed safety autonomous platform and SMG prototype. Fig. 6 shows an example of the test performance evaluation of the ensemble AI model generated with video information among the training datasets. Further research may be able to build a general artificial intelligence learning database by creating and evaluating an ensemble AI model based on video information and biometric information in various Republic of Korea electric power industry field situations. Fig. 7 shows an example of an application service implemented with the proposed safety autonomous platform. According to the SOPs of the power industry site shown on the left, video information and biometric information acquired from the SMG can be monitored in real time.Fig. 6 Performance evaluation of ensemble AI models.

Fig. 6

Fig. 7 Application for the safety autonomous platform.

Fig. 7

3 Discussion

This study proposed a safety autonomous platform architecture that can automatically and accurately manage the safety of the workplace in the Republic of Korea electric power industry site using the ensemble AI models that were generated from video information and workers' biometric information as learning data. The estimation results of the ensemble AI model are based on the SOPs of the workplace using risks and worker profiles as the parameter and based on the safety rules for estimating risk index. The proposed architecture is largely composed of a Hadoop ecosystem-based safety data management module, an AI-based safety-estimation main engine, and a GIS-based worker safety management visualization and it is designed to work with the SMG of the workplace. Through implementation and functional evaluation, it was confirmed that the main functions of the safety autonomous platform operate successfully according to the estimation results of the ensemble AI model and safety rules. The findings of this study can help the Republic of Korea electric power industry to have an advanced safety management system in correspondence with the Korean Serious Accidents Punishment Act.

Funding

This work was supported by Republic of Korea Electric Power Corporation's main research and development project.

[Task name: Development of autonomous safety solution for construction sites based on risk assessment/Task number: R22IA05].

CRediT authorship contribution statement

Dongyeop Lee: Writing – original draft, Validation, Formal analysis, Conceptualization. Daesik Lim: Methodology, Formal analysis, Data curation. Jongseok Park: Software, Resources, Methodology. Soojeong Woo: Validation, Software, Resources. Youngho Moon: Software, Methodology, Data curation. Aesol Jung: Software, Methodology, Formal analysis.

Conflicts of interest

The author declares that there is no conflict of interest.

Acknowledgments

We would like to thank the construction site workers at KEPCO offices for their cooperation in the field verification of the research results.
==== Refs
References

1 Floyd I.I. H.L. Reflections on a 50-year journey in electrical safety: further steps toward risk reduction IEEE Ind Appl Mag 29 2023 59 65
2 Sun X. He Y. Wu D. Huang J.Z. Survey of distributed computing frameworks for supporting big data analysis Big Data Min Anal 6 2023 154 169
3 Asaithambi S.P.R. Venkatraman R. Venkatraman S. A thematic travel recommendation system using an augmented big data analytical model Tech 11 2023 1 28
4 Calderon G. del Campo G. Saavedra E. Santamaría A. Monitoring framework for the performance evaluation of an IoT platform with Elasticsearch and Apache Kafka Inf Syst Front 4 2023 1 17
5 Hussain M. YOLO-v5 variant selection algorithm coupled with representative augmentations for modelling production-based variance in automated lightweight pallet racking inspection Big Data Cogn Comput 7 2023 1 15
6 Lee J. Cho S. Yu D. Comparative study of time series anomaly detection based on generative deep learning models JKDISS 34 2023 377 394 [in Korean]
