
==== Front
Data Brief
Data Brief
Data in Brief
2352-3409
Elsevier

S2352-3409(24)00756-X
10.1016/j.dib.2024.110791
110791
Data Article
Video dataset for the detection of safe and unsafe behaviours in workplaces
Önal Oğuzhan oguzhanonall
a
Dandıl Emre emre.dandil@bilecik.edu.tr
emredandil
b⁎
a Department of Electronic and Automation, Vocational School, Bilecik Seyh Edebali University, Bilecik, Türkiye
b Department of Computer Engineering, Faculty of Engineering, Bilecik Seyh Edebali University, Bilecik, Türkiye
⁎ Corresponding author. emre.dandil@bilecik.edu.tremredandil
03 8 2024
10 2024
03 8 2024
56 11079111 6 2024
18 7 2024
29 7 2024
© 2024 The Author(s)
2024
https://creativecommons.org/licenses/by-nc/4.0/ This is an open access article under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).
Real-time detection of safe and unsafe behaviours in production facilities is very important to prevent these behaviours before they occur. In this context, this study presents a high-resolution video-based dataset of safe and unsafe behaviours obtained from a closed production facility for use in occupational accident prevention. The dataset was collected from the security cameras of a production facility operating in an organised industrial zone in Eskişehir, Turkey, in November and December 2022, after obtaining the necessary permissions from company officials and employees. A total of 8 behaviour classes, 4 classes of safe and 4 classes of unsafe behaviours, were identified for the dataset and 691 video clips containing these behaviours were obtained. The video clips created for the dataset are in MP4 format at 1920×1080 pixels and 24 frames per second. In the dataset, the safe behaviour classes are Safe Walkway, Authorized Intervention, Closed Panel Cover and Safe Carrying, while the unsafe behaviour classes are Safe Walkway Violation, Unauthorized Intervention, Opened Panel Cover and Carrying Overload with Forklift.

Keywords

Workplace
Occupational health and safety
Safe and unsafe behaviours
Deep learning
Computer vision
==== Body
pmcSpecifications TableSubject	Computer Vision and Pattern Recognition	
Specific subject area	Safe and unsafe behaviours in workplaces	
Data format	raw	
Type of data	Video (.mp4)	
Data collection	This dataset is prepared as an original video dataset to be used in deep learning models for real-time automatic detection of safe and unsafe behaviours in work environments. The work environment videos were collected between 5 November 2022 and 13 December 2022 from the “Kafaoğlu Metal Plastik Makine San. ve Tic. A.Ş.” plant in Eskişehir, Turkey. The video data was obtained from two different IP cameras over a period of 39 days, totalling approximately 4000 h. The IP camera brand is UNV and the model is IPC2122CR3-PF40-A. The IP cameras have a resolution of 1920 × 1080 and can record in Full HD mp4 format at 24 fps using the H.264 codec. This video set is divided into two subgroups as safe and unsafe behaviour for both understanding the video content and recognising the objects in the video content.	
Data source location	Department of Computer Engineering, Faculty of Engineering, Bilecik Seyh Edebali University, Bilecik, Turkiye	
Data accessibility	Repository name: Mendeley Data [1]
Data identification number: 10.17632/xjmtb22pff.1
Direct URL to data: https://data.mendeley.com/datasets/xjmtb22pff/1	
Related research article	Önal, O., & Dandıl, E. (2024). Unsafe-Net: YOLO v4 and ConvLSTM based computer vision system for real-time detection of unsafe behaviours in workplace. Multimedia Tools and Applications, 1–27. https://doi.org/10.1007/s11042–024–19276–8	

1 Value of the Data

• The dataset presented is useful for training algorithms, including the proposed computer vision-based methods for detecting and interpreting safe and unsafe movements to prevent workplace accidents.

• Researchers working on artificial intelligence, deep learning, computer vision and video understanding to increase worker productivity and prevent industrial accidents in industrial facilities can use this dataset of high-resolution videos to train and test their methods.

• Most occupational health and safety studies are conducted in the construction industry. The complexity of industrial production sites, the difficulty of collecting data in closed and poorly illuminated environments, and the legal barriers to obtaining the necessary permits are seen as solutions.

• In addition to training workers and employees on health and safety issues, this dataset can also be used in visual and audible warning processes during monitoring and control of unsafe and safe behaviours.

2 Background

In irregular and complex workplaces, careless and dangerous behaviour and failure to use appropriate personal protective equipment can lead to occupational accidents. Similarly, when rules and instructions are not followed in work environments, occupational accidents occur as a result of unsafe behaviour [2]. As a result of such incidents, both material and moral damage and loss of performance can occur in workplaces [3]. International Labour Organization (ILO) estimates that approximately 3 million workers die each year from work-related accidents or diseases [4]. It is therefore very important to eliminate, reduce or prevent unsafe behaviours that can cause accidents in the workplaces. This situation requires employers to give priority to workplace safety [5]. Studies and activities to prevent occupational accidents in production workplaces have been carried out for many years [6]. Although these studies have resulted in a significant reduction in workplace accidents, the loss of life and property can still be devastating [7]. To this end, this dataset has been created because there is still a need to identify and classify safe and unsafe behaviours in enclosed workplaces where production takes place.

3 Data Description

The proposed video dataset has been developed for use in the fields of computer science, pattern recognition, deep learning and computer vision, in particular for the detection of dangerous and unsafe behaviours in the workplace, and for the classification and prevention of safe and unsafe behaviours. New methods and approaches developed using the proposed dataset can help to reduce the labour and performance costs caused by workplace accidents. It is envisaged that the proposed dataset will be used for the following activities:I. Monitoring and recording safe and unsafe behaviour in the workplace.

II. Understanding the reasons for dangerous and unsafe behaviour by workers.

III. Learning from unsafe behaviour and reducing the recurrence of workplace accidents.

IV. Predicting dangerous and unsafe behaviour in the workplace.

During the creation of the dataset, 4 different unsafe behaviours were identified in the workplaces, namely Safe Walkway Violation, Unauthorized Intervention, Opened Panel Cover and Carrying Overload with Forklift, which are most common in the working environment, in cooperation with the factory managers and the occupational safety specialist where the videos were collected. In addition, 4 safe behaviours, namely Safe Walkway, Authorized Intervention, Closed Panel Cover and Safe Carrying, were identified for real-time detection, classification and evaluation in computer vision-based systems. In the study, the class number, behaviour class and behaviour type information regarding the safe and unsafe behaviour classes are shown in Table 1.Table 1 Class number, class of behaviour and behaviour type information for dataset.

Table 1Class number	Class of behaviour	Behaviour type	
0	Safe Walkway Violation	Unsafe	
4	Safe Walkway	Safe	
1	Unauthorized Intervention	Unsafe	
5	Authorized Intervention	Safe	
2	Opened Panel Cover	Unsafe	
6	Closed Panel Cover	Safe	
3	Carrying Overload with Forklift	Unsafe	
7	Safe Carrying	Safe	

During the creation of the dataset, 39 days of surveillance video were collected from the companyʼs cameras to identify the safe and unsafe behaviours of workers and employees. After the videos were collected, the frames with safe and unsafe behaviours were identified by experts and video clips of 1–20 s were created. For the dataset, the number of training and test sets, consisting of video clips of the classes Safe Walkway Violation, Unauthorized Intervention, Opened Panel Cover, Carrying Overload with Forklift, Safe Walkway, Authorized Intervention, Closed Panel Cover and Safe Carrying is given in Table 2. In the data set there are 566 video segments for 8 classes in the training set, while 125 video segments were prepared for 8 classes in the test set. The dataset consists of 691 videos in total, including the training and test sets.Table 2 Number of videos in the training and test set in the dataset consisting of video clips.

Table 2Class of behaviour	Number of video clip in training set	Number of video clip in test set	Total	
Safe Walkway Violation	178	32	210	
Safe Walkway	50	25	75	
Unauthorized Intervention	97	11	108	
Authorized Intervention	23	15	38	
Opened Panel Cover	129	13	142	
Closed Panel Cover	19	13	32	
Carrying Overload with Forklift	48	8	56	
Safe Carrying	22	8	30	
Total	566	125	691	

Fig. 1 shows sample frames taken from the full camera angle for the classes identified for the video dataset prepared for the detection of safe and unsafe workplace behaviours. Fig. 1(a) shows a safe behaviour for the Safe Walkway class, which defines walking along the green path in the work environment, while Fig. 1(b) shows an unsafe worker behaviour for the Safe Walkway Violation class, walking outside the green path. Fig. 1(c) shows an authorised intervention on the board by a worker wearing a green vest (Authorized Intervention), while Fig. 1(d) shows an unauthorized intervention on the board by a worker without an intervention vest (Unauthorized Intervention). Similarly, in Fig. 1(e), safe behaviour occurred for the Closed Panel Cover class after an intervention on the panel connected to a machine, whereas in Fig. 1(f), on the contrary, unsafe behaviour occurred for the Opened Panel Cover class after an employee intervention on the panel. Furthermore, in Fig. 1(g) a safe worker behaviour is demonstrated by carrying 2 blocks or less with a forklift (Safe Carrying), whereas in Fig. 1(h) an example of an unsafe worker behaviour occurs by carrying 3 blocks or more with a forklift (Carrying Overload with Forklift).Fig. 1 Example frames of safe and unsafe behaviour video clips in the dataset.

Fig 1

Fig. 2 shows an illustration of the folder organisation of the dataset. The dataset, which consists of a total of 691 video clips, is divided into two subfolders, training and test. Each subfolder contains video clips for a total of 8 classes: Safe Walkway Violation, Unauthorized Intervention, Opened Panel Cover, Carrying Overload with Forklift, Safe Walkway, Authorized Intervention, Closed Panel Cover and Safe Carrying. The length of each video clip varies from 1 to 20 s on average. There may be a single behaviour class in the same video, while some videos consist of more than one behaviour class. The naming prefix for the video clips in the train (tr) and test (te) directories in the dataset is classnumber_trvideonumber for the train directory and classnumber_tevideonumber for the test directory. For example, the name of the first video clip for the Safe Walkway Violation class in the train directory is 0_tr1, while the name of the first video clip for the same class in the test directory is 0_te1. This naming convention was applied in the same way to videos in all classes.Fig. 2 The structure of dataset folder with train and test sets.

Fig 2

Table 3 summarises the total number of video clips, the average video duration in seconds, the longest and shortest clip duration in seconds for the safe and unsafe behaviour classes in the dataset.• For the Safe Walkway Violation class, a total of 210 video clips in mp4 format, including 178 training and 32 test clips, had an average duration of 9.7 s. For this unsafe behaviour class, the longest clip is 17 s and the shortest clip is 4 s.

• For Safe Walkway, a safe behaviour class, the average video duration was 9.2 s and there were 50 video clips for training and 25 video clips for testing. In this class, which has a total of 75 video clips, the longest clip is 20 s and the shortest is 4 s.

• The average video duration for the Unauthorized Intervention class is 10.2 s. A total of 108 video clips of unsafe behaviour were created for this class, 97 training and 11 test. In this class the longest video clip was 20 s and the shortest was 6 s.

• There are a total of 38 video clips in this class, 23 in the training set and 15 in the test set. In the Authorized Intervention class, the longest clip is 15 s, while the shortest video is 4 s.

• The average video duration for the Opened Panel Cover class is 5.8 s. A total of 142 video clips, 129 training and 13 test, were selected for this unsafe behaviour class. It was observed that the longest video clip in this unsafe behaviour class was 12 s, while the shortest video clip was 2 s.

• The average video duration for the Closed Panel Cover class, which is a safe behaviour class in the dataset, is 4.6 s. There are 32 video clips in this class, 19 for the training set and 13 for the test set. The longest video clip in the class has a duration of 10 s, while the shortest video has a duration of 1 second.

• The average video duration for Carrying Overload with Forklift, another unsafe behaviour class, is 6.9 s. In this class, there are 48 training videos and 8 test videos. In this class, which has a total of 56 video clips, the longest clip is 13 s and the shortest is 3 s.

• The average duration for the Safe Carrying is 7.2 s. In this class, there are 22 video clips for the training set and 8 video clips for the test. In this class, which has a total of 30 video clips, the duration of the longest clip is 12 s, while the duration of the shortest video is 2 s.

Table 3 The total number of video clips, the average video duration in seconds, the longest and shortest clip duration in seconds for the safe and unsafe behaviour classes in the dataset.

Table 3Class of behaviour	Number of video clip in total	Average video duration (s)	The longest clip duration (s)	The shortest clip duration (s)	
Safe Walkway Violation	210	9.7	17	4	
Safe Walkway	75	9.2	20	4	
Unauthorized Intervention	108	10.2	20	6	
Authorized Intervention	38	8.6	15	4	
Opened Panel Cover	142	5.8	12	2	
Closed Panel Cover	32	4.6	10	1	
Carrying Overload with Forklift	56	6.9	13	3	
Safe Carrying	30	7.2	12	2	

4 Experimental Design, Materials and Methods

The workplace videos for this dataset were obtained from “Kafaoğlu Metal Plastik Makine San. ve Tic. A.Ş.” located in Eskisehir, Türkiye. The video data was obtained from two different IP cameras (numbered 14 and 9) for 39 days in the form of surveillance videos and the recordings were continuous over the 39 days. Since the same clip contains behaviours for both the safe and unsafe classes, the raw version of the dataset consists of approximately 4000 h of video recordings in total. The brand of the IP cameras from which the surveillance videos were collected is UNV and the model is IPC2122CR3-PF40-A. The video recordings collected from these cameras for the dataset are in MP4 format, have a resolution of 1920×1080 and have a full HD structure at 24 FPS. The video clips also have H.264 codec infrastructure. Simulation of the working environment of the factory and the position and placement angles of the cameras Fig. 3.Fig. 3 Simulation of the working environment of the factory and the position and placement angles of the cameras.

Fig 3

There are 20 security cameras in the factory where the video clips in this dataset prepared for the detection and classification of safe and unsafe behaviours in the workplace are collected, and the videos from these cameras can be stored in the recorder for about 14 days. In the process of preparing the dataset, two of these security cameras were selected and the videos recorded in November-December 2022 were used according to the behaviour classes. Two different forklift ways are monitored by cameras at the site. The cameras are mounted about 4 m above the ground. The working environment in the factory is illuminated by sunlight during the day and special lamps are used for lighting at night. Standard calibration settings were used during camera recording. The placement of Camera 9 and Camera 14 on the factory workplace is shown in Fig. 4(a) and (b) respectively. In addition, Fig. 5(a) and (b) shows the fields of view of Cameras 9 and 14, respectively.Fig. 4 Placement of Camera 9 and Camera 14 at the factory workplace.

Fig 4

Fig. 5 The fields of view of cameras 9 (a) and 14 (b) in workplace of factory.

Fig 5

Videos are recorded on a storage device connected to the cameras. To save storage space and to facilitate playback of the videos, the video recordings were stored in H.264 file format. A central file system with daily backups was used to manage the data. Raw video was collected every 10 days using an external disc, as the system allowed for 14-day backups. Pre-processing steps included normalising video resolutions and filtering out noisy data to prepare the data for analysis. The video clips were then split into small parts according to the 8 classes, with the help of experts from the factory.

The video clips in the dataset were divided into two subgroups, Safe and Unsafe, in order to make sense of the video content, to recognize objects in the video content and to classify safe and unsafe behaviours. The dataset was divided into 8 classes: Safe Walkway Violation, Unauthorized Intervention, Opened Panel Cover, Carrying Overload with Forklift, Safe Walkway, Authorized Intervention, Closed Panel Cover and Safe Carrying. Accordingly, two block moves were classified as safe in the Safe Carrying class, while more than two block moves were classified as unsafe in the Carrying Overload with Forklift class. The videos in these two classes were obtained from camera 14. The reason for this is that forklifts are used in this area of the factory. On the other hand, the footage from camera 9 shows a long pedestrian walkway and a vehicle road. Therefore, walking on this walkway is classified as a Safe Walkway and violating this walkway is classified as a Safe Walkway Violation. There is also a production machine in the field of view of this camera. Here the panel cover connected to this production machine is left open, an Opened Panel Cover is an unsafe behaviour, while a Closed Panel Cover is a safe employee behaviour. Furthermore, the intervention on this production machine by an authorized person with protective equipment is a safe behaviour as Authorized Intervention, while Unauthorized Intervention defines an unsafe behaviour as unauthorized interferences.

Limitations

Although pedestrians were walking on the carriageway in the footpath violation videos, their projections fall on the footpath in two dimensions, which created a constraint in some videos. Therefore, in the dataset, walking very close to the pavement was accepted as a pedestrian crossing. In addition, many unsafe and unsafe behaviours can occur in workplaces, resulting in accidents that cause injury or death. However, in this dataset only 8 classes were created for safe and unsafe behaviours that can occur in workplaces. Therefore, the high number of classes can be considered as another limitation of this dataset. In addition, the dataset was created by collecting data from only 2 cameras in the workplace and with only 39 days. For more comprehensive studies, the fact that more days of video recordings were not collected for more cameras can also be considered as a limitation. Furthermore, in this dataset, behaviours are divided into two categories according to their type: safe and unsafe. The distribution of video clips across the 8 behaviour classes within the dataset shows some imbalance. For example, the Opened Panel Cover class contains 142 videos, while the opposite class, Closed Panel Cover, contains only 32 videos. The primary reason for the imbalance in the dataset is due to practical constraints, as it is more important to detect unsafe behaviour in workplaces. Therefore, a larger number of videos were collected for unsafe types of behaviour, indicating that safe types of behaviour are more likely to be used for validation in the proposed deep learning-based systems.

Ethics Statement

All employees involved in the video clips obtained from the cameras for the dataset have read, accepted and signed the informed consent form that is kept at Department of Computer Engineering, Faculty of Engineering, Bilecik Şeyh Edebali University. In addition, it was approved by the decision of the Ethics Committee of the Rectorate of Bilecik Şeyh Edebali University dated 11/06/2021 and numbered 10 that there is no ethical violation in the conduct of the study.

As part of the study, the necessary permissions were obtained from “Kafaoğlu Metal Plastik Makine San. ve Tic. A.Ş.” in Eskişehir, Türkiye, for the use of video recordings and images obtained from the factory in academic studies. The relevant permission document is kept at Department of Computer Engineering, Faculty of Engineering, Bilecik Şeyh Edebali University, Bilecik, Türkiye.

CRediT authorship contribution statement

Oğuzhan Önal: Data curation, Writing – review & editing, Visualization, Methodology, Investigation, Validation, Resources. Emre Dandıl: Data curation, Writing – original draft, Writing – review & editing, Visualization, Supervision, Project administration, Methodology, Conceptualization, Validation.

Data Availability

Video Dataset for Safe and Unsafe Behaviours (Original data) (Mendeley Data)

Acknowledgements

The authors of this study would like to thank the employees and officials of “Kafaoğlu Metal Plastik Makina Sanayi ve Ticaret A.Ş.” and Erhan Önal, Hatice Önal, Leyla Hazar, Gökhan Yurtcan, Osman Yurtcan and Mesut Yurtcan for their contributions in collecting videos and preparing the dataset. In addition, this study was financially supported by Coordinatorship of Scientific Research Projects of Bilecik Seyh Edebali University with project number 2019-02.BŞEÜ.01-03.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
==== Refs
References

1 Önal O. Dandıl E. Video Dataset for Safe and Unsafe Behaviours 2024 Mendeley Data V1 10.17632/xjmtb22pff.1
https://data.mendeley.com/datasets/xjmtb22pff/1.

2 Önal O. Dandıl E. Unsafe-Net: YOLO v4 and ConvLSTM based computer vision system for real-time detection of unsafe behaviours in workplace Multimedia Tools Appl. 2024 1 27
3 Ding L. Fang W. Luo H. Love P.E. Zhong B. Ouyang X. A deep hybrid learning model to detect unsafe behavior: integrating convolution neural networks and long short-term memory Autom. Constr. 86 2018 118 124
4 ILO. “Nearly 3 million people die of work-related accidents and diseases.” 2023. https://www.ilo.org/resource/news/nearly-3-million-people-die-work-related-accidents-and-diseases accessed 06 June 2024.
5 Bhana R. Mahmoud H. Idrissi M. Smart industrial safety using computer vision 2023 28th International Conference on Automation and Computing (ICAC) 2023 IEEE 1 6
6 Hofmann D.A. Burke M.J. Zohar D. 100 years of occupational safety research: from basic protections and work analysis to a multilevel view of workplace safety and risk J. Appl. Psychol. 102 3 2017 375 28125258
7 Jilcha K. Kitaw D. A literature review on global occupational safety and health practice & accidents severity Int. J. Qual. Res. 10 2 2016 279
