
==== Front
Sci Rep
Sci Rep
Scientific Reports
2045-2322
Nature Publishing Group UK London

72523
10.1038/s41598-024-72523-8
Article
Dense pedestrian face detection in complex environments
Gao Qiang 1
Ding Bingru 1073622817@qq.com

2
Jia Xu 3
Xie Yinghong 2
Han Xiaowei 1
1 https://ror.org/04ddfwm68 grid.412562.6 0000 0001 1897 6763 Institute of Innovation Science and Technology, Shenyang University, Shenyang, 110044 China
2 https://ror.org/04ddfwm68 grid.412562.6 0000 0001 1897 6763 School of Information Engineering, Shenyang University, Shenyang, 110044 China
3 https://ror.org/05ay23762 grid.440819.0 0000 0001 1847 1757 School of Electronics and Information Engineering, Liaoning University of Technology, Jinzhou, 121001 China
13 9 2024
13 9 2024
2024
14 2146029 12 2023
9 9 2024
© The Author(s) 2024
2024
https://creativecommons.org/licenses/by-nc-nd/4.0/ Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
To address the problem of dense crowd face detection in complex environments, this paper proposes a face detection model named Deep and Compact Face Detection (DCFD), which adopts an improved lightweight EfficientNetV2 network to replace the backbone network of RetinaFace. A large kernel attention mechanism is introduced to address the face detection task more accurately. The backbone network, an improved efficient channel attention (ECA) mechanism, is added to further improve the algorithm performance. The feature fusion module is an improved neural architecture search feature pyramid network (NAS-FPN) that significantly improves the face detection accuracy in different scenes. To balance the training process of positive and negative samples, we use the focus loss function to replace the traditional cross-entropy loss function. In different environments, the DCFD algorithm has shown efficient face detection performance. This algorithm provides not only a feasible and effective solution for solving the problem of face detection in dense groups but also an important basis for improving the accuracy of face detection models in practical applications.

Keywords

Dense pedestrian
Face detection
RetinaFace
EfficientNet
NAS-FPN
Subject terms

Mathematics and computing
Computer science
Foundation of Liaoning Educational CommitteeLJKMZ20221827 Liaoning Provincial Science and Technology Plan Project2023JH2/101300205 Shenyang Science and Technology Plan Project23-407-3-33 Applied Basic Research Project of Liaoning Province2022JH2/101300279 Jia Xu issue-copyright-statement© Springer Nature Limited 2024
==== Body
pmcIntroduction

Recently, with the widespread application of artificial intelligence across various fields and the emergence of deep learning, face recognition technology has become seamlessly integrated into products and daily life. In this domain, the face detection module, as a pivotal component, has experienced notable progress and achievements. Face detection serves as a crucial element in the realm of face recognition. Only by simultaneously detecting the face and extracting relevant content can it be applied to meet practical needs. However, face detection continues to encounter significant challenges in complex environments, characterized by factors such as intricate backgrounds, low resolution, invisibility, and insufficient signals—elements that are beyond our control.

In the early stages of face recognition, algorithms relied on modular matching techniques, employing a predefined face template image to compare various locations in the detection image and determine the presence or absence of a face. For example, Rowley et al.1 proposed a neural network-based face detection algorithm that utilized a dataset trained with 20 × 20 pixels as the gradient collector model2. Although this approach achieves high accuracy, it suffers from relatively slow processing speeds. In 1997, Margineantu et al.3 proposed a face recognition algorithm within the framework of AdaBoost, a machine learning method based on probabilistic approximate correct (PAC) learning theory. In 2004, Viola and Jones4 designed a groundbreaking face detection algorithm using simple Haar-like features and a cascading AdaBoost classifier. This method, known as the Viola‒Jones (VJ) framework, represents a major breakthrough in the field of face detection. Although this approach achieved high accuracy, its processing speed was relatively slow.

With the development of artificial intelligence, deep learning technology has been used in human detection. Face detection algorithms in deep learning can be broadly categorized into two categories: one-stage network detection algorithms and two-stage network detection algorithms. The single-stage network detection algorithm is a method that directly employs neural networks for object detection. Representative examples include those based on the YOLO5–7 series (V1–V7), SSD8, MTCNN9, and RetinaFace10, which have achieved relatively balanced performance in terms of speed and accuracy. On the other hand, the two-stage network detection algorithm initially generates candidate regions and subsequently performs object prediction through the shaped neural network. Representative models include the R-CNN11, fast region-based convolutional network (Fast R-CNN)12, Faster R-CNN13, and spatial pyramid pooling network (SPP-Net)14. While these algorithms are characterized by high detection accuracy, they exhibit relatively slow detection because they perform object generation and object prediction in two distinct phases.

In recent years, with the further development of deep learning technology, the efficiency and detection accuracy of various algorithms have been fully balanced, and notable progress has been made through the introduction of new methods. In 2017, Yang Shuo and Luo Ping15 proposed a face detection algorithm based on deep convolutional neural networks, named Faceness-Net. This method initially detects local features of the face, uses multiple classifiers based on deep convolutional networks to score each facial part (nose, eyes, etc.), and subsequently combines these scores to determine the most likely face region. Following this, the CNN is trained to further enhance the effectiveness of detection. In 2020, Niu Zuodong et al.16 improved the RetinaFace algorithm by incorporating an attention mechanism, achieving outstanding performance in the task of mask detection.

In 2021, Li Yanling et al.17 enhanced the O-Net and R-Net modules of the MTCNN, optimizing the image candidate boxes and classification confidence through the Better-NMS algorithm. This addresses the issue of missed detection in candidate boxes when the intersection over union (IoU) value exceeds the preset threshold. In 2022, Yuan Chao et al.18 addressed the accuracy of face detection algorithms for indoor security. They replaced the backbone network of YOLO-v4 with a deep separable residual network and introduced an attention mechanism to adaptively adjust channel features and spatial feature weights. Additionally, in the same year, Bochkovskiy et al. proposed the YOLOv7 model to further increase the accuracy of face detection. Alibaba DAMO Academy has also open-sourced its lightweight face detection model, DamoFD, and the new face recognition framework, TransFace. The DamoFD model, with its lightweight and high-performance characteristics, provides a solution to face detection on mobile devices. Moreover, the TransFace framework19 has further improved the accuracy and robustness of face recognition by introducing deep learning techniques. Although both the accuracy and speed of face detection have improved significantly in recent years, there is still room for improvement. The Better-NMS algorithm notably enhances the accuracy of face detection through optimizing the candidate box selection. However, this advancement may increase the computational complexity, thereby affecting the overall speed of face detection. Moreover, face detection in real-world scenarios may encounter challenges such as occlusion or extreme poses, which can lead to false positives or missed detections by the model. Advanced models such as YOLOv7 may adopt more complex network structures and utilize more parameters. While these strategies can enhance model performance, they also incur additional costs. Therefore, while pursuing high-precision face detection, we must also balance the computational complexity and storage requirements of the model.

To address the above problems, a lightweight and compact method named DCFD is proposed. The DCFD algorithm balances the computational complexity and storage requirements while ensuring fast detection speed and maintaining a high level of detection accuracy. The improved RetinaFace model is used for pedestrian face detection. The EfficientNetV2 network serves as the backbone network, and a large kernel attention mechanism is introduced to better fuse context information and improve the utilization of fused feature information. By reconstructing the information fusion module and using a focus loss function, the detection of small target faces in complex environments has high accuracy. The advantages of the proposed algorithm are as follows:This paper uses the lightweight EfficientNetV2 network and introduces large kernel attention (LKA) to better fuse local context information and consider long-range dependencies so that the model focuses more on facial features.

This paper optimizes the loss function, using the focus loss function instead of the cross-entropy loss function, to balance the training of positive and negative samples and reduce the loss of easy-to-train positive samples.

An improved efficient channel attention mechanism (ECA) is introduced between the backbone network feature extraction network and the feature fusion module to improve the utilization of feature information in the feature fusion module.

This paper reconstructs the feature fusion module via the improved NAS-FPN image pyramid instead of the original feature pyramid network (FPN). This improvement improves the detection accuracy for small faces and enhances the accuracy and robustness of detection in dense crowd environments.

Methods

Model network architecture

The RetinaFace network conducts face detection on pixels of varying sizes in different orientations through self-supervised and jointly supervised multitask learning. The network model comprises four components: the backbone extraction network, FPN, single-stage headless (SSH) feature extraction network, and detection layer (Head). The structural diagram of the network is depicted in Fig. 1.Fig. 1 RetinaFace network structure diagram.

First, the training dataset is fed into the MobileNet0.25 backbone network, and the outputs of the output layer are labeled c2–c5. After the feature maps are extracted, they are fused, and the upsampling (bilinear interpolation) method is used to ensure that the two layers formed are the same size20. The two-layer feature map matrices are subsequently added to obtain the feature pyramid structure. Second, the feature maps of each selected layer are used as input to the context module. Last, after the classification and regression branch are processed, the algorithm obtains the final prediction result. Figure 2 shows the main flow of the RetinaFace algorithm.Fig. 2 Main process of the RetinaFace algorithm.

In this paper, we propose the DCFD model on the basis of the RetinaFace model. Recognizing the limited generalizability and low robustness of RetinaFace, this model incorporates design improvements in six key aspects:Input: the image to be detected is input.

Backbone Network: The improved EfficientNetV221 is utilized as the backbone network, and the large kernel attention is incorporated.

Feature Pyramid: NAS-FPN22 is applied to fuse features from different levels, and feature extraction capabilities are enhanced.

Prediction: Three types of predictions—classification, face regression boxes, and facial key point locations—are conducted.

Decoding: The predictions from the previous step are adjusted through the decoding process, and the position of the facial frame and key points are refined.

NMS: EIOU is implemented as the NMS method to effectively remove highly overlapping prediction boxes.

The DCFD algorithm enhances the backbone network, improving the expression and learning capabilities of the network while reducing the storage space and memory occupation of the model. This optimization is designed to enhance the generalization ability and information-gathering capability of the model for effective context modeling. During the feature pyramid reconstruction process, the network optimizes the search process for vertical field issues, thus enhancing the overall performance of the network. Through the prediction and decoding of images, our algorithm exhibits heightened sensitivity to the accuracy of object bounding boxes, enabling it to better handle inputs of varying sizes and generate more precise evaluation models.

The effective collaboration of the main components in this model achieves accurate face recognition in dense crowds. The network architecture of the DCFD is illustrated in Fig. 3.Fig. 3 DCFD network architecture.

The SSH module comprises three main components: a single 3 × 3 convolution on the left, a 5 × 5 convolution replaced by two 3 × 3 convolutions in the middle, and a 7 × 7 convolution replaced by three 3 × 3 convolutions on the right. The output of the Head layer includes feature maps of sizes 80 × 80, 40 × 40, and 20 × 20. The first feature map is utilized for SoftMax-based binary classification, the second feature map is employed for face box regression, the third feature map is dedicated to facial regression keypoints, and the prior box is adjusted to capture facial keypoint information.

By stacking multiple small convolution kernels, the same receptive field can be equivalently achieved as a large convolution kernel, which can reduce not only the number of network parameters but also the computational complexity to effectively accelerate the training and inference process of the network. In addition, small kernels are more convenient to implement and optimize in hardware. Adopting multiple small convolution kernels means that more nonlinear activation functions are introduced into the network, which helps increase the expressive power of the network. The SSH method further improves the detection performance of small faces by skillfully integrating context information into the feature map. This improved strategy not only makes the network structure more efficient, but also improves the accuracy and reliability of face detection.

Design of the improved EfficientNetV2 module

To address the issue that the original backbone network MobileNet0.25 in RetinaFace cannot simultaneously account for both detection accuracy and speed, this paper employs the enhanced EfficientNetV2 network as the backbone network for RetinaFace. Additionally, the LKA is designed to further increase the detection accuracy and speed of RetinaFace. This addresses the problems of overfitting and false detection that were associated with the original network.

The primary module of EfficientNetV2 is MBConv, as depicted in Fig. 4. In this paper, the convolutional features, after pooling by the MBConv module, are concatenated and then input to the fully connected layer as a whole. The enhanced EfficientNetV2 network can effectively capture the underlying texture information and high-level semantic details of an image, thereby improving its detection capability.Fig. 4 Improved MBConv structure.

Existing networks encounter challenges such as low face extraction accuracy, poor generalization ability, and high time complexity in face detection tasks. To address these challenges, this paper introduces the LKA module into the EfficientNetV2 framework to handle face detection tasks effectively in complex environments. This module effectively combines the strengths of convolutions and transformers, overcoming the limitations of convolutions in handling long-range dependencies and addressing the challenges that transformers face in adapting to local information and channel dimensions.

The LKA mechanism, while maintaining the local feature extraction strength of the CNN, effectively captures information from a larger context within images or video data through its unique large kernel design. This design not only enables LKA to extract local features as efficiently as CNNs do but also overcomes the potential limitations of CNNs in handling long-range dependencies.

Furthermore, the combination of LKA with the flexibility of transformers provides a more efficient solution to sequence modeling. Through their self-attention mechanism, transformers are able to directly identify and focus on the most significant parts of the sequence for the current prediction. On this basis, LKA assigns different attention weights to the information within the large kernel range, achieving more fine-grained feature selection and attention allocation.

When enhancing the original EfficientNetV2, this paper uses the LKA convolution block to replace the traditional convolution block, as depicted in Fig. 5. The objective of this innovative design is to endow the long range of the network with the ability to support and implement a global receptive field, thereby facilitating the comprehensive extraction of coarse-grained global features from images. The introduction of the LKA module further reduces the number of output channels and repeated layers in EfficientNetV2, allowing for the extraction of multidimensional and multiscale fine-grained features from global image features.Fig. 5 LKA convolution block.

Tables 1 and 2 provide details on the original and improved network structures of EfficientNetV2, respectively. In Table 1, Conv 3 × 3 signifies the use of a standard 3 × 3 convolution operation paired with the SiLU activation function and batch normalization regularization. SE represents the self-attention module, and 0.25 is the coefficient of the first fully connected layer in the SE module, which is equivalent to a quarter of the number of channels in the feature matrix of this module. K represents the convolution kernel size, and the coefficient after MBConv is the dilation factor.Table 1 Original network architecture of the EfficientNetV2 Algorithm.

Module number	Module name	Step length	Output channel	Number of repeating layers	
0	Conv3 × 3	2	24	1	
1	MBConv1, K = 3 × 3	1	24	2	
2	MBConv4, K = 3 × 3	2	48	4	
3	MBConv4, K = 3 × 3	2	64	4	
4	MBConv4, K = 3 × 3, SE = 0.25	2	128	6	
5	MBConv6, K = 3 × 3, SE = 0.25	1	160	9	
6	MBConv6, K = 3 × 3,SE = 0.25	2	256	15	

Table 2 Improved network structure of the EfficientNetV2 algorithm.

Module number	Module name	Step length	Output channel	Number of repeating layers	
0	LKA	2	24	1	
1	MBConv1, K = 3 × 3	1	24	2	
2	MBConv4, K = 3 × 3	2	36	4	
3	MBConv4, K = 3 × 3	2	48	4	
4	MBConv4, K = 3 × 3, SE = 0.25	2	92	6	
5	MBConv6, K = 3 × 3, SE = 0.25	1	128	3	
6	MBConv6, K = 3 × 3, SE = 0.25	2	192	5	

Considering the limited and slow expansion of the receptive field in the traditional convolution module, which causes inefficiency in utilizing distant pixels of the image, we introduce LKA at the start of the EfficientNetV2 network. Leveraging the long-range dependence of LKA, the network can acquire global receiver field characteristics, facilitating the extraction of coarse-grained global features from the image and effectively enhancing the accuracy of the network.

To enhance the EfficientNetV2 network, the output channels and the number of repeat layers of each module following the LKA module are condensed. Throughout this process, the network search algorithm is fully utilized to explore the depth of the network. Once the depth is determined, the number of repetitions between the network layers is systematically investigated. The parameter configuration of the network is subsequently further determined by exploring the width of the network.

Improved ECA module

To address the challenges of accurate face detection and slow detection speed in complex environments, this paper explores the introduction of an attention mechanism after the backbone network. The ECA mechanism21 is a technique used to increase the performance of convolutional neural networks; it is known for its plug-and-play advantages and is widely used in deep learning for face detection. However, owing to its limitations in terms of global performance and tendency to overlook spatial information, this paper restructures the ECA mechanism and designs a fusion spatial attention module to enhance feature extraction, particularly for small faces in dense crowds. This approach improves the accuracy and robustness of the model.

The ECA mechanism does not require dimensional reduction and uses one-dimensional convolution to achieve local cross-recurrence, effectively enabling cross-channel interaction and extracting dependencies between channels. The ECA mechanism structure is illustrated in Fig. 6, where the front and back 'C' represent the input and output feature maps, respectively, 'GAP' represents global average pooling, 'α' represents the activation function, and 'k' represents the scope of local cross-channel coverage.Fig. 6 Structure of the ECA mechanism.

The modification to the backbone network of RetinaFace affects the detection accuracy, especially for dense small object detection at low resolution. While retaining the advantages of the one-dimensional convolution of the ECA mechanism, a spatial attention module (SA module) is added to better capture features in different spatial locations of face images. This new addition enhances global information attention and network performance. The improved ECA mechanism is shown in Fig. 7. Through 1D convolution, the feature extraction ability is enhanced without dimensionality reduction, thus improving model performance with minimal parameters and computation. This enhancement achieves adaptability for face detection in complex backgrounds.Fig. 7 Structure of the improved ECA mechanism.

Refactoring the NAS-FPN module

In RetinaFace, a FPN is used to address object changes at different scales. However, the traditional FPN structure extracts feature maps of different scales from different network levels and performs a 1 × 1 convolution on each extracted feature map to reduce the number of convolution kernels. With increasing depth, the resolution of the FPN decreases, which affects the keypoint detection of small target faces. To solve this problem, this paper designs an improved NAS-FPN, which better utilizes the semantic information contained in the fusion feature map and then improves the network accuracy.

NAS23 repeats the training of the FPN24 in the given search space and performs cross-range feature fusion through top-down and bottom-up connections of the feature maps obtained from the original FPN structure. The best accuracy and speed can be measured according to the number of repetitions.

NAS-FPN uses an automatic architecture search network to select a new restructuring scheme for feature maps across five scales, providing greater flexibility in achieving an improved structure. The pyramid network architecture of NAS-FPN is illustrated in Fig. 8, where pink P3–P7 represent the input feature layer, yellow P3–P7 represents the output feature layer, and GP represents the global pooling layer. The semantic information contained in the high-level feature map is computed as a global feature map. 'R–C–B' represents ReLU–Conv–BatchNorm, indicating the sequence of operations involving the ReLU activation function, convolution, batch operation, and batch normalization.Fig. 8 NAS-FPN structure.

The FPN enhances the accuracy of object detection by fusing features across different scales. However, in the case of small face detection, the traditional implementation of the FPN may not effectively extract and fuse relevant information. NAS-FPN uses NAS techniques to explore various cross-scale connections and feature fusion strategies, ultimately discovering the optimal feature fusion pattern tailored specifically for small face detection, significantly improving detection accuracy.

The design of NAS-FPN adopts a modular approach, in which each component of the FPN is considered a searchable module. This approach increases not only the flexibility of the search process but also the search efficiency. Moreover, NAS-FPN supports early exit and anytime-prediction functionalities. When the model has sufficient confidence in detecting a small face in a specific region, it can terminate further processing for that region, thereby effectively conserving computational resources and enhancing detection speed while maintaining detection accuracy.

The traditional FPN structure diagram is shown in Fig. 2. Since the structure of the FPN cannot fully utilize the feature map, the improved NAS-FPN structure is used to improve the network accuracy. The EfficientNetV2 network is used as the backbone network in the DCFD algorithm. To improve the network accuracy without increasing the number of network parameters and calculations, the feature map is recombined and fused with the feature map via the NA-FPN structure. To perceive the feature map more effectively, the feature network is modified, and the improved network structure diagram is shown in Fig. 9. This modification optimizes the feature extraction and fusion mechanism of the network, thus improving the performance of the DCFD algorithm in the object detection task.Fig. 9 Improved network.

Focal loss function

In the dense pedestrian face detection task, owing to the unbalanced distribution of face and nonface samples in the dataset, the traditional cross-entropy loss function performs poorly in the face of this problem. To address this challenge, we introduce the focal loss function, which is unique in its ability to adaptively focus on difficult samples. Higher weights are assigned to those that are easily misclassified, and lower weights are assigned for those that are relatively easy to classify. This mechanism helps the model focus more on key regions, thereby improving the performance of face detection. The cross-entropy loss function used is shown in Eq. (1).1 L=1NΣiLi=1NΣi-[yi∗logpi+1-yi∗log(1-pi)]

where yi is the label value of the i-th sample, which can be 0 or 1, and pi is the probability that the model predicts that the i-th sample is positive with a value of [0, 1]. This loss function is used to predict the difference between the prediction of the current model and the actual label.

If the sample is positive (yi = 1), the following form of Eq. (1) can be obtained:2 CE=-log(pi)

In Eq. (2), when pi is closer to 1, it is better to predict the positive sample.

If the sample is negative (yi = 0), the cross-entropy loss of pi is given by Eq. (3). When pi is closer to 0, it is better to predict the negative sample:3 CE=-log1-pi

To solve the problems of sample imbalance and difficulty in classification, the focal loss function23 is introduced, which adjusts the loss by introducing a focal parameter, and its prediction accuracy Pt is defined as Eq. (4):4 Pt=p,wherey=11-p,otherwise

To solve the sample imbalance problem, this paper introduces the weighting factor α, which ranges from [0, 1]. When yi = 1, training sample i is regarded as a positive sample, and its weight is set to α. However, when yi = 0, training sample i is regarded as a negative sample, and the weight is set to (1 − α). On the basis of this definition of the weighting factor α, a weighted cross-entropy formula is defined to adjust the cross-entropy loss. According to Eq. (2), the focal loss function with weights can be obtained as follows:5 CEpt,αt=-αtlogpt

The weight factor α is introduced to adjust the weight ratio of positive samples to negative samples in the cross-entropy to solve the problem of unbalanced sample data αt. To address the problem of distinguishing "hard" samples, a conditioning factor γ in the range [0, + ∞] is introduced into the cross-entropy loss function. According to Eq. (3), the focal loss function with weights can be obtained as follows:6 CEpt,γ=-(1-pt)γlog(pt)

The regulator γ can be used as follows:When the samples are misclassified and pt is small (representing samples that are difficult to classify), adjusting the parameter (1 − pt)γ is close to 1 and has a minimal effect on the result of the loss function. This means that the model will maintain a large loss in this case, which makes people pay more attention to those samples that are difficult to classify.

The tuning parameter (1 − pt)γ is 0 when sample pt is 1 (samples that are completely correctly classified), which means that the loss of samples that are completely correctly classified is 0, and the model will not pay attention to such samples.

When sample pt is 0.9 (correct samples that are easy to classify) and γ = 2, the parameter (1 − pt)γ is adjusted to 0.01, and the sample loss function is reduced to 1/100 of the original value so that the model pays less attention to such samples that are easy to classify.

When γ = 0, the focal loss function becomes the cross-entropy loss function, which does not consider the case in which the sample is difficult to classify and all samples are subjected to the same weight.

In summary, according to Eqs. (4)–(6), the focus loss function can be obtained as shown in Eq. (7). According to the classification of the samples and the adjustment factor γ, the loss function is dynamically adjusted to better process the samples.7 FLpt=-α(1-pt)γlog(pt)

The above results indicate that the ability of the model to detect small faces has improved. However, in the postprocessing stage, many faces due to too many prior boxes causes oversensitive detection. In some "hard images", thousands of faces may be present, significantly slowing the NMS algorithm. Therefore, to address the above problems, a strategy is adopted in this paper. The cross-entropy loss function is used in the first 145 cycles. This strategy has the advantage of reducing the number of prior boxes, thereby alleviating the burden on the NMS stage.

Experiments and analysis of the results

Dataset

This paper conducted experiments with the widely used WiderFace dataset and LFW dataset. The WiderFace dataset comprises 32,203 images and includes 393,703 accurately labeled face instances. The dataset exhibits cognitive diversity, featuring images from 61 different scenes where faces demonstrate variations in blur degree, expression, illumination, occlusion, and pose, as shown in (a)–(f) in Fig. 10.Fig. 10 WiderFace dataset.

In this work, we systematically labeled, classified, and filtered the WiderFace dataset to create a new subset of WiderFace data, which comprises approximately 29,000 images. For experimentation, the collated dataset was then divided into two subsets. All scenes were randomly sampled, with 70% of the images allocated to the training set for model training and parameter tuning and the remaining 30% of the images designated as the test set for evaluating the performance and generalization ability of the model.

The LFW dataset comprises 13,233 face images, representing 5749 unique identities and reflecting the diversity of faces in terms of age, illumination, and pose. The substantial number of images and individuals in this dataset ensures ample sample diversity. In the example plots (a) to (f) in Fig. 11, typical images from the LFW dataset are displayed.Fig. 11 LFW dataset.

Experimental environment

The algorithm implemented in this paper is based on the Ubuntu 18.04 64-bit operating system. PyTorch is used as the deep learning framework, with CUDA version 11.0.2, and the programming language is Python 3.8. The hardware and software environments for model training are detailed in Table 3, and the stability and reliability of these environments provide a solid foundation for the implementation and performance evaluation of the algorithm in this paper.Table 3 Software and hardware configuration.

Category	Parameter	
Operating system	Ubuntu 18.04	
GPU	NVIDIA T4	
CPU	Intel Core i7-9750H	
CUDA	11.0.2	
Python Framework	Pytorch 1.12	
Python Version	3.8	

Parameter settings and evaluation metrics

In the experiments of this paper, precision and recall serve as the primary evaluation metrics for a comprehensive assessment of model performance. The model was tested by using a dataset comprising 1000 images.

The prediction accuracy refers to the proportion of all samples predicted by the model to be actual faces. The initial calculation can be expressed as Eq. (8), where TP represents the number of positive samples correctly predicted as faces, whereas FP represents the number of negative samples incorrectly predicted as faces.8 Precision=TPTP+FP

Recall refers to the proportion of samples that the model correctly predicts as faces from all the samples that are actual faces, and it can be expressed by Eq. (9). Here, FN represents the number of positive samples incorrectly predicted as nonfaces.9 Recall=TPTP+FN

In the training stage, anchor boxes with EIOU values greater than 0.5 are defined as positive samples, whereas those with EIOU values less than 0.3 are defined as negative samples, and a positive-to-negative sample ratio of 1:3 is maintained after sample screening. The model training data for the DCFD are outlined in Table 4.Table 4 Training parameter configuration.

Categories	Parameters	
Batch size	5	
Image size	640 × 640	
Learning rate	0.001	
Epoch	150	
optimization	SGD	
SGD gamma	0.1	

The feature pyramid, prior box and SHH setting parameters in the model are shown in Table 5:Table 5 Model parameter configuration.

Feature layer	Output channel	Feature map size	Step	Pior box size	Prior box quantity	LeakyReLU	
Stage 1	40	80 × 80	8	[16, 32]	80 × 80 × 2	0.1	
Stage 2	112	40 × 40	16	[64, 128]	40 × 40 × 2	0.1	
Stage 3	160	20 × 20	32	[256, 512]	20 × 20 × 2	0.1	

Ablation experiment

To verify the optimization effect of the DCFD algorithm on the RetinaFace face detection model, this paper designs two sets of ablation experiments on the WiderFace and LFW datasets. These experiments cover the comparisons of four key improvement points: the replacement of the backbone network, the introduction of a focal loss function, an improved ECA mechanism, and the NAS-FPN feature fusion module. By gradually integrating these improved measures and comparing and analyzing the detection results, the superiority of the improved algorithm is confirmed. In Table 6, "√" indicates the application of this improved method in the RetinaFace face detection network, whereas easy, medium, and hard represent the detection accuracies in three different difficulty modes of the dataset.Table 6 Ablation experiment on the WiderFace dataset.

Number	EfficientNet	Focal Loss	ECA	NAS-FPN	Easy (AP)	Medium (AP)	Hard (AP)	
1	–	–	–	–	93.45%	92.86%	83.23%	
2	√	–	–	–	95.06%	94.31%	84.47%	
3	√	√	–	–	95.11%	94.39%	84.69%	
4	√	√	√	–	96.15%	95.57%	85.32%	
5	√	√	√	√	96.64%	96.3%	86.73%	

The data in Table 6 clearly show that on the WiderFace dataset, after replacing the backbone network, the accuracy of the model in the three difficulty modes of easy, medium and hard is improved by 1.61%, 1.45% and 1.24%, respectively. The detection accuracy of the model is subsequently further improved by approximately 1% by introducing the ECA mechanism and the focal loss function. Notably, the addition of the NAS-FPN feature fusion module improves not only the detection accuracy but also the overall network performance. This comprehensive improvement strategy not only ensures the accuracy of face detection but also balances the detection speed, thus comprehensively improving the performance of the RetinaFace face detection model.

The data in Table 7 show that the DCFD algorithm demonstrates comparable effectiveness on the LFW dataset, which is similar to its performance on the WiderFace dataset. Each experimental component has a positive impact on the model and underscores the versatility of the DCFD algorithm across different datasets. The results of this series of ablation experiments further affirm the robustness and superiority of the DCFD algorithm in addressing various datasets and scenarios.Table 7 Ablation experiment on the LFW dataset.

Number	EfficientNet	Focal Loss	ECA	NAS-FPN	Easy (AP)	Medium (AP)	Hard (AP)	
1	–	–	–	–	93.58%	93.02%	83.17%	
2	√	–	–	–	95.26%	94.51%	84.32%	
3	√	√	–	–	95.41%	94.96%	84.76%	
4	√	√	√	–	96.35%	95.23%	85.95%	
5	√	√	√	√	97.04%	96.43%	87.01.%	

In the ablation experiment, to better evaluate the effectiveness of the proposed DCFD algorithm, the contribution of each part is gradually verified, as shown in Fig. 12, to show the effect of each experimental part.Fig. 12 Plot of the results of the ablation experiment.

Comparative experiments

In the reconstructed WiderFace dataset, the model loss starts to converge after 150 training rounds. To further evaluate the performance of the proposed model, 10 different network models were trained in the same experimental environment. A detailed analysis and comparison were conducted, and the experimental results are presented in Table 8. The results clearly indicate that the DCFD model in this paper has significant advantages in terms of accuracy and detection speed. Compared with other network models, the algorithm in this paper performs well in dense crowd face detection tasks, further validating the effectiveness of the improved algorithm.Table 8 Comparative experiment on the WiderFace dataset.

Method	Easy (AP)	Medium (AP)	Hard (AP)	Model size (MB)	
TinaFace	96.07%	95.25%	85.15%	144	
SCRFD-34GF	95.49%	94.93%	84.32%	–	
MTCNN	92.35%	92.1%	82.86%	11.1	
RetinaFace	93.45%	92.86%	83.23%	1.7	
Yolov7-tiny	95.23%	93.21%	81.65%	11.8	
SSD	92.84%	91.62%	93.23%	97	
SSH	93.10%	92.15%	83.12%	–	
FANet	94.53%	94.75%	89.52%	82	
V-J	43.35%	35.91%	14.85%	–	
DPM	70.01%	46.23%	21.45%	–	
Yolov7-face	97.02%	95.12%	85.03%	–	
DCFD (ours)	96.64%	96.3%	86.73%	14.3	

As shown in Table 8, in the three validation subsets of the reconstructed WiderFace dataset, the average accuracy of the DCFD model reaches 96.64%, 96.3%, and 86.73%; these values are 3.19%, 3.44%, and 3.50% higher, respectively, than those of RetinaFace. In contrast, traditional detection methods that do not use convolutional neural networks, such as V‒J and DPM, perform poorly in terms of accuracy. The average accuracy of Faceness-Net is also lower because of its lack of multiscale feature extraction methods, making it challenging to adapt to different face sizes. Moreover, the accuracies of the MTCNN and SCRFD-34GF deep network models, which lack an attention mechanism, are relatively low.

To ensure the complete consistency of the experimental environment, this paper conducts 150 rounds of training on the LFW dataset. As shown in Table 9, in the LFW dataset, the average accuracy of the DCFD model reached 97.04%, 96.43% and 87.01%; the size of the model was 15.3 MB, achieving a balance between detection accuracy and detection speed; and the overall model was more lightweight. These values are 3.59%, 3.57% and 3.78% higher than those of RetinaFace. Newer networks such as TinaFace and YOLOv7-tiny have significant advantages over YOLOv7-face at both the medium and hard difficulty levels. Owing to the change in the backbone network and the addition of an attention mechanism, the DCFD algorithm has better performance. Through comparative experiments, we further confirm the robustness and excellent performance of the DCFD algorithm on different datasets. According to the data analysis in Table 9, the experimental results align with the conclusions obtained for the WiderFace dataset, confirming the strong performance of the DCFD model on the LFW dataset.Table 9 Comparative experiment on the LFW dataset.

Method	Easy (AP)	Medium (AP)	Hard (AP)	Model size (MB)	
TinaFace	96.07%	95.25%	85.15%	168	
SCRFD-34GF	95.49%	94.93%	84.32%	–	
MTCNN	92.35%	92.1%	82.86%	12.3	
RetinaFace	93.45%	92.86%	83.23%	1.9	
Yolov7-tiny	95.23%	93.21%	81.65%	12.6	
SSD	92.84%	91.62%	93.23%	103	
SSH	93.10%	92.15%	83.12%	–	
FANet	94.53%	94.75%	89.52%	–	
V-J	43.35%	35.91%	14.85%	–	
DPM	70.01%	46.23%	21.45%	–	
Yolov7-face	97.87%	96.02%	85.93%	–	
DCFD (ours)	97.04%	96.43%	87.01%	15.3	

The face detection curves for different difficulty levels (easy, medium, and hard) are presented in Fig. 13. Compared with other network models, the DCFD algorithm has higher prediction accuracy and an excellent recognition rate. The proposed algorithm achieves significant advantages in face detection tasks with varying difficulty levels.Fig. 13 Model precision‒recall curves.

To design a face detection algorithm that is suitable for various environments, this paper strives to maintain high detection accuracy while reducing the number of parameters and computational complexity. To verify the effectiveness of MobileNetV2 as the backbone network in improving accuracy, RetinaFace-ResNet50 and RetinaFace-MobileNetV1 are selected as the comparison algorithms. All algorithms have been strictly tested on the WiderFace dataset to ensure the fairness and reliability of the results. The experimental results show that the algorithm proposed in this article has significant advantages over the other algorithms. The detailed test and comparison results are shown in Table 10.Table 10 Backbone network comparison test.

Backbone	Easy (AP)	Medium (AP)	Hard (AP)	
RetinaFace-Res	94.92%	91.26%	71.05%	
RetinaFace-V1	89.76%	86.96%	75.05%	
RetinaFace-V2	95.01%	92.57%	82.31%	
RetinaFace-V2 + LKA	95.06%	94.31%	84.47%	

According to Table 10, the algorithm proposed in this work significantly outperforms MobileNetV1 in terms of the easy, medium, and hard difficulty levels, with increases of 5.3%, 7.35%, and 13.42%, respectively. Moreover, compared with ResNet50, the proposed algorithm also demonstrates good competitiveness. The optimization of the backbone network in this work not only improves the overall accuracy of the network but also shows a distinct advantage over other networks.

As shown in Table 11, after replacing the cross-entropy loss function originally used by the RetinaFace network model with the focal loss function, the model achieves the best performance at the hard difficulty level. This improvement not only enables the model to better address facial conditions in different scenarios but also significantly improves the accuracy of face detection. The optimization of the loss function can significantly improve the performance of the network model in addressing complex face detection tasks.Table 11 Comparative tests of loss functions.

Loss function	Easy (AP)	Medium (AP)	Hard (AP)	
Cross-entropy loss function	93.45%	92.86%	83.23%	
Focal loss function	93.5%	92.95%	83.45%	

The experimental results presented in Table 12 indicate that the accuracy generally improves after the ECA mechanism is introduced. Furthermore, when the ECA mechanism is improved and integrated into the SA module, the accuracy rate achieves a significant increase of more than 1.0% in the difficulty levels of easy, medium and hard. Compared with the original ECA mechanism, this improvement not only yields greater performance improvement but also fully verifies the positive promotion effect of the attention mechanism improvement scheme proposed in this paper on the overall performance of the network model.Table 12 Comparison tests of attention mechanisms.

Attention mechanism	Easy (AP)	Medium (AP)	Hard (AP)	
None	93.45%	92.86%	83.23%	
ECA	94.02%	93.51%	83.45%	
ECA + SA	94.49%	94.04%	83.86%	

The detection effect of the original RetinaFace algorithm is illustrated in a(1)–a(3) of Fig. 14, whereas the detection effect of the DCFD algorithm is shown in b(1)–b(3) of Fig. 14. These visualizations vividly demonstrate the improvement achieved by the DCFD model over the original RetinaFace. In a(1) and b(1), the recognition ability of RetinaFace is poor on face images with a large occlusion area in the last row on the right, whereas DCFD performs better on such images. In a(2) and b(2), RetinaFace has difficulty recognizing faces at a long distance, whereas DCFD can recognize almost all faces at this distance. In a(3) and b(3), RetinaFace performs poorly in face recognition at long distances, whereas DCFD can successfully recognize 80% of faces at a long distance.Fig. 14 Comparison of the detection results before and after improvement for the RetinaFace Network.

Experimental analysis

Through a series of experiments on the LFW and WiderFace datasets, the backbone network, attention mechanism, loss function and feature fusion module are carefully optimized. To verify the effectiveness of these improvements, not only are the performances of the classical network and the newer network compared but also detailed module ablation experiments are carried out. The experimental results show that the improvement of each module has a positive effect on the accuracy and running speed of the network model.

For face detection tasks, the EfficientNetV2 model is improved, and LKA is introduced to make the model more focused when facial feature expression is captured. Through this improvement, the model can more effectively capture the underlying texture information and high-level semantic information of the image to achieve significant compression of the output channels and the number of repeated layers while ensuring the accuracy of the network. The experimental results on the LFW and WiderFace datasets show that the improved EfficientNetV2 model with LKA achieves significant performance improvement on face detection tasks. By effectively controlling the complexity of the model, the running speed of the model is improved, and it is more competitive in practical applications.

To further improve the performance of the model, an improved ECA mechanism is introduced after the backbone network, which effectively avoids the dimensionality reduction process so that the model can extract the relationships between channels more accurately, thus significantly improving the performance of the model while maintaining the minimum number of parameters and calculations. Moreover, the utilization of backbone network features is enhanced, and the ability of the model to address small objects is subsequently improved. This not only optimizes the ability of the model to capture subtle facial features in the image but also strengthens the ability of the model to recognize faces in complex backgrounds.

In view of the limitations of the traditional cross-entropy loss function in face detection tasks, this paper improves upon the traditional cross-entropy loss function. The traditional cross-entropy loss function often performs poorly when it is difficult to obtain a single sample, especially for tasks such as face detection, and its limitations are particularly prominent. The focal loss function can more effectively detect facial regions that were difficult to detect before, especially for those small target faces that are easy to miss and show better detection performance. In addition, the loss function also reduces the loss of correct samples that are easy to train by balancing the training samples of faces and nonfaces, which further improves the generalization ability and robustness of the model. After the focal loss function is used on different datasets, the performance of the model on the face detection task is significantly improved.

In this paper, NAS-FPN is selected as the feature fusion module. When the depth of the network increases, the resolution of the traditional FPN decreases, which slightly affects the keypoint detection performance of small target faces. To solve this problem, the NAS-FPN module is introduced. NAS-FPN can dynamically adjust the combination of feature layers to adapt to the needs of object detection at different scales. This cross-scale feature combination strategy enables the model to fuse the feature information of different levels more effectively, thereby improving the accuracy of face detection.

In the experiments, the NAS-FPN module shows good adaptability for diverse scenarios. For both large target faces and small target faces, the module is able to optimize the detection effect by dynamically adjusting the feature layer. In addition, NAS-FPN allows the network to adapt better to various environmental changes and improves the robustness and generalizability of the model.

Conclusions

To address the challenges of RetinaFace in face detection in complex environments, especially in dense crowds, we made a series of key improvements that led to significant performance gains.

The enhanced lightweight EfficientNetV2 network serves as the backbone for RetinaFace, which uses a large kernel attention mechanism to extract facial features more effectively. To better concentrate on spatial information expression and capture facial spatial relationships in dense crowds, a spatial attention module is introduced into the ECA mechanism. To achieve finer model adjustments, the model now directly addresses pixel-level predictions, enhancing accuracy in both the prediction of the model structure and its surroundings. The reconstruction of the spatial fusion module enables adaptive gain for image targets, reduces model parameters, improves computational efficiency, and offers a more robust solution to practical application scenarios in subsequent face detection problems.

In summary, through experiments on different datasets, the DCFD algorithm has significantly improved detection performance and accuracy, particularly in dense crowd face detection tasks. The DCFD algorithm presents an effective solution to face detection challenges in practical application scenarios.

Acknowledgements

This research was supported by the Foundation of Liaoning Educational Committee under grant No. LJKMZ20221827. This work was supported by the Liaoning Provincial Science and Technology Plan Project 2023JH2/101300205, the Applied Basic Research Project of Liaoning Province 2022JH2/101300279 and the Shenyang Science and Technology Plan Project 23-407-3-33.

Author contributions

B.R.D. conceived the experiment and wrote the manuscript with the help of Q.G., X.J., Y.H.X., and X.W.H. B.R.D. conducted the experiment. All the authors helped oversee the project, explore improvements, discuss the results, and write the article.

Data availability

The WiderFace dataset in this paper can be accessed from http://shuoyang1213.me/WIDERFACE/, and the LFW dataset can be accessed at https://vis-www.cs.umass.edu/lfw/.

Competing interests

The authors declare no competing interests.

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
==== Refs
References

1. Liqiao HU Runhe QIU A face recognition algorithm for adaptive weighted HOG features Comput. Eng. Appl. 2017 53 03 164 168
Liqiao, H. U. & Runhe, Q. I. U. A face recognition algorithm for adaptive weighted HOG features. Comput. Eng. Appl. 53(03), 164–168 (2017).
2. Shen, X., Lin, Z., Brandt, J. et al. Detecting and aligning faces by image retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 3460–3467 (2013).
3. Xie M Qinghua Su Wang L The research of traffic cones detection based on Haar-like features and adaboost classification IET Conf. Proc. 2022 2022 18 437 442
Xie, M. et al. The research of traffic cones detection based on Haar-like features and adaboost classification. IET Conf. Proc. 2022(18), 437–442 (2022).
4. Viola P Jones MJ Robust real-time face detection Int. J. Comput. Vis. 2004 57 2 137 154 10.1023/B:VISI.0000013087.49260.fb
Viola, P. & Jones, M. J. Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004).10.1023/B:VISI.0000013087.49260.fb
5. Li, C., Wang, R., Li, J. et al. Face detection based on YOLOv3. In Recent Trends in Intelligent Computing, Communication and Devices 277–284 (Springer, 2020).
6. Yu J Zhang W Face mask wearing detection algorithm based on improved YOLO-v4 Sensors 2021 21 9 3263 10.3390/s21093263 34066802
Yu, J. & Zhang, W. Face mask wearing detection algorithm based on improved YOLO-v4. Sensors 21(9), 3263 (2021).34066802 10.3390/s21093263
7. Thuan, D. Evolution of Yolo Algorithm and Yolov5: The State-of-the-Art Object Detention Algorithm 1301–1361 (Oulu University of Applied Sciences, 2021).
8. Liu, W., Anguelov, D., Erhan, D. et al. Ssd: Single shot multibox detector. In European Conference on Computer Vision 21–37 (Springer, 2016).
9. Zhang, N., Luo, J., Gao, W. Research on face detection technology based on MTCNN. In 2020 International Conference on Computer Network, Electronic and Automation (ICCNEA) 154–158 (IEEE, 2020).
10. Deng, J., Guo, J., Ververas, E. et al. Retinaface: Single-shot multi-level face localisation in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 5203–5212 (2020).
11. He, K., Gkioxari, G., Dollár, P. et al. Mask r-cnn. In Proceedings of the IEEE International Conference on Computer Vision 2961–2969 (2017).
12. Girshick, R. Fast r-cnn. In Proceedings of the IEEE International Conference on Computer Vision 1440–1448 (2015).
13. Ren, S., He, K., Girshick, R. et al. Faster r-cnn: Towards real-time object detection with region proposal networks. Adv. Neural Inf. Process. Syst. 28 (2015).
14. GIRSHICKR. Fast R-CNN. In Proceedings of the IEEE International Conference on Computer Vision 1440–1448 (IEEE, 2015).
15. Yang S Luo P Loy CC Faceness-net: Face detec––tion through deep facial part responses IEEE Trans. Pattern Anal. Mach. Intell. 2017 40 8 1845 1859 10.1109/TPAMI.2017.2738644 28809674
Yang, S. et al. Faceness-net: Face detec––tion through deep facial part responses. IEEE Trans. Pattern Anal. Mach. Intell. 40(8), 1845–1859 (2017).28809674 10.1109/TPAMI.2017.2738644
16. Niu Z Qin T Li H Chen J Improved mask wearing detection algorithm for natural scenes of RetinaFace Comput. Eng. Appl. 2020 56 12 1 7
Niu, Z., Qin, T., Li, H. & Chen, J. Improved mask wearing detection algorithm for natural scenes of RetinaFace. Comput. Eng. Appl. 56(12), 1–7 (2020).
17. Li, Y., Wang, S., Yang, Z. An improved face detection algorithm for multi-task cascading convolutional neural network. J. Xinyang Norm. Univ. (Nat. Sci. Ed.) 1–5.
18. Chao Y Liu W Tang H Ma C Wang Y Indoor face rapid detection method based on improved YOLO-v4 Comput. Eng. Appl. 2022 58 14 105 113
Chao, Y., Liu, W., Tang, H., Ma, C. & Wang, Y. Indoor face rapid detection method based on improved YOLO-v4. Comput. Eng. Appl. 58(14), 105–113 (2022).
19. Zhejiang University & Alibaba DAMO Academy Research Team. TransFace: A new framework for face recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (IEEE, 2023).
20. Kirillov, A., Girshick, R., He, K. et al. Panoptic feature pyramid networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 6399–6408 (2019).
21. Wang, Q., Wu, B., Zhu, P. et al. Supplementary material for ‘ECA-Net: Efficient channel attention for deep convolutional neural networks. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition 13–19 (IEEE, 2020).
22. Ghiasi, G., Lin, T. Y., Le, Q. V. NAS-FPN: Learning scalable feature pyramid architecture for object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 7036–7045 (2019).
23. Lin, T. Y., Goyal, P., Girshick, R. et al. Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision 2980–2988 (2017).
24. Lin, T. Y., Dollár, P., Girshick, R. et al. Feature pyramid networks for object detection. In Proceedings of IEEE/CVF International Conference on Computer Vision and Pattern Recognition 2117−2125 (IEEE, 2017).
