
==== Front
Heliyon
Heliyon
Heliyon
2405-8440
Elsevier

S2405-8440(24)12182-2
10.1016/j.heliyon.2024.e36151
e36151
Research Article
Image restoration model for microscopic defocused images based on blurring kernel guidance
Wei Yangjie weiyangjie@cse.neu.edu.cn
⁎
Li Qifei
Hou Weihan
Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, College of Computer Science and Engineering, Northeastern University, Wenhua Street 3, Shenyang, 110819, China
⁎ Corresponding author. weiyangjie@cse.neu.edu.cn
10 8 2024
30 8 2024
10 8 2024
10 16 e3615115 8 2023
6 8 2024
10 8 2024
© 2024 The Authors. Published by Elsevier Ltd.
2024

https://creativecommons.org/licenses/by-nc/4.0/ This is an open access article under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).
Defocus blurring imaging seriously affects the observation accuracy and application range of optical microscopes, and the blurring kernel function is a key parameter for high-resolution image restoration. However, its solving process is complicated and high in computational cost. Image restoration based on most neural networks has high requirements on data sets and the image resolution after restoration is limited because of the lack of quantitative estimation of blurring kernels. In this study, an image restoration method guided by blurring kernel estimation for microscopic defocused images is proposed. First, to reduce the blurring kernel estimation error caused by the positive and negative difference in microscopic defocused imaging, a defocused image classification network is designed to classify the input defocused images with different defocus distances and directions, and its output images are input into the blurring kernel extraction network composed of the feature extraction, correlation, and blurring kernel reconstruction layers. Second, a non-blind defocused image restoration model to restore the high-resolution images is proposed by introducing the blurring kernel extraction module into the restoration network based on U-Net, and the blurring kernel estimation and image restoration losses are jointly trained to realize image restoration guided by blurring kernel estimation. Finally, the experimental results of our proposed method demonstrate significant improvements in both the peak signal-to-noise ratio and structural similarity index measure when compared to other methods.

Keywords

Image restoration
Optical microscope
Blurring kernel
Defocused image
==== Body
pmc1 Introduction

Optical microscopes play a crucial role in various fields, including biomedicine, material science, and industrial manufacturing [1,2]. However, due to the limitation of depth-of-field (DOF) of a high-resolution optical microscope, defocus aberrations are very common, which seriously limits the application range of microscopes in some high-precision observation fields [3,4]. Therefore, the study of defocused image restoration is of great significance to improve the observation resolution of optical microscopes and promote their further application in biology, medicine and other fields [5].

Defocus imaging occurs when there is a deviation from the ideal object distance between the sample and the camera in an optical system. This deviation leads to the diffusion of light energy on the image plane, resulting in a blurred image. To measure the blurring degree of a blurred image, one can analyze the energy diffusion property of the blur. This property can be mathematically described using a function that represents the blurring kernel [6]. Therefore, the restoration of defocused blurred images involves an inverse process known as deconvolution, aiming to reconstruct a high-resolution image by convolving the defocused blurred images with the corresponding blurring kernel function [7]. According to whether the blurring kernel function is accurately known, the commonly used defocusing restoration methods are divided into blind deblurring methods [8,9] and non-blind deblurring methods [10,11]. In comparison, the non-blind deblurring methods are more suitable for optical microscopy imaging where higher image resolution is desired, because these methods have the advantages of accurate kernel estimation, noise removal, and enhanced image resolution. However, it is very difficult to directly obtain the blurring kernel function from an actual blurred image, because of complicated imaging process and various dynamic image features at the micro/nano scale. This seriously affects the accuracy of high-resolution image reconstruction.

In contrast, image restoration methods based on deep learning leverage the exceptional feature learning capability of convolutional neural networks [[12], [13], [14], [15]]. These methods can deblur images by learning the transformations of features from blurred to clear images. These methods avoid the process of establishing complex imaging models and solving blurring kernel functions; therefore, the complexity of image restoration is reduced, and the efficiency is improved. In conventional approaches, convolutional neural networks (CNNs) are trained by inputting pairs of blurred and clear images, creating a mapping model through network training and parameter optimisation. However, current CNN-based image restoration methods predominantly focus on deblurring macroscopic motion images and are rarely applied to deblurring microscopic defocused images. The reasons include the following: (1) Both the blurred and corresponding clear images of the same scene are simultaneously required in the training dataset when establishing a mapping model between them. However, small morphological changes on the sample surface may cause local defocus imaging because of the small DOFs of the cameras in microscopic imaging, and that makes obtaining clear images that strictly correspond to the defocused images difficult. (2) Because the quantitative blurring kernel function is not obtained when mapping from a blurred image to a clear image, these methods are essentially blind deblurring methods that cannot avoid the problems of conventional blind deblurring methods, such as poor robustness and limited accuracy. Particularly at the microscale, a small error in blurring kernel estimation may cause serious detail loss in the restored image.

Therefore, in this study, an image restoration method for microscopic defocused images based on CNNs is proposed, where a quantitative blurring kernel function is automatically evaluated and various sample images with different defocus degrees are non-blind reconstructed. First, to reduce the blurring kernel extraction error caused by the positive and negative difference of defocused imaging, a defocused image classification network is introduced into the extraction network to estimate the blurring kernels with different defocus distance and defocus direction. Second, an image restoration model is built based on U-Net, which is an improved model based on Fully Convolutional Networks (FCN) with a “Encoder-Decoder” structure. That structure allows both the low-level and high-level features in images are efficiently utilized. In the restoration process, the networks of blurring kernel extraction and image restoration are effectively combined together by using the blurring kernel extraction module, and then the blurring kernel estimation loss and the image restoration loss are jointly trained to realize image deblurring based on blurring kernel guidance. Compared with the traditional non-blind deblurring methods based on optical imaging models, the method we propose has high accuracy, efficiency, and strong robustness because the blurring kernels can be extracted from different defocused images. More importantly, besides defocus distance, the defocus direction relative to the ideal object distance is also considered by classifying the blurred images, which effectively reduces the blurring kernel estimation error caused by the positive and negative difference in microscopic defocused imaging. Furthermore, our method does not require one-to-one correspondence between defocused images and clear images in the training data, which increases the practicality of our image restoration.

The remainder of this paper is organized as follows. Section 2 contains an overview of the related work. Section 3 contains the proposed image restoration model. Section 4 conducts the experiments and Section 5 concludes the paper.

2 Related work

Currently, some scholars have established different mathematical models of optical microscopy imaging based on the energy propagation characteristics of light, and the blurring kernel function can be obtained by solving the mathematical equations of the dynamic diffusion of optical energy on the imaging plane under different camera parameters [[16], [17], [18]]. However, most microscopy imaging models are very complex and difficult to simplify to be analytical functions. Besides the mathematical methods, the blurring kernels can also be measured using embedded microbeads with fluorescent properties in the optical cement at different heights, or fixed fluorescent microbeads on an inclined surface [19,20]. This is challenging as it is difficult to precisely control the position of a microbead in optical cement, which leads to inaccurate measurement results. Moreover, several factors, such as light diffraction, light scattering, scene textures, aberrations, and other optical phenomena, can introduce complexities and inaccuracies in the measurement of blurring kernels [21]. These factors can affect the accuracy of estimating the exact blurring kernel function used in the deconvolution process.

In comparison, image deblurring methods based on deep learning normally utilize the excellent feature learning ability of neural networks to deblur images or generate high-resolution images. In this field, CNNs and Generative Adversarial Networks (GANs) are two commonly used networks. Because CNNs usually have good generalization capabilities across different types of images and images with different blurring causes, while DeblurGAN performs well for a specific type of blurring, such as motion blur [22]. Meanwhile, DeblurGAN has higher requirements on the quality and diversity of training datasets, and insufficient or biased training data may affect the quality of the generated images, whereas CNNs are usually less dependent on datasets [23]. Therefore, CNNs are widely used in image deblurring. Xu et al. proposed a separable structure to deconvolute images using the image intrinsic features obtained from CNNs [24]. This method reduces the dependence on the physical features of blurred images, but may lose some detailed texture information; Nah utilized the multi-scale CNNs and employed a coarse to fine training strategy to achieve blind image deblurring [25]. The network achieves good restoration results in different types of images, however, it easily loses the high-frequency information in the original images; Tao proposed a scale loop network, which combines the multi-scale structure and encoder-decoder structure, to achieve better deblurring effect by using weight sharing between scales [26]; Cai proposed an embedded network that introduces the dark channel priors and bright channel priors into the objective function [27]. On the other hand, since generative adversarial networks (GANs) were first proposed to generate more real data by learning, the deblur GANs have been used to achieve blind deblurring. Nimisha combined the global skip connections with the dense structural blocks and proposed a deep filter based on GANs [28,29]; in 2018 Kupyn proposed the image deblurring method DeblurGAN and subsequently DeblurGAN version 2 [30,31]. However, this network framework is relatively complex. In 2020, Zhang used two GANs to achieve image deblurring, one is to generate blurred data and the other is to restore a clear image based on the generated data [32]. This method has better deblurring effect compared to other GAN network algorithms, but when the inputs are images with complex blur causes, its deblurring effect is not satisfying because the network cannot learn features well.

Benefited with the independent analysis and learning capability of deep learning, it is reasonable to adaptively extracts the features of blurring kernel functions of different blurred images from multiple sets. Furthermore, the microscopic imaging often focuses on image blurring caused by small-scale defocus distance, and the distribution of blurring kernel functions in the entire image is generally compact and concentrated. Therefore, the inverse process of blurring imaging, deconvolution, is a global process, and the computational complexity is high [33]. Compared with traditional deconvolution computation methods, the deconvolution methods based on CNNs have higher adaptability and robustness, and can handle more diverse and complex images [34]. Therefore, a restoration model for defocused images based on CNNs guided by blurring kernel estimation is proposed in this study, which can simultaneously realize the automatic extraction of blurring kernel functions and quantitative image restoration.

3 Image restoration model for defocused microscopic images

The defocused blurring imaging process can be expressed as convolution the clear image with the blurring kernel function,(1) B(x,y)=I(x,y)*K(x,y)+n(x,y)

where B(x, y) is the defocused image; K(x, y) is the blurring kernel function; I(x, y) is the clear image or the focused image, n(x, y) represents the noise term and (x, y) is the two-dimensional coordinate of a point on the imaging plane.

According to the energy diffusion law of a point light source, the blurring kernel function K(x, y) can be represented by a two-dimensional circular function or Gaussian function, in which the radius of the circular function or the variance of the Gaussian function can be used to quantitatively describe the size of the energy-diffusion area. Generally, when the energy loss during the imaging process is not considered, all the energy emitted from a point light source is transmitted to the image plane, and the blurring kernel function can also be approximated by other types of functions, as long as,(2) ∬K(x,y)dxdy=1

According to Eqs. (1), (2), inversely, we can reconstruct the focused image with the deconvolution operation. The process can be described as,(3) I(x,y)=B(x,y)*K−1(x,y)

From Eq. (3), we can find that when the blurred image of a sample is known, its defocused image can be reconstructed with K(x, y). Based on this theory, an image deblurring method guided by blurring kernel estimation is proposed for defocused microscopic images in this study. The overall flow of our method is shown in Fig. 1, which encompasses three core subnetworks: microscopic image classification, blurring kernel extraction and image restoration.Fig. 1 The overall flow of the image restoration model in this study.

Fig. 1

First, the defocused image classification network is utilized to accurately classify the dataset containing microscopy images with different defocus degrees according to the defocus level. Second, the defocus images are fed into the blurring kernel extraction network to estimate the blurring kernel features. Finally, the defocused microscopic images with the estimated blurring kernel features are fed into the clear image restoration network to complete image deblurring.

3.1 Defocused image classification

As is well known, optical camera imaging follows the principle of pinhole imaging. According to the geometric optics, when the object moves the same distance away from/near the camera along the optical axis at the ideal object distance, while the camera parameters remain fixed, two similar blurred images can be captured on the imaging plane, but their blurring degree is not completely the same, as shown in Fig. 2.Fig. 2 Positive and negative defocus HE-stained tissue sections images on both sides of the focal plane.

Fig. 2

This phenomenon in Fig. 2 is called the positive and negative difference of defocused imaging, which is caused by different energy diffusion speed of the point light source on both sides of the ideal object distance of the convex lens. This defocus speed difference is not significant in the macroscopic camera imaging and is often ignored. However, due to small DOFs, the defocus speed of microscopic imaging is fast, and the parameter change for blurring kernel functions caused by the positive and negative defocus distance directly affects the accuracy of image restoration. Therefore, to improve the extraction accuracy of blurring kernel functions, a defocused image classification network is built before image restoration, which classifies the input defocused images into different categories according to the defocus distance and defocus direction.

The classification network proposed in this study uses the "convolution-maxpooling-fully-connected layer" network as the main structure, and the specific network structure is shown in Fig. 3. First, the channel normalization and original color channel exchange are performed on the input images, and then they are sequentially input into four image feature extraction blocks. Each block contains four layers: the convolution layer, activation function, channel attention layer and pooling layer. Among them, the convolutional layer is used to extract the internal features of the input images. To ensure that the receptive field of the network is large enough, the filter size is 5 × 5, and the stride is 1. The rectified linear unit (ReLU) function is chosen as the activation function of the convolution layer and the activated convolutional layer is combined with the channel attention module to achieve weighted calculation of extracted features. Then, the feature map size is reduced to half of its original size by the pooling layer.Fig. 3 The structure of our defocused image classification network.

Fig. 3

The original defocused images and the images after channel normalization are shown in Fig. 4(a) and (b), respectively, where the image contrast significantly increases after normalization. The images before and after exchanging color channels are shown in Fig. 5, where Fig. 5 (a) is the original red-green-blue image, and (b) and (c) represent the green-blue-red and blue-red-green images after exchanging color channels, respectively. From Fig. 5(a)–(c), we can observe that the color of the original image has changed after exchanging color channels, and the structure and detail features of the image have not changed. Therefore, the data augmentation strategy of exchanging color channels not only increases the amount of effective data and improves the generalization ability of our model, but also reduces the impact of color changes of samples on the classification accuracy of the network.Fig. 4 Image comparison before and after channel normalization. (a) Images before channel normalization; (b) Images after channel normalization.

Fig. 4

Fig. 5 Image comparison before and after exchanging the color channels. (a) Red-green-blue image; (b) Green-blue-red image; (c) Blue-red-green image. (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)

Fig. 5

Subsequently, a channel attention module is added to our classification network to increase the weight of effective feature channels. The specific implementation is shown in Fig. 6, where the height, width, and channel number of the input feature map are denoted by H, W, and C. First, the input features U are compressed by the squeeze operation Fsq, the dimension of the features is transformed from (H, W, C) to (1,1, C), and the global channel features are obtained. Then, the weights are generated for each feature channel through the excitation operation Fex, and the correlation between these channels is constructed through two fully-connected layers. The number of output weights is the same as that of the input feature maps. Finally, the scale operation Fscale is carried out to multiply the normalized weights by the features of each channel to obtain the weighted feature map.Fig. 6 The calculation process of channel attention mechanism.

Fig. 6

After four times of image feature extraction, a 7 × 7 feature map with 128 channels is obtained. Then, a 7 × 7 feature map with 32 channels is achieved through convolution (filter size is 1 × 1) to realize feature fusion and dimension compression, and finally it is input into the fully-connected layers in series. Finally, the Softmax function is used as the activation function, and its calculation formula is given in Eq. (4).(4) yn=ean∑n=1Nean

where n represents the number of image categories; N is the total number of image categories; yn is the probability that the input defocused image belongs to class n; an is the node value with activation.

In this study, to denote the blurring characteristic difference in the images on both sides of the ideal object distance, the sample images are classified into five categories according to the defocus distance (μm) and direction. The category type includes the second level near-focus f-2 = [−11, −5.5], the first-level near-focus f-1=(−5.5, 0), focus f0 = 0, the first-level far-focus f+1=(0, 5.5], and the second-level far-focus f+2=(5.5, 11]. Therefore, the probability that a defocused image belongs to these categories can be obtained using the Softmax function, and the category with the highest probability is the classification result. After that the defocused images are input into the next network for blurring kernel extraction.

3.2 The network for blurring kernel extraction

The blurring kernel extraction network in this study is divided into three layers: feature extraction, correlation, and blurring kernel reconstruction. The network structure is illustrated in Fig. 7. The feature extraction layer implements multi-scale feature extraction and pooling. Considering the feature extraction accuracy and many parameters, two layers of convolution with 64 convolutional kernels with a size of 3 × 3 are used for feature extraction in this study, and 32 convolutional kernels are used to reduce the number of channels before calculating the cross-correlation between features at each scale. Subsequently, pooling is performed. Such a network structure retains the main features extracted by the convolution to the maximum possible extent and avoids the degradation of the network computing speed caused by many parameters and large computations.Fig. 7 The structure of our blurring kernel extraction network.

Fig. 7

The correlation layer is to compute cross-correlation between multi-scale feature maps,(5) ci=∑x,yFi(x−s,y−t)Fj(x,y)

where F denotes the feature map output from the convolution layer; i, j are the scale level of the feature map, they are equal to 1, 2, 3; and the pixel space range in the x, y directions are: 2−im ≤ s, t ≤ 2−im, where ci∈Rn2(n2−1)×(2−im+1)×(2−im+1), where m is the spatial dimension of the blurring kernel, n = 64.

The number of output channels after correlation calculation in Eq. (5) is 32 × 31 = 992. To reduce it, 32 convolutional kernels with a size of 1 × 1 are used, and the size of the feature map is ci∈R32×(2−im+1)×(2−im+1). Notably, the correlation features calculated in the correlation layer are all obtained by calculating the correlation of different-scale features in the same images. If the incremental function is used as a filter in the previous convolutional layer, the auto-correlation function is calculated. That is, the correlation within each channel ci corresponds to the autocorrelation value of the feature map at this scale.

In the blurring kernel reconstruction layer, the blurring kernels are reconstructed from coarse to fine by subjecting the correlation features to multiple unpooling and deconvolution levels. First, the feature maps extracted from the correlation layer are integrated to achieve the desired size or resolution, and the unpooling and deconvolution operations are used recursively. Then, the feature maps of the next scale level are connected. Here, the deconvolution operation uses 32 convolution kernels of 3 × 3. After that, the number of channels is reduced from 64 to 32 by using 32 convolution kernels of 1 × 1. Second, the feature map of 32 channels is mapped to 24, 16, and 8 channels step-by-step through a series of convolution operations (convolution kernel size is 3 × 3) until the blurring kernel of a single channel is obtained. With the extracted blurring kernels, the blurred images are reconstructed with the network for image restoration.

3.3 The image restoration network

In this study, we use U-Net as the main structure of the restoration network to convolute B with K−1, and the U-Net structure is the encoder-decoder architecture proposed in Ref. [35]. As shown in Fig. 8, each layer of the restoration network has the same connection depth in the encoder and decoder, and the channel number in each layer of the feature map is set to 128. The red arrows in Fig. 8 represent the introduction of kernel-guided convolution modules in the convolutional operations in the encoder and decoder layers. After unpooling in the decoder layer, the feature maps are deconvoluted with 128 kernels, each with a size of 3 × 3. At the same time, the feature map from the corresponding encoder layer convolves with the convolution kernels. After that, a feature map with 128 channels is obtained by the encoder and decoder layers, and a feature map with 256 channels is finally obtained through patching two feature maps.Fig. 8 The structure of our image restoration network.

Fig. 8

To connect the restoration network with the blurring kernel function K, a nonlinear mapping between the blurring kernel and the U-Net weights needs to be computed. If the weights of each layer of U-Net are defined by K, the number of fully-connected layers is very large. In addition, learning the correlation between the weights of each U-Net layer also increases the nonlinearity degree and may affect the training and optimisation of the network. Therefore, in this study, we introduce a kernel-guided convolution module in the restoration network to modulate the output of each layer of U-Net by mapping K as a list of weights and biases, so that the blurring kernel affects the deconvolution operation uniformly in the whole spatial domain. In Fig. 9, the structure of the restoration network and the kernel-guided convolution module are depicted.Fig. 9 The structure of our kernel-guided convolution module.

Fig. 9

More specifically, a corresponding kernel-guided convolution module is added to each layer of U-Net, and each kernel-guided convolution module consists of three fully-connected layers and a ReLU activation layer (except the last layer). The input dimension of the kernel-guided convolution module is the total number of pixels in the blurring kernel, and the output dimension is equal to the number of channels in the corresponding layer. The kernel-guided convolution module simulates the functional correspondence between the weights, biases, and the blurring kernel. That is, the weights and biases of the feature map after convolution are adjusted with Eq. (6).(6) r=r⊙(1+x(k))+b(k)

where ⊙ denotes the dot product operation; x (k) is the weight vector; b (k) is the bias vector; k is the channel label of the blurring kernel.

In the model training process, we train the blurring kernel estimation loss function and image restoration loss functions simultaneously. First, the extraction and restoration networks are pre-trained to optimise these losses independently. Then, the entire extraction-restoration network is trained in an end-to-end manner to optimise the image restoration loss and ignore the kernel estimation loss. The training process is illustrated in Fig. 10. In practical applications, because the true blurring kernel functions corresponding to many blurred images are unknown and the difference between the estimated and true blurring kernels cannot be calculated directly, a new idea to calculate the kernel estimation loss is proposed in this study. That is, the difference between the blurred and original defocused images is taken as the blurring kernel estimation loss, and the blurred image is obtained by convolving the estimated blurring kernel with the focused image. Because the focused images are known, the image restoration loss during training can be directly used as the difference between the restored and focused images.Fig. 10 The training flow of the extraction - restoration network.

Fig. 10

The loss function L1 of blurring kernel estimation is defined as Eq. (7).(7) L1=1N∑N∈P|X(x,y)−XT(x,y)|

where X denotes the blurred image obtained by convolving the focused image the estimated with the blurring kernel; XT denotes the original defocused image; P denotes the set of all pixel points in the image, and the number of pixels is N.

The image restoration loss function L2 is defined as Eq. (8).(8) L2=1N∑N∈Q(G(x,y)−GT(x,y))2

where G denotes the restored clear image; GT denotes the focused image; Q denotes the pixel set.

4 Experiment and analysis

4.1 Experimental setup

All experiments in this study were carried out on a publicly available dataset of defocused HE-stained tissue sections images in Ref. [36]. The images in the dataset were captured by a color camera Nikon Eclipse motorized microscope with the following parameters: the numerical aperture of 0.75, the objective magnification of 20, the pixel number of 5 million, and the pixel size of 3.45 μm, and the acquired samples were hematoxylin and eosin-stained tissue sections. First, the origin or reference point for the sample positions was defined as the position at the ideal object distance from the objective lens. Second, the samples were acquired by moving them along the optical axis relative to the objective lens. They were moved in both positive (+) and negative (−) directions, with a range of −10 to +10 μm. To capture a comprehensive dataset, measurements were taken at 41 different defocus positions, with a step size of 0.5 μm between each position. Finally, the acquired images were divided into a fixed resolution of 224 × 224 pixels. The training set comprises nearly 130,000 defocused images. Within this training set, a subset equivalent to 10 % of the images was selected for validation purposes. This validation set served as an independent subset to assess the performance of the model during the training process and tune hyperparameters. The test set was divided into two connections: sameProtocol and diffProtocol. SameProtocol retains the same acquisition conditions and the data distributions as the training set, including the imaging conditions, staining condition of tissue samples, tissue structure, external light level, and data distribution. While diffProtocol differs from the training set in terms of acquisition conditions. Fig. 11 shows five groups of defocused blurred images with different shape and color, and each row from left to right is the f-2, f-1, f0, f+1, and f+2 categories.Fig. 11 Partial images of the dataset used in this study.

Fig. 11

4.2 Experiments of defocused image classification

To verify the effectiveness of our defocused image classification network, ablation experiments of our classification network were conducted in the sameProtocol and diffProtocol sets. The learning rate of the model training stage was set to 1 × 10−4, using the cross-entropy loss function and Adam gradient descent algorithm. The training had 250 epochs, the batch size was 128, and the training took approximately 23 h. Fig. 12 shows the convergence of the loss function during the training process using the training and validation sets. Evidently, when using the training set data, the loss function converged quickly with the increase in epochs, and the convergence process was very smooth. In contrast, for the data in the validation set, the loss function had small fluctuations in the convergence process, and it gradually stabilised after the epoch reached 100.Fig. 12 Loss curves of the classification network.

Fig. 12

First, to evaluate the validity of our classification network model, we conducted an ablation experiment, and the experimental results are shown in Table 1, where “baseline” model represents a typical classification network model consisting of the convolution, pooling and fully-connected layers, excluding data pre-processing and data augmentation; "baseline + channel normalization" model adds channel normalization to the baseline network structure; "baseline + channel normalization + exchange of color channels" model adds data augmentation and color channel exchanging to the second model; the classification network in our method adds channel normalization, data augmentation and color channel exchanging to the baseline network.Table 1 Classification accuracy of different network structures on different datasets.

Table 1	Accuracy on sameProtocol (%)	Accuracy on diffProtocol (%)	
baseline	96.21	85.01	
baseline + channel normalization	96.45	92.37	
baseline + channel normalization + exchange of color channels	97.17	94.22	
our method	98.37	96.49	

As shown in Table 1, the difference in classification accuracy between the two test sets was more than 10 %; in particular, the classification accuracy on the diffProtocol test set was only 85.01 %, indicating that the generalization ability of the baseline network was poor. After the channel normalization operation, the difference in classification accuracy between two test sets was reduced to approximately 4 %, and the classification accuracy on the diffProtocol test set was also improved. This indicates that channel normalization could effectively improve the generalization ability of the classification model; after using the data augmentation strategy based on the exchanging original color channels, the classification accuracy of the model on both test sets was further improved, which illustrated the effectiveness of original color channel exchanging. With the addition of the channel attention module, there was a significant improvement in the classification accuracy pf our classification network on both the sameProtocol and diffProtocol sets, and the difference between them was only 1.88 %, which illustrates that our method in this study has a good classification accuracy and excellent generalization.

Then, to prove the effectiveness of our classification method at different defocus distance, we compared it with the widely used image classification networks: Visual Geometry Group Network (VGGNet) [35], Residual Neural Network (ResNet) [37], and GoogleNet [38], and the evaluation indexes were precision, recall, and F1 score. The results are presented in Table 2, where we can draw the following conclusions.(1) The precision of VGGNet and GoogleNet for all image categories was lower than that of our classification method. The precision of ResNet with the images in the categories of f-2, f0, f+1, f+2 was lower than that of our method, except for the images in the f-1 category where the precision was 0.04 higher than our method.

(2) The recall of VGGNet for images in the f-2 and f+2 categories were higher than that of our classification method, that means the recall value of VGGNet was higher when the defocus distance was longer; the recall of ResNet was lower than that of our method for the focus and second-level far-focus category images, and higher for all other categories.

(3) The F1-Score of VGGNet and GoogleNet for f-2, f-1, f0, f+2 categories were lower than that of our method, except for VGGNet in the f+1 category where F1-Score was 0.09 higher than our method. This demonstrates the effectiveness of the proposed method.

Table 2 Precision of different models for different types of defocused images.

Table 2	f-2	f-1	f0	f+1	f+2	
Pre	Rec	F1	Pre	Rec	F1	Pre	Rec	F1	Pre	Pre	Pre	Pre	Rec	F1	
VGG	0.77	0.95	0.85	0.85	0.83	0.83	0.82	0.87	0.84	0.87	0.87	0.87	0.79	0.94	0.86	
ResNet	0.83	0.92	0.87	0.92	0.95	0.93	0.82	0.84	0.83	0.89	0.89	0.89	0.75	0.77	0.76	
GoogleNet	0.62	0.43	0.51	0.74	0.68	0.71	0.69	0.83	0.75	0.55	0.55	0.55	0.61	0.82	0.70	
Our	0.87	0.90	0.89	0.88	0.89	0.89	0.93	0.92	0.92	0.95	0.95	0.95	0.90	0.93	0.92	

Finally, to further verify the classification ability and generalization ability of our classification method, we compared the classification accuracy of different models on the test set, and the result is shown in Table 3, where we can observe that the classification accuracy of the proposed classification network on the sameProtocol test set was similar to that of ResNet, but considerably higher than that of VGGNet and GoogleNet. The classification accuracy of our method on the diffProtocol test set was 91.49 %; considerably higher than that of VGGNet (75.01 %), AlexNet (89.44 %) and GoogleNet (80.30 %). The experimental result demonstrates that the classification network in this study has good classification performance and generalisation ability.Table 3 Classification accuracy of different models in different test sets (%).

Table 3	VGGNet	ResNet	GoogleNet	Our method	
Accuracy on sameProtocol	86.21	91.21	88.73	96.37	
Accuracy on diffProtocol	75.01	89.44	80.30	91.49	
Accuracy on both datasets	78.94	90.05	83.31	93.32	

4.3 Experiments of image restoration

First, the image restoration model was trained using the TensorFlow framework and the GEFORCE RTX 3080Ti hardware platform. The training parameters are as follows: 250 epochs, Adam optimiser, and initial learning rate of 10−4. When the descent of the loss function stopped for 5 epochs, the learning rate was decreased by 20 %. The batch size was set to 128, and the total training time was approximately 37 h. The training results are shown in Fig. 13, where Fig. 13(a) and (b) show the loss functions of the blurring kernel estimation and image restoration, respectively. As seen from Fig. 13, with the increase of epoch number, the loss functions gradually converged on the training and validation sets; in comparison, the curves of image restoration loss function on the training and validation sets did not coincide after convergence, because the blurring kernel loss function and image restoration loss function are jointly trained in this study, and the estimation error of the blurring kernel may accumulate in the restoration network.Experiment 1 Blurring kernel extraction

Fig. 13 Loss curves of the extraction and restoration networks. (a) Loss curve of the extraction network; (b) Loss curve of the restoration network.

Fig. 13

First, the accuracy of the blurring kernel extraction in this study was verified using the test set. The experimental results are shown in Fig. 14, Fig. 15, Fig. 16, where Fig. 14 represents different blurred images of sample S16_16, S17_14, S204_15 and S209_14, respectively, where Fig. 14(a) and (b) are from the sameProtocol test set, and Fig. 14(c) and (d) are from the diffProtocol test set. Fig. 15(a)–(d) show the blurring kernel function extracted from the corresponding blurred images of S16_16, S17_14, S204_15 and S209_14 in Fig. 14. Because the true blurring kernel functions were unknown in our experiments, in this study the focused images in Fig. 14 were convolved with the estimated blurring kernel functions, and the result was compared with the original defocused images to validate the extracted blurring kernels. Fig. 16 shows the convolution result using the corresponding focus image from the database with the blurring kernel functions in Fig. 15. To compare the convolved blurred images with the original blurred images in Fig. 14, the SSIM for them was calculated. Besides, we also calculated the average SSIM on the sameProtocol and diffProtocol sets, as well as the average SSIM on both sets, to test the generalization of our blurring extraction method. The results are listed in Table 4.Fig. 14 Different blurred images in the datasets. (a) Image of S16_16; (b) Image of S17_14; (c) Image of S204_15; (d) Image of S209_14.

Fig. 14

Fig. 15 The blurring kernel functions extracted from the corresponding blurred images. (a)Kernel of S16_16; (b) Kernel of S17_14; (c) Kernel of S204_15; (d) Kernel of S209_14.

Fig. 15

Fig. 16 Convoluted blurring images with extracted blurring kernels in Fig. 15. (a) Image of S16_16; (b) Image of S17_14; (c) Image of S204_15; (d) Image of S209_14.

Fig. 16

Table 4 Similarity comparison between the convoluted blurred images and the original defocused images.

Table 4Images	Fig. 14, Fig. 16	Fig. 14, Fig. 16	Fig. 14, Fig. 16	Fig. 14, Fig. 16	Average on samePro.	Average on diffPro.	Total average	
SSIM	0.981	0.976	0.970	0.974	0.980	0.968	0.972	

Comparing the corresponding blurred images in Fig. 14, Fig. 16, we can see that the convolved blurred images were very close to the original defocused images in terms of image resolution and luminance distribution. From the experimental results in Table 4, we can see that the blurred images obtained by convolving the estimated blurring kernel with the focus image were very similar to the original defocused images in Fig. 14, and the SSIM values were all more than 0.96. Moreover, the results were very stable for two test sets and their average SSIM were 0.98 and 0.968, respectively. This proves that the blurred images obtained by convolution of the estimated blurring kernel with focused image are very similar to the actual blurred images; that is, the blurring kernels estimated from our method in this study are accurate.Experiment 2 Image restoration

First, to verify the performance of the proposed defocused image restoration method in this study, we selected five defocused images from the sameProtocol and diffProtocol datasets for image restoration experiments (S16–16, S17-14, and S101-11 were from sameProtocol; S204–15 and S209-14 were from diffProtocol). The color contrast and textures of these experimental images were all different so that the robustness of our method could also be validated. The experimental results are shown in Fig. 17, where the first column in Fig. 17(a)–(e) represents different blurred images; the second column in Fig. 17(a)–(e) represents the focus images corresponding to the first column images; the third column in Fig. 17(a)–(e) represents the restored images of the corresponding first column images. Based on the experimental results, it can be observed that after the extraction-restoration process the clarity and visibility of the defocused images were effectively improved. In Fig. 17, the defocused images in the first column were transformed into clearer images after undergoing an extraction-restoration process, as shown in the third column of the same row. Furthermore, the restoration process enhanced the visibility of tissue structures in the samples, making them distinguishable and more prominent. In addition, the image restoration performance of our method was not affected when the texture and color of the samples changed. That proves that the proposed method is insensitive to the light and texture of the images.Fig. 17 Comparison of deblurring with different blurred images. (a) The blurred, clear, and restored images of S16-16; (b)The blurred, clear, and restored images of S17-14; (c) The blurred, clear, and restored images of S101-11; (d) The blurred, clear, and restored images of S204-15; (e)The blurred, clear, and restored images of S209-14.

Fig. 17

Second, to further compare our defocused image restoration method with commonly used image restoration methods, we selected the dark channel prior [39], graph-based deblur [40] methods based on the iterative optimisation of optical imaging modelling and the DeblurGAN method [29], and UNeXtmethod based on CNNs [41]. In this experiment, we used two blurred images with different brightness values under identical experimental conditions. The results are shown in Fig. 18, Fig. 19, where Fig. 18(a)–(b) represent the input defocused image S102_l3 and its focus image; Fig. 18(c)–(g) represent the restored images of S102_l3 using the dark channel prior method, the UNeXt, the graph based deblur method, the deblur GAN, and our method. Fig. 19(a)–(b) represent the input defocused image S209_l3 and its focus image; Fig. 19(c)–(g) represent its restored images using the dark channel prior method, the UNeXt, the graph based deblur method, the deblur GAN, and our method. From Fig. 18, Fig. 19, we can observe that by restoring the images, the color, structure, and other features of the tissue in the images were enhanced, and the restored images based on our method were clearer and more detailed than those restored by optical model-based methods. In particular, in Fig. 19, when the tissue in the sample was dense and dark, the restored image was very close to the ideal focused image in terms of brightness and clarity, whereas the outline of individual cells in the image restored by the DeblurGAN method still had obvious blurring problems. For the UNeXt method, the restored image had some deviation from the original image in terms of color and texture.Fig. 18 Restoration comparison of different deblurring methods on image S102_l3. (a) The input image; (b) The focus image; (c) The restored image using dark channel prior method; (d) The restored image using UNeXt; (e) The restored image using graph based deblur method; (f) The restored image using deblur GAN; (g) The restored image using our method.

Fig. 18

Fig. 19 Restoration comparison of different deblurring methods on image S209_l3. (a) The input image; (b) The focus image; (c) The restored image using dark channel prior method; (d) The restored image using UNeXt; (e) The restored image using graph based deblur method; (f) The restored image using deblur GAN; (g) The restored image using our method.

Fig. 19

Third, to compare the quantitative quality of the restored images using these methods, we calculated the PSNR and SSIM of the restored images compared to the real-focus images. The experimental results were shown in Table 5, and the restoration images of sample S16_l6 through different methods were shown in Fig. 20, where Fig. 20(a)–(b) represent the input defocused image of sample S16_l6 and its focus image; Fig. 20(c)–(g) represent its restored images using the dark channel prior method, the UNeXt, the graph based deblur method, the deblur GAN, and our method. As can be seen from Table 5, the PSNR and SSIM values of the restored images using iterative modelling of optical imaging were low, and the difference in both PSNR and SSIM values were very obvious in two test sets because these methods were essentially blind deblurring with unknown blurring kernel functions and their generalization abilities were not ideal due to their excessive focus on the global optimal evaluation of the entire images; DeblurGAN and our method were both deblurring methods based on convolutional neural networks, which learned the blurring features of the images through the network training. Therefore, their PSNR and SSIM values of the deblurred images had more significant improvement after the deblurring process, and the difference of PSNR and SSIM values were also reduced on different test sets, which indicated that these deblurring methods based on convolutional neural networks had higher accuracy and stronger robustness. By contrast, the PSNR and SSIM values of the images restored by our method were higher than those of the DeblurGAN method. That means, by incorporating a blurring kernel extraction network, our method can estimate or adjust the blurring kernels estimated in the defocused images during the restoration process. For the UNeXt method, the PSNR and SSIM values of the restored images were all lower than thoss of our method, because the network structure is a simplification of U-Net with fewer convolutional layers, which greatly reduces the computation complexity compared to U-Net, but the drastic reduction in the parameter quantity leads to its poor ability to image deblurring. According to Fig. 20, it can be seen that our proposed method for sample S16_l6 achieves the best restoration results in terms of color, texture, and clarity compared to other methods.Experiment 3 Robustness validation

Table 5 Deblurring performance of different image deblurring methods.

Table 5	Dark channel prior	Graph-based	DeblurGAN	UNeXt	Our method	
PSNR	SSIM	PSNR	SSIM	PSNR	SSIM	PSNR	SSIM	PSNR	SSIM	
S16-l6	28.782	0.896	30.125	0.898	33.673	0.879	25.232	0.884	33.673	0.979	
S17-l4	27.112	0.863	28.228	0.859	31.599	0.880	25.230	0.837	31.599	0.924	
S101-l1	28.673	0.877	29.683	0.883	31.363	0.901	25.229	0.872	31.363	0.948	
S204-l5	25.359	0.853	26.141	0.878	30.022	0.884	25.228	0.855	30.022	0.903	
S209-l4	26.493	0.843	26.032	0.867	29.456	0.879	25.290	0.902	29.456	0.911	

Fig. 20 Restoration comparison of different deblurring methods on the images of S16_l6. (a) The input image; (b) The focus image; (c) The restored image using dark channel prior method; (d) The restored image using UNeXt; (e) The restored image using graph based deblur method; (f) The restored image using deblur GAN; (g) The restored image using our method.

Fig. 20

To further demonstrate the robustness of our image restoration method with the images in different defocus categories, we randomly selected twenty blurred images in each defocus category for deblurring, and compared the deblurred images with those using other deblurring methods. The result is shown in Table 6, and the restoration images in defocus category f-2 through different methods are shown in Fig. 21, where Fig. 21 (a) represents the input defocused image in defocus category f-2; Fig. 20(b)–(f) represent its restored images using the dark channel prior method, the UNeXt, the graph based deblur method, the deblur GAN, and our method. As can be seen from Table 6, the proposed image restoration method is very stable for processing test images in different defocus categories, and the average PSNR and SSIM of the test images in different categories are significantly higher than those of the other four methods, which proves the accuracy and stability of our proposed method are both higher According to Fig. 21, it can be seen that the restoration effect of the defocused images in f-2 is better than other methods in terms of color, texture, and clarity.Table 6 Comparison of the deblurring performance with different defocus category images.

Table 6	Dark channel	Graph-based	DeblurGAN	UNeXt	Our method	
PSNR	SSIM	PSNR	SSIM	PSNR	SSIM	PSNR	SSIM	PSNR	SSIM	
f-2	26.833	0.855	27.239	0.895	28.800	0.881	25.225	0.823	30.536	0.918	
f-1	27.002	0.851	28.121	0.847	29.697	0.873	25.226	0.813	31.208	0.947	
f0	28.209	0.864	28.475	0.871	29.979	0.866	25.231	0.810	32.077	0.961	
f+1	25.871	0.875	26.342	0.881	28.975	0.894	25.227	0.811	31.133	0.925	
f+2	26.984	0.861	27.544	0.874	29.363	0.879	25.233	0.802	31.239	0.938	

Fig. 21 Restoration comparison of different deblurring methods on the images of defocus category f-2. (a) The input image; (b) The restored image using dark channel prior method; (c) The restored image using UNeXt; (d) The restored image using graph based deblur method; (e) The restored image using deblur GAN; (f) The restored image using our method.

Fig. 21

In addition, we calculated the average PSNR and SSIM values of all the deblurred images using our method in this study on the sameProtocol and diffProtocol test sets and compared them with those of several commonly used methods. The experimental results are shown in Table 7. From Table 7, we can observe that when tissue sections images under different acquisition conditions are used, the average PSNR and SSIM of the images restored by our method are significantly higher than those of the other four methods on the test sets, indicating that the proposed defocused image restoration method has good robustness.Table 7 Restoration performance of different methods on the sameProtocol and diffProtocol test sets.

Table 7Sets	Dark channel prior	Graph-based deblur	DeblurGAN	UNeXt	Our method	
PSNR	SSIM	PSNR	SSIM	PSNR	SSIM	PSNR	SSIM	PSNR	SSIM	
Same	28.821	0.878	29.473	0.890	29.782	0.887	25.231	0.812	32.158	0.955	
Diff	26.066	0.852	26.580	0.866	29.154	0.875	25.468	0.890	30.780	0.930	

5 Discussion

To address the limitations of current microscopic image defocus restoration methods and the positive-negative defocus variability of optical imaging, this study investigates several aspects, including the classification of defocus images, the automatic extraction of blurring kernels, and the construction of a joint network of blurring kernel estimation and image restoration. The ultimate goal is to achieve high-precision and clear restoration of defocus microscopic images. Numerous experiments have proved that the method we proposed is more accurate and robust compared with other image deblur methods. However, it still needs further research and improvement. For example, to improve its processing speed to make it suitable for real-time image deblurring.

6 Conclusions

The proposed non-blind image restoration model aims to address the challenges associated with limited DOF and defocused blurring in high-magnification optical microscopy. It offers a solution to improve image resolution and expand the range of applications for microscopes. First, a microscopic defocused image classification network is designed to achieve the automatic classification of defocused images with different defocus distances and directions to reduce the estimation error of blurring kernel functions caused by the positive and negative difference of defocus imaging. Second, a defocused image restoration model consisting of blurring kernel extraction and image restoration networks is proposed. These components are effectively combined by introducing a kernel guidance convolution module in the image restoration network. Non-blind image deblurring is achieved by training the blurring kernel estimation and image restoration loss functions simultaneously for quantitative deblurring. Third, the proposed image-restoration model is validated for accuracy, generalization and robustness using extensive biomedical microscopic images.

Data availability statement

Data will be made available on request.

Funding

10.13039/501100001809 National Natural Science Foundation of China (62373084).

CRediT authorship contribution statement

Yangjie Wei: Supervision, Project administration, Methodology, Funding acquisition, Conceptualization. Qifei Li: Writing – original draft, Investigation, Formal analysis, Data curation. Weihan Hou: Writing – review & editing.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
==== Refs
References

1 Yao P. Liu R. Construction and use of an adaptive optics two-photon microscope with direct wavefront sensing Nat. Protoc. 18 12 2023 3732 3766 10.1038/s41592-019-0434-7 37914781
2 Priyamvada G. Rai N. Verma A. Microscopy based methods for characterization, drug delivery, and understanding the dynamics of nanoparticles Med. Res. Rev. 44 1 2024 138 168 10.1002/med.21981 37294298
3 Chen L.W. Zhou Y. Zhou R. Hong M.H. Microsphere-toward future of optical microscopes iScience 23 101211 2020 1 20 10.1016/j.isci.2020.101211
4 Wei Y.J. Dong Z.L. Wu C.D. Depth measurement using single camera with fixed camera parameters IET Comput. Vis. 6 2012 29 39 10.1049/iet-cvi.2010.0017
5 Roider C. Heintzmann R. Piestun R. Jesacher A. Deconvolution approach for 3d scanning microscopy with helical phase engineering Opt. Exp. 24 2016 15456 15467 10.1364/OE.24.015456
6 Zhang X. Wang R. Jiang X. Spatially variant defocus blur map estimation and deblurring from a single image J. Vis. Commun. Image Represent. 35 2016 257 264 10.1016/j.jvcir.2016.01.002
7 Liu X.W. Huang L.H. A new nonlocal total variation regularization algorithm for image denoising Math. Comput. Simulat. 97 2014 224 233 10.1016/j.matcom.2013.10.001
8 Li X. Jia J. Two-phase kernel estimation for robust motion deblurring Proc. ECCV 2010 157 170 10.1007/978-3-642-15549-9_12
9 Aharon M. Elad M. Bruckstein A. K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation IEEE Trans. Signal Process. 54 2006 4311 4322 10.1109/TSP.2006.881199
10 Sarder P. Nehorai A. Deconvolution methods for 3-d fluorescence microscopy images IEEE Signal Process. Mag. 23 2006 32 45 10.1109/MSP.2006.1628876
11 Kim J. An S. Ahn S. Kim B. Depth-variant deconvolution of 3d widefield fluorescence microscopy using the penalized maximum likelihood estimation method Opt. Exp. 21 2013 27668 27681 10.1364/OE.21.027668
12 Xie J. Xu L. Chen E. Image denoising and inpainting with deep neural networks Proc. NIPS 2012 341 349
13 Harold B. Christian C. Schuler J. Harmeling S. Image denoising with multi-layer perceptions, part 1: comparison with existing algorithms and with bounds Comput Sci 38 2012 1544 1582 10.48550/arXiv.1211.1544
14 Zhang K. Zuo W.M. Chen Y.J. Meng D.Y. Zhang L. Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising IEEE Trans. Image Process. 26 2017 3142 3155 10.1109/TIP.2017.2662206 28166495
15 Xu L. Ren J. Liu C. Deep convolutional neural network for image deconvolution Proc. NIPS 2014 1790 1798
16 Patwary N. Preza C. Image restoration for three-dimensional fluorescence microscopy using an orthonormal basis for efficient representation of depth-variant point-spread functions Biomed. Opt. Exp. 6 2015 3826 3841 10.1364/BOE.6.003826
17 Roider C. Heintzmann R. Piestun R. Jesacher A. Deconvolution approach for 3d scanning microscopy with helical phase engineering Opt. Exp. 24 2016 15456 15467 10.1364/OE.24.015456
18 Wei Y.J. Hou W.H. Blurring kernel extraction and super-resolution image reconstruction based on style generative adversarial networks Opt. Exp. 29 2021 44024 44044 10.1364/OE.441507
19 Diezmann A. Lee M.Y. Lew M.D. Moerner W. Correcting field-dependent aberrations with nanoscale accuracy in three dimensional single-molecule localization microscopy Optica 2 2015 985 993 10.1364/OPTICA.2.000985 26973863
20 Aristov A. Lelandais B. Rensen E. Zimmer C. ZOLA-3D allows flexible 3D localization microscopy over an adjustable axial range Nat. Commun. 9 2018 1 8 10.1038/s41467-018-04709-4 29317637
21 Jain V. Seung H.S. Natural image denoising with convolutional networks Proc. NIPS 2008 MIT Press 1 8
22 Cho S.J. Ji S.W. Hong J.P. Rethinking coarse-to-fine approach in single image deblurring Proceedings of the IEEE/CVF International Conference on Computer Vision 2021 10.1109/iccv48922.2021.00460
23 Quan Y.H. Lin P.K. Xu Y. Nonblind image deblurring via deep learning in complex field IEEE Transact. Neural Networks Learn. Syst. 33 10 2021 5387 5400 10.1109/TNNLS.2021.3070596
24 Nah S. Kim T.H. Lee K.M. Deep multi-scale convolutional neural network for dynamic scene deblurring Proc. CVPR 2017 3883 3891 10.1109/cvpr.2017.35
25 Tao X. Gao H. Wang Y. Scale-recurrent network for deep image deblurring Proc. CVPR 2018 8174 8182 10.1109/cvpr.2018.00853
26 Cai J. Zuo W. Zhang L. Dark and bright channel prior embedded network for dynamic scene deblurring IEEE Trans. Image Process. 99 2020 1 10.1109/TIP.2020.2995048 1
27 Goodfellow I.J. Abadie J.P. Mirza M. Generative adversarial networks Proc. NIPS 2014 2672 2680 10.1145/3422622
28 Nimisha T.M. Singh A.K. Rajagopalan A.N. Blur-invariant deep learning for blind-deblurring Proc. CVPR 2017 4762 4770 10.1109/iccv.2017.509
29 Kupyn O. Budzan V. Mykhailych M. DeblurGAN: blind motion deblurring using conditional adversarial networks Proc. CVPR 2018 257 265 10.1109/cvpr.2018.00854
30 Kupyn O. Martyniuk T. Wu J. DeblurGAN-v2: deblurring (orders-of-magnitude) faster and better Proc. ICCV 2019 8878 8886 10.1109/iccv.2019.00897
31 Zhang K. Luo W. Zhong Y. Deblurring by realistic blurring Proc. CVPR 2020 2734 2743 10.1109/cvpr42600.2020.00281
32 Ronneberger O. Fischer P. Brox T. U-Net: convolutional networks for biomedical image segmentation Proc. MICCAI 2015 234 241 10.1007/978-3-662-54345-0_3
33 Riad S.M. The deconvolution problem: an overview Proceedings of the IEEE 1986 82 85 10.1109/proc.1986.13407
34 Xu L. Deep convolutional neural network for image deconvolution Adv. Neural Inf. Process. Syst. 27 2014
35 Jiang S. Liao J. Bian Z. Transform- and multi-domain deep learning for single-frame rapid autofocusing in whole slide imaging Biomed. Opt Express 9 2018 1601 1612 10.1364/boe.9.001601 29675305
36 Simonyan K. Zisserman A. Very deep convolutional networks for large-scale image recognition Proc. ICLR 2015 1 14 10.48550/arXiv.1409.1556
37 He K. Zhang X. Ren S. Deep residual learning for image recognition Proc. CVPR 2016 770 778 10.1109/cvpr.2016.90
38 Szegedy C. Liu W. Jia Y. Going deeper with convolutions Proc. CVPR 2014 1 9 10.1109/cvpr.2015.7298594
39 Pan J. Sun D. Pfister H. Blind image deblurring using dark channel prior Proc. CVPR 2016 1628 1636 10.1109/cvpr.2016.180
40 Bai Y.C. Cheung G. Liu X.M. Gao W. Graph-based blind image deblurring from a single photograph IEEE Trans. Image Process. 28 2018 1404 1418 10.1109/tip.2018.2874290
41 Chan S.X. Huang C. Bai C. Res2-UNeXt: a novel deep learning framework for few-shot cell image segmentation Multimed. Tool. Appl. 81 10 2022 13275 13288 10.1007/s11042-021-10536-5
