
==== Front
Endosc Ultrasound
Endosc Ultrasound
EUSJ
Endoscopic Ultrasound
2303-9027
2226-7190
Lippincott Williams & Wilkins

EUSJ_240080
10.1097/eus.0000000000000079
00004
3
Original Research
Computed tomography–based radial endobronchial ultrasound image simulation of peripheral pulmonary lesions using deep learning
Zhang Chunxi zhcx1999@163.com
1 2 3
Zhou Yongzheng z13052344526@163.com
1 2 3
Sun Chuanqi chuanqisun077@gmail.com
4
Zhang Jilei zjlei2012@126.com
4
Chen Junxiang cjx666125@163.com
1 2 3
Zheng Xiaoxuan milozheng59@163.com
1 2 3
Li Ying myliying@126.com
1 2 3
Liu Xiaoyao xiaoyao.liu@microport.com
4
Liu Weiping 4 ∗
Sun Jiayuan 1 2 3 ∗
1 Department of Respiratory Endoscopy, Shanghai Chest Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
2 Department of Respiratory and Critical Care Medicine, Shanghai Chest Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
3 Shanghai Engineering Research Center of Respiratory Endoscopy, Shanghai, China
4 Shanghai Intelligent Surgery Center, Shanghai MicroPort MedBot (Group) Co, Ltd, Shanghai, China.
∗ Address for correspondence: Department of Respiratory Endoscopy, Department of Respiratory and Critical Care Medicine, Shanghai Chest Hospital, Shanghai Jiao Tong University School of Medicine, 241 West Huaihai Road, Shanghai 200030, China. E-mail: xkyyjysun@163.com (J. Sun); Shanghai Intelligent Surgery Center, Shanghai MicroPort MedBot (Group) Co, Ltd, 1601 Zhang Dong Road, ZJ Hi-Tech Park, Shanghai 201203, China. E-mail: Weiping.Liu@microport.com (W. Liu).
Jul-Aug 2024
20 8 2024
13 4 239247
30 8 2023
27 5 2024
Copyright © 2024 The Author(s). Published by Wolters Kluwer Health, Inc on behalf of Scholar Media Publishing.
2024
Wolters Kluwer on behalf of Scholar Media Publishing.
https://creativecommons.org/licenses/by-nc-nd/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial-No Derivatives License 4.0 (CCBY-NC-ND), where it is permissible to download and share the work provided it is properly cited. The work cannot be changed in any way or used commercially without permission from the journal.

ABSTRACT

Background and Objectives

Radial endobronchial ultrasound (R-EBUS) plays an important role during transbronchial sampling of peripheral pulmonary lesions (PPLs). However, existing navigational bronchoscopy systems provide no guidance for R-EBUS. To guide intraoperative R-EBUS probe manipulation, we aimed to simulate R-EBUS images of PPLs from preoperative computed tomography (CT) data using deep learning.

Materials and Methods

Preoperative CT and intraoperative ultrasound data of PPLs in 250 patients who underwent R-EBUS–guided transbronchial lung biopsy were retrospectively collected. Two-dimensional CT sections perpendicular to the biopsy path were transformed into ultrasonic reflection and transmission images using an ultrasound propagation model to obtain the initial simulated R-EBUS images. A cycle generative adversarial network was trained to improve the realism of initial simulated images. Objective and subjective indicators were used to evaluate the similarity between real and simulated images.

Results

Wasserstein distances showed that utilizing the cycle generative adversarial network significantly improved the similarity between real and simulated R-EBUS images. There was no statistically significant difference in the long axis, short axis, and area between real and simulated lesions (all P > 0.05). Based on the experts’ evaluation, a median similarity score of ≥4 on a 5-point scale was obtained for lesion size, shape, margin, internal echoes, and overall similarity.

Conclusions

Simulated R-EBUS images of PPLs generated by our method can closely mimic the corresponding real images, demonstrating the potential of our method to provide guidance for intraoperative R-EBUS probe manipulation.

Key Words

Radial endobronchial ultrasound
Computed tomography
Ultrasound simulation
Peripheral pulmonary lesion
Deep learning
SDCT
OPEN-ACCESSTRUE
==== Body
pmcINTRODUCTION

Peripheral pulmonary lesions (PPLs) are lesions beyond the segmental bronchi that are invisible during bronchoscopy.[1,2] With the wide application of chest computed tomography (CT), a growing number of PPLs are being detected, many of which may represent peripheral lung cancer. For accurate diagnosis, tissue sampling is often performed, and transbronchial lung biopsy (TBLB) is one of the most commonly used sampling methods.[3]

Radial endobronchial ultrasound (R-EBUS) is an important auxiliary technique for the TBLB, helping physicians to confirm lesion localization before taking a biopsy.[4] In addition, because a diagnostic result is more likely to be obtained when the R-EBUS probe is located within rather than adjacent to the target lesion, physicians can choose an ideal biopsy location based on the R-EBUS view.[5–7] With the assistance of R-EBUS, the diagnostic yield of TBLB in patients with PPLs has significantly improved.[8,9] However, there is currently no technology that can predict the target lesion appearance in R-EBUS images and provide guidance for intraoperative R-EBUS probe manipulation. Even with the use of a navigational bronchoscopy system, the bronchoscope can only be navigated to the vicinity of the lesion due to its large size. To further confirm lesion localization, physicians have to manipulate the R-EBUS probe without any guidance, and repeated movements of the probe are often required to determine the optimal biopsy location, which prolongs the procedure time and may increase the risk of complications. Furthermore, factors such as respiratory motion, bleeding, and atelectasis can interfere with the interpretation of R-EBUS images and even lead to a false-positive R-EBUS finding, ultimately hindering the success of diagnosis.[10] Thus, an R-EBUS image simulation method for predicting the target lesion appearance and guiding R-EBUS probe manipulation is greatly needed.

Some previous studies have attempted to simulate ultrasound images using CT data.[11–13] However, these studies were limited to traditional physical models, and complex postprocessing techniques were performed to enhance the realism of the simulated images. With the development of deep learning, the cycle generative adversarial network (CycleGAN) is increasingly used to synthesize fake medical images.[14–16] To the best of our knowledge, no study has yet investigated the use of a CycleGAN for CT-based R-EBUS image simulation.

In this study, to predict the target lesion appearance in R-EBUS images and provide guidance for intraoperative R-EBUS probe manipulation, we aimed to develop a method integrating an ultrasound propagation model with a CycleGAN to simulate R-EBUS images of PPLs using preoperative CT data.

MATERIALS AND METHODS

Patients and data set

This retrospective study enrolled consecutive patients who underwent R-EBUS–guided TBLB between July 2018 and December 2021 at Shanghai Chest Hospital. Inclusion criteria included (1) chest CT that was performed preoperatively and stored in DICOM (digital imaging and communication in medicine) format and (2) an ultrasonographic video that was recorded during the procedure and clearly showed the target lesion. Exclusion criteria included (1) lesions that exceeded the field of view of the ultrasonographic video, (2) lesions with a ground-glass opacity (GGO) component of >50% (GGO-dominant lesions), and (3) low quality of chest CT or ultrasonographic video (such as severe noises in the ultrasonographic video and motion artifacts in the CT scan). Ultrasonographic videos and corresponding preoperative chest CT data of 250 patients constituted the data set of this study. The data set was randomly divided into training (n = 150), validation (n = 50), and testing (n = 50) sets [Figure 1]. This study was approved by the Ethics Committee of Shanghai Chest Hospital (KS23025), and the requirement for informed consent was waived.

Figure 1 Selection flowchart of patients in this study. GGO, ground-glass opacity.

Data acquisition

All preoperative CT scans were acquired on 1 of the following 5 scanners: 64-detector row scanner (Brilliance; Philips, Cleveland, OH), 128-detector row scanner (Ingenuity Core 128; Philips, Suzhou, China), 256-detector row scanner (Revolution CT; GE Healthcare, Waukesha, WI), Discovery CT750HD CT scanner (GE Healthcare), and 16-detector row scanner (uCT S160; United Imaging, Shanghai, China). The image acquisition parameters were as follows: 220–300 mA, 120 kVp, 0.625- to 1.25-mm detector collimation, 0.64 beam pitch, 0.625- to 1.25-mm slice thickness, 512 × 512 or 1024 × 1024 matrix size, and 350- to 400-mm field of view.

All R-EBUS–guided TBLBs were performed by experienced bronchoscopists. Virtual bronchoscopic navigation or electromagnetic navigation bronchoscopy was used to navigate the bronchoscope to the target lesion. R-EBUS was used to confirm target lesion localization before taking a biopsy. R-EBUS was performed using an EBUS system (EU-ME2; Olympus, Tokyo, Japan), which was equipped with a 20-MHz mechanical radial-type probe (UM S20-17S or UM S20-20R; Olympus). The guide sheath (SG-200C or SG-201C; Olympus) was used in most cases. In some cases, fluoroscopy was used. The probe was adjusted continuously based on the real-time R-EBUS image until the bronchoscopist considered that the lesion had been reached. Then, the probe was used to scan the lesion from its proximal to distal end, and an ultrasonographic video of at least 10-second duration was recorded. The long and short axes of the lesion were measured at its maximum cross section. A biopsy that resulted in a malignant or specific benign (eg, tuberculosis, aspergillosis, etc) process was considered diagnostic. Biopsy specimens with nonspecific benign findings were considered diagnostic only if (1) the diagnosis was confirmed by a subsequent surgical or CT-guided biopsy; or (2) follow-up imaging demonstrated improvement or resolution of the lesion. Specimens that showed atypical cells or normal pulmonary elements were considered nondiagnostic regardless of the final diagnosis.

Data preprocessing

As shown in Figure 2, data preprocessing consisted of 3 steps that enabled subsequent simulation. First, because the R-EBUS probe was advanced through bronchi and captured a 360° 2-dimensional (2D) scan-plane view perpendicular to the probe, 2D CT sections perpendicular to the bronchi through which the R-EBUS probe was advanced were derived from original CT data. The R-EBUS probe had a scanning depth of 20 mm, and the same 40 × 40-mm2 field of view of the 2D CT section was reconstructed. Second, bilinear interpolation was used to up-sample the spacing of 2D CT sections from 0.5 mm/pixel to 0.1 mm/pixel, which was comparable to that of real R-EBUS images. Finally, the 2D CT sections were mapped from rectangular coordinates to polar coordinates.

Figure 2 Flowchart of the data preprocessing. A, Original chest CT data. B, 2D CT section perpendicular to the biopsy path. C, High-resolution 2D CT section with 0.1 mm/pixel spacing. D, Corresponding polar-form image. 2D, 2-dimensional; CT, computed tomography.

R-EBUS simulation

The R-EBUS simulation framework mainly consisted of 2 steps: (1) initial R-EBUS simulation using an ultrasound propagation model and (2) enhancement of the realism of initial simulated R-EBUS images using a CycleGAN.

The R-EBUS probe emits high-frequency sound waves and collects the returned signals from tissues with varying acoustic impedances. Previous studies demonstrated a linear correlation between CT Hounsfield units and acoustic impedances. Consequently, a physical model of ultrasound propagation was developed, and the provided CT image was linked to the model.[11,17] As shown in Figure 3, high-resolution 2D CT sections perpendicular to the biopsy path were transformed into ultrasonic reflection and transmission images using the ultrasound propagation model. To obtain initial simulated R-EBUS images, ultrasonic reflection and transmission images were merged together, and some commonly observed R-EBUS noises were added, including comet tails and speckles. Additionally, a circle was overlaid on the center of the initial simulated R-EBUS image to indicate the position of the R-EBUS probe, the diameter being consistent with the actual probe size. The details of the initial R-EBUS simulation process are explained in the Supplementary File, http://links.lww.com/ENUS/A360.

Figure 3 Example of the initial R-EBUS simulation using the ultrasound propagation model. A, High-resolution 2D CT section perpendicular to the biopsy path. B and C, Corresponding ultrasonic reflection and transmission images. D, The initial simulated R-EBUS image obtained by merging ultrasonic reflection and transmission images together and adding some commonly observed R-EBUS noises. 2D, 2-dimensional; R-EBUS, radial endobronchial ultrasound.

A CycleGAN was trained using real and initial simulated R-EBUS images to simulate ultrasound-like noises and improve realism [Figure 4]. The network had 2 discriminators (DX and DY) and 2 generators (G: X → Y and F: Y → X) within 2 cycles. X and Y represented initial simulated and real R-EBUS images, respectively. The Res-Net model served as the generator, whereas a 3-layer discriminator jointly supervised the simulation performance of the proposed network. During training, the generator aimed to produce images as realistic as possible while the discriminator’s ability to distinguish between fake and real images was continuously improved.[18] In the zero-sum game between the generator and discriminator, the choice of loss function was critical. The full objective function (Lfull) in the developed network is defined as the sum of losses from the generator and discriminator:

Figure 4 The model architecture of the cycle generative adversarial network. The network had 2 discriminators (DX and DY) and 2 generators (G: X → Y and F: Y → X) within 2 cycles. A–C, The forward cycle. Fake Y was synthesized from real X by G: X → Y, and reconstructed X can be generated from fake Y by F: Y → X. D–F, The backward cycle. Fake X was synthesized from real Y by F: Y → X, and reconstructed Y can be generated from fake X by G: X → Y. X, initial simulated R-EBUS image; Y, R-EBUS image; R-EBUS, radial endobronchial ultrasound.

Lfull=LGANGDYXY+LGANFDXYX+λLcycleGF

where λ is a weighting factor, and LGAN and Lcycle are the GAN loss and cycle-consistent loss, respectively. These parameters are defined as:

LGANGDYXY=ElogDYy+Elog1−DYGx

LGANFDXYX=ElogDXx+Elog1−DXFy

LcycleGF=EFGx−x1+EGFy−y1

where mapping functions G: X → Y and F: Y → X are called generators, and x, y are elements belonging to the X and Y fields, respectively. G tries to generate images G(x) that look similar to images from domain Y, whereas DY aims to distinguish between translated samples G(x) and real samples y. G aims to minimize this objective against an adversary D that tries to maximize it.

The experiments were conducted using PyTorch on a workstation equipped with an NVIDIA GeForce RTX 3090 24G GPU. During the training phase, the CycleGAN was fine-tuned using the Adam optimizer and carefully chosen hyperparameters, such as an image size of 400 × 400 pixels, a basic learning rate of 2 × 10−4, and a batch size of 2. The model was trained using these hyperparameters and subsequently evaluated using the validation set. Finally, the model was tested using the testing set to determine its generalizability and assess its performance.

Objective image evaluation

Objective and subjective indicators were used to evaluate the similarity between real and simulated R-EBUS images. The real R-EBUS image containing the maximum cross section of the lesion and the corresponding simulated R-EBUS image were selected for the evaluation.

In the objective evaluation, the Wasserstein distance was calculated. The Wasserstein distance is a distance metric based on probability distributions. It provides an intuitive way of measuring similarity in image generation and image style transfer. A smaller Wasserstein distance indicates a higher degree of similarity between 2 images. In addition, the long and short axes of lesions in real and simulated R-EBUS images were compared. The long axis was the lesion’s longest diameter at the maximum cross section, whereas the short axis was the diameter perpendicular to the long axis.[19] The areas of lesions were also compared. The long axis, short axis, and area were measured using the open-source software 3D Slicer version 5.2.1 (https://www.slicer.org).

Subjective image evaluation

The similarity between lesions in real and simulated R-EBUS images was assessed by 2 experts (Y.L. and J.C.) based on following 5 aspects[19]: size, shape, margin, internal echoes, and overall similarity. For each aspect, a score was assigned: 1 (poor, 0%–20% similarity), 2 (fair, 21%–40% similarity), 3 (moderate, 41%–60% similarity), 4 (good, 61%–80% similarity), and 5 (excellent, 81%–100% similarity). To ensure the reliability of results, 2 experts each performed 2 independent evaluations (1 month apart), and intraobserver agreement was calculated. Interobserver agreement was calculated after each expert independently made their ultimate conclusions.[20,21] Disagreements between 2 experts were discussed with a third expert (X.Z.) to acquire a final consensus. Each expert has more than 500 cases of experience in R-EBUS–guided TBLB.

Statistical analysis

Variables were expressed as mean ± standard deviation (SD), median (interquartile range [IQR]), or number (percentage). For objective indicators, including long axis, short axis, and area, the paired t test or the Wilcoxon signed-rank test was used for paired comparisons. The Shapiro-Wilk test was used to assess the normality of the variables. P values less than 0.05 were considered statistically significant. To examine intraobserver and interobserver agreement, Cohen κ method was applied, with κ values denoting the following levels of agreement: >0.80, almost perfect; 0.61–0.80, substantial; 0.41–0.60, moderate; 0.21–0.40, fair; and 0.00–0.20, slight.[22] All statistical analyses were performed using SAS 9.4 (Cary, North Carolina, USA).

RESULTS

Patient characteristics

Two hundred fifty patients with PPLs who underwent R-EBUS–guided TBLB were enrolled in this study. Clinical characteristics of included patients and lesions are summarized in Table 1. In these patients, a diagnostic yield of 81.6% was obtained. The details of diagnostic information are shown in Supplementary Table 1, http://links.lww.com/ENUS/A360.

Table 1 Clinical characteristics of included patients and lesions.

	Training set (n = 150)	Validation set (n = 50)	Testing set (n = 50)	
Age, y				
 Mean ± SD	62.6 ± 11.3	61.5 ± 12.5	64.1 ± 12.9	
 Median (IQR)	65.0 (55.0–70.0)	63.5 (55.0–69.0)	66.0 (59.0–74.0)	
Sex, n (%)				
 Male	90 (60.0)	28 (56.0)	35 (70.0)	
 Female	60 (40.0)	22 (44.0)	15 (30.0)	
Lesion size, n (%)				
 Mean ± SD, mm	27.4 ± 8.4	27.1 ± 9.1	25.3 ± 8.5	
 Median (IQR), mm	25.6 (21.6–33.7)	26.0 (20.2–32.6)	24.4 (18.4–32.3)	
 >20 mm	120 (80.0)	39 (78.0)	36 (72.0)	
 ≤20 mm	30 (20.0)	11 (22.0)	14 (28.0)	
Lesion appearance, n (%)				
 Solid	132 (88.0)	42 (84.0)	43 (86.0)	
 Part-solid*	18 (12.0)	8 (16.0)	7 (14.0)	
Lesion location, n (%)				
 RUL	60 (40.0)	15 (30.0)	15 (30.0)	
 RML	16 (10.7)	5 (10.0)	2 (4.0)	
 RLL	21 (14.0)	3 (6.0)	12 (24.0)	
 LUL	41 (27.3)	20 (40.0)	17 (34.0)	
 LLL	12 (8.0)	7 (14.0)	4 (8.0)	
Bronchus sign, n (%)				
 Leading to	142 (94.7)	48 (96.0)	47 (94.0)	
 Adjacent to	8 (5.3)	2 (4.0)	3 (6.0)	
*The GGO ratio of these lesions was small (<25%).

IQR: interquartile range; LLL: left lower lobe; LUL: left upper lobe; RLL: right lower lobe; RML: right middle lobe; RUL: right upper lobe; SD: standard deviation.

R-EBUS simulation results

Simulation examples are depicted in Figure 5. Regardless of their size and shape, simulated lesions had well-defined contours. Bronchi and blood vessels outside the target lesion in CT images were not incorrectly identified as lesions in simulated R-EBUS images. Simulated lesions closely mimicked the real ones, and the ultrasound-like noises were well learned using deep learning, giving simulated R-EBUS images a high degree of authenticity and similarity to real R-EBUS images. Also, the R-EBUS probe was simulated, and the probe position relative to the target lesion was well preserved. We also included results from 2 physical ultrasound simulation methods proposed by Zhao et al. and Mattausch et al. for comparison in Figure 5.[13,23]

Figure 5 Simulation examples. A1–A3, Original chest CT data. B1–B3, High-resolution 2D CT sections perpendicular to the biopsy path. C1–C3, Initial simulated R-EBUS images obtained by the ultrasound propagation model. D1–D3, Simulated R-EBUS images after deep learning. E1–E3, Real R-EBUS images. F1–F3, Simulated R-EBUS images obtained using the method proposed by Mattausch et al.[23] G1–G3, Simulated R-EBUS images obtained using the method proposed by Zhao et al.[13] In the first case (the top row), the R-EBUS probe is leading to the lesion (concentric). In the second case (the middle row), the R-EBUS probe is leading to the lesion (eccentric). In the third case (the bottom row), the R-EBUS probe is adjacent to the lesion. 2D, 2-dimensional; R-EBUS, radial endobronchial ultrasound.

Objective image evaluation

In the testing set, by calculating the Wasserstein distance, it was found that utilizing the CycleGAN significantly improved the similarity between real and simulated R-EBUS images. The results also indicate that our method outperformed other traditional ultrasound simulation methods. Lesions in real and simulated R-EBUS images had the following respective dimensions: mean long axis ± SD, 20.2 ± 7.6 mm and 20.1 ± 7.3 mm; mean short axis ± SD, 17.3 ± 7.5 mm and 17.0 ± 7.0 mm; median area (IQR), 223.5 mm2 (115.5–371.2 mm2) and 212.5mm2 (138.2–402.3 mm2). No statistically significant differences were noted (all P > 0.05). More information is displayed in Table 2 and Table 3.

Table 2 Wasserstein distance.

	CT	Step 1*	Step 2†	Zhao‡	Mattausch§	
Wasserstein distance (validation set)	65.0 ± 20.8	27.7 ± 7.3	9.5 ± 5.7	34.0 ± 6.5	34.2 ± 5.8	
Wasserstein distance (testing set)	73.4 ± 15.7	28.8 ± 7.9	10.3 ± 6.1	36.2 ± 10.5	37.8 ± 10.7	
*Initial simulated R-EBUS images obtained by the ultrasound propagation model.

†Our simulated R-EBUS images after deep learning.

‡Simulated R-EBUS images obtained using the method proposed by Zhao et al.[13]

§Simulated R-EBUS images obtained using the method proposed by Mattausch et al.[23]

Table 3 Objective comparison between real and simulated R-EBUS images.

	Validation set	Testing set	
Real images	Simulated images	P	Real images	Simulated images	P	
Long axis, mm							
 Mean ± SD	21.1 ± 4.8	21.1 ± 4.6	0.84	20.2 ± 7.6	20.1 ± 7.3	0.82	
 Median (IQR)	20.8 (17.8–24.7)	21.1 (17.8–23.5)		19.7 (14.5–25.5)	18.9 (14.1–25.1)		
Short axis, mm							
 Mean ± SD	18.2 ± 4.7	17.8 ± 4.6	0.07	17.3 ± 7.5	17.0 ± 7.0	0.25	
 Median (IQR)	18.0 (15.5–21.4)	17.8 (14.8–20.9)		16.0 (12.3–22.6)	16.0 (12.1–21.5)		
Area, mm2							
 Mean ± SD	283.9 ± 123.6	282.3 ± 126.8	0.63	280.3 ± 207.5	281.9 ± 203.4	0.67	
 Median (IQR)	266.2 (191.8–350.2)	250.8 (199.1–377.9)		223.5 (115.5–371.2)	212.5 (138.2–402.3)		
The P value indicates the result of paired t test or Wilcoxon signed-rank test, comparing the objective evaluation results between lesions in real and simulated R-EBUS images.

IQR: interquartile range; R-EBUS: radial endobronchial ultrasound; SD: standard deviation.

Subjective image evaluation

In the testing set, median scores of the similarity between lesion size, shape, margin, and internal echoes in real and simulated R-EBUS images were 5 (IQR, 4–5), 4 (IQR, 4–5), 5 (IQR, 4–5), and 4.5 (IQR, 4–5), respectively, and the overall similarity score was 4.5 (IQR, 4–5). For both intraobserver and interobserver agreement, all aspects had κ values greater than 0.6, indicating that substantial agreement was attained for all 5 aspects [Table 4].

Table 4 Subjective image evaluation between real and simulated R-EBUS images.

	Observer 1	Observer 2	Final score	Interobserver agreement	
Score	Intraobserver agreement	Score	Intraobserver agreement	
Validation set							
 Size	5 (4–5)	0.702	5 (4–5)	0.834	5 (4–5)	0.640	
 Shape	5 (4–5)	0.667	5 (4–5)	0.632	5 (4–5)	0.614	
 Margin	5 (4–5)	0.724	5 (4–5)	0.693	5 (4–5)	0.742	
 Internal echoes	5 (4–5)	0.698	5 (4–5)	0.895	5 (4–5)	0.788	
 Overall similarity	5 (4–5)	0.699	5 (4–5)	0.715	5 (4–5)	0.631	
Testing set							
 Size	5 (4–5)	0.700	5 (4–5)	0.811	5 (4–5)	0.695	
 Shape	4 (4–5)	0.766	5 (4–5)	0.890	4 (4–5)	0.650	
 Margin	4.5 (4–5)	0.895	5 (4–5)	0.890	5 (4–5)	0.822	
 Internal echoes	4 (4–5)	0.739	4.5 (4–5)	0.853	4.5 (4–5)	0.854	
 Overall similarity	4.5 (4–5)	0.888	5 (4–5)	0.894	4.5 (4–5)	0.822	
Scores are presented as median (interquartile range): 1: poor, 0%–20% similarity; 2: fair, 21%–40% similarity; 3: moderate, 41%–60% similarity; 4: good, 61%–80% similarity; 5: excellent, 81%–100% similarity. Intraobserver and interobserver agreement is presented as: κ value: >0.80: almost perfect agreement; 0.61–0.80: substantial agreement; 0.41–0.60: moderate agreement; 0.21–0.40, fair agreement; 0.00–0.20: slight agreement.

R-EBUS, radial endobronchial ultrasound.

DISCUSSION

This study introduced a method based on the CycleGAN to simulate R-EBUS images of PPLs from preoperative CT data. Simulation results showed a satisfactory similarity to corresponding real R-EBUS images. Comparisons between real and simulated R-EBUS images revealed no statistically significant differences in the long axis, short axis, or area of lesions. A median similarity score of ≥4 on a 5-point scale was obtained for lesion size, shape, margin, internal echoes, and overall similarity.

Ultrasound simulation usually adopts physical methods. Wein et al. proposed a physical model of ultrasound propagation to simulate ultrasound images and used CT-ultrasound registration for abdominal surgery.[11] Zhao et al. adapted Wein’s model to simulate EBUS images.[13] To enhance the realism, Mattausch et al. simulated the noise of ultrasound images by Gaussian noise,[23] whereas Zhao et al. added rotating dynamic blur.[13] However, these techniques were somewhat complicated, and the realism of simulated ultrasound images was limited. Moreover, the probe was not included in EBUS simulations, and the evaluation of simulated images was insufficient.

In this article, we utilized the CycleGAN on top of traditional acoustic simulation to make the noise distribution of simulated ultrasound images more realistic. The CycleGAN is a deep learning method developed for unpaired image-to-image translation and has been successfully applied to medical image synthesis. Nevertheless, in terms of intensity distribution, image noise, imaging angle, and field of view, the dissimilarities between ultrasound and CT images are substantial and considerably greater than those among magnetic resonance imaging, CT, and x-ray images. As a result, it is challenging to use a CycleGAN for modality conversion between ultrasound and CT images. To date, only one study has used a CycleGAN to synthesize ultrasound images from CT data, and the simulated images were applied to kidney segmentation.[24] Because R-EBUS images are affected by factors such as probe movement speed, probe pose, and respiratory motion, R-EBUS images and 2D CT sections of one patient cannot be fully matched one by one. Thus, the CycleGAN was chosen in this study. As far as we know, our study is the first to investigate the use of a CycleGAN to simulate R-EBUS images of PPLs from CT data.

Compared with previous studies, the realism of simulated images was further improved by using the CycleGAN. Although substantial dissimilarities exist between ultrasound and CT images, our study demonstrated that it was feasible to use a CycleGAN for modality conversion. The CT images were not directly input to the CycleGAN but first fed into the ultrasound propagation model to construct a transition data set, which alleviated problems associated with the immense discrepancy between ultrasound and CT images. This may explain the remarkable performance of the CycleGAN used in this study. Besides, the probe was taken into account in our simulations. This may be important to the future application because the probe position relative to the target lesion can assist physicians in determining the ideal biopsy location. Both objective and subjective evaluations demonstrated that the simulated images closely mimicked the real ones.

Simulated R-EBUS images of the target lesion could be a useful tool to guide R-EBUS probe manipulation. During preoperative planning, with simulated R-EBUS images of the target lesion synthesized according to different possible biopsy paths, physicians will be able to determine the optimal one. Then, the simulated R-EBUS images, synthesized according to the selected biopsy path, could serve as a reference to guide the R-EBUS probe manipulation during the procedure. By comparing real-time R-EBUS images with simulations, physicians can clearly know whether the ideal biopsy location has been reached.

This study has some limitations. First, this was a single-center retrospective study, which can limit the generalizability of the model. Thus, multicenter and prospective data should be collected to optimize and validate our model. Second, a simulation method integrating an ultrasound propagation model with a CycleGAN was used. Direct simulations using deep learning techniques should also be explored. In addition, the CycleGAN is a classic model for unpaired image-to-image translation. Inspired by the CycleGAN, several succeeding GAN-based models have been proposed, such as U-GAT-IT,[25] ACL-GAN,[26] and CUT.[27] Recently, diffusion models provide an alternative method to unpaired image-to-image translation.[28–30] Whether these models can further improve the quality of simulated images should be studied. Third, the majority of lesions included in this study were solid. GGO-dominant lesions should be collected to investigate the effect of our method on these lesions. Fourth, the R-EBUS probe position was not all confirmed using fluoroscopy during the procedure in this study. However, the interpretation of R-EBUS images of solid lesions is not as easily disturbed by factors such as hemorrhage and atelectasis as GGO, so we believe that almost all of the R-EBUS images that indicated the lesion had been reached were true positive. Last, although the simulated R-EBUS images closely resembled the real ones, their value in clinical practice remains to be determined. In a future study, the effects of our model on the diagnostic performance of TBLB in patients with PPLs will be assessed.

CONCLUSIONS

The proposed method can simulate R-EBUS images of PPLs from preoperative CT data. Objective and subjective evaluations showed a close resemblance between real and simulated images, demonstrating the potential of our method to predict the target lesion appearance in R-EBUS images and provide guidance for intraoperative R-EBUS probe manipulation. Further studies are still needed to assess the value of simulated R-EBUS images in the bronchoscopic diagnosis of PPLs.

Acknowledgments

The authors thank all bronchoscopists who performed the procedures and recorded the intraoperative ultrasonographic videos. The authors also thank all doctors in the Department of Radiology for CT imaging support.

Source of Funding

This study was supported by the National Natural Science Foundation of China (82000104), Science and Technology Commission of Shanghai Municipality (21XD1434400, 22511101500), SJTU Trans-med Awards Research (20210101), and Medical Engineering Cross Fund of MicroPort Scientific Corporation (MP2022Q1B001).

Conflicts of Interest

Jiayuan Sun is an Editorial Board Member of the journal. The article was subjected to the standard procedures of the journal, with a review process independent of the editors and their research group. The other authors declare no conflicts of interest.

Author Contributions

Chunxi Zhang, Yongzheng Zhou, and Chuanqi Sun participated in the performance of the research, the writing of the article, data analysis, and research design. Jilei Zhang and Xiaoyao Liu participated in the performance of the research. Junxiang Chen, Xiaoxuan Zheng, and Ying Li participated in data analysis. Jiayuan Sun and Weiping Liu participated in the critical revision of the article, research design, project administration, and supervision. Jiayuan Sun had primary responsibility for final content. All authors read and approved the final manuscript.

Data Availability Statement

Data described in the manuscript are available from the corresponding author on reasonable request.

Published online: 20 August 2024

C. Z., Y. Z., and C. S. contributed equally to this article.

Supplemental digital content is available for this article. Direct URL citations are provided in the HTML and PDF versions of this article on the journal’s Web site (www.eusjournal.com).
==== Refs
References

1 Chao TY Lie CH Chung YH Wang JL Wang YH Lin MC . Differentiating peripheral pulmonary lesions based on images of endobronchial ultrasonography. Chest 2006;130 :1191–1197.17035455
2 Xie F Yang H Huang R , . Chinese expert consensus on technical specifications of electromagnetic navigation bronchoscopy in diagnosing peripheral pulmonary lesions. J Thorac Dis 2021;13 :2087–2098.34012559
3 Gould MK Donington J Lynch WR , . Evaluation of individuals with pulmonary nodules: when is it lung cancer? Diagnosis and management of lung cancer, 3rd ed. American College of Chest Physicians evidence-based clinical practice guidelines. Chest 2013;143 :e93S–e120S.23649456
4 Ozgul G Cetinkaya E Ozgul MA , . Efficacy and safety of electromagnetic navigation bronchoscopy with or without radial endobronchial ultrasound for peripheral lung lesions. Endosc Ultrasound 2016;5 :189–195.27386477
5 Ali MS Trick W Mba BI Mohananey D Sethi J Musani AI . Radial endobronchial ultrasound for the diagnosis of peripheral pulmonary lesions: a systematic review and meta-analysis. Respirology 2017;22 :443–453.28177181
6 Minezawa T Okamura T Yatsuya H , . Bronchus sign on thin-section computed tomography is a powerful predictive factor for successful transbronchial biopsy using endobronchial ultrasound with a guide sheath for small peripheral lung lesions: a retrospective observational study. BMC Med Imaging 2015;15 :21.26092497
7 Chen A Chenna P Loiselle A Massoni J Mayse M Misselhorn D . Radial probe endobronchial ultrasound for peripheral pulmonary lesions. A 5-year institutional experience. Ann Am Thorac Soc 2014;11 :578–582.24635641
8 Wang Memoli JS Nietert PJ Silvestri GA . Meta-analysis of guided bronchoscopy for the evaluation of the pulmonary nodule. Chest 2012;142 :385–393.21980059
9 Nadig TR Thomas N Nietert PJ , . Guided bronchoscopy for the evaluation of pulmonary lesions: an updated meta-analysis. Chest 2023;163 :1589–1598.36640994
10 Sagar AS Sabath BF Eapen GA , . Incidence and location of atelectasis developed during bronchoscopy under general anesthesia: the I-LOCATE trial. Chest 2020;158 :2658–2666.32561439
11 Wein W Brunke S Khamene A Callstrom MR Navab N . Automatic CT-ultrasound registration for diagnostic imaging and image-guided intervention. Med Image Anal 2008;12 :577–585.18650121
12 Gill S Abolmaesumi P Fichtinger G , . Biomechanically constrained groupwise ultrasound to CT registration of the lumbar spine. Med Image Anal 2012;16 :662–674.21126904
13 Zhao W Ahmad D Toth J Bascom R Higgins WE . Endobronchial ultrasound image simulation for image-guided bronchoscopy. IEEE Trans Biomed Eng 2023;70 :318–330.35819999
14 Kieselmann JP Fuller CD Gurney-Champion OJ Oelfke U . Cross-modality deep learning: contouring of MRI data from annotated CT data only. Med Phys 2021;48 :1673–1684.33251619
15 Wang H Wu Y Huang Z , . Deep learning–based dynamic PET parametric K(i) image generation from lung static PET. Eur Radiol 2023;33 :2676–2685.36399164
16 Couteaux V Zhang C Mulé S , . Synthetic MR image generation of macrotrabecular-massive hepatocellular carcinoma using generative adversarial networks. Diagn Interv Imaging 2023;104 :243–247.36681532
17 Zhao W Bascom R Toth J Higgins W . Guidance system development for radial-probe endobronchial ultrasound bronchoscopy. SPIE Medical Imaging 2019: Image-Guided Procedures, Robotic Interventions, and Modeling 2019;109511P–1–109511P–7.
18 Zhu JY Park T Isola P Efros AA . Unpaired image-to-image translation using cycle-consistent adversarial networks. IEEE Int Conf Comput Vis 2017;2242–2251.
19 Zheng X Wang L Chen J Xie F Jiang Y Sun J . Diagnostic value of radial endobronchial ultrasonographic features in predominant solid peripheral pulmonary lesions. J Thorac Dis 2020;12 :7656–7665.33447458
20 Garcia-Olivé I Radua J Serra P , . Intra- and interobserver agreement among bronchial endosonographers for the description of intrathoracic lymph nodes. Ultrasound Med Biol 2012;38 :1163–1168.22579537
21 Zhi X Chen J Wang L , . Endobronchial ultrasound multimodal imaging for the diagnosis of intrathoracic lymph nodes. Respiration 2021;100 :898–908.34077944
22 Landis JR Koch GG . The measurement of observer agreement for categorical data. Biometrics 1977;33 :159–174.843571
23 Mattausch O Makhinya M Goksel O . Realistic ultrasound simulation of complex surface models using interactive Monte-Carlo path tracing. Comput Graph Forum 2018;37 :202–213.
24 Song Y Zheng J Lei L Ni Z Zhao B Hu Y . CT2US: cross-modal transfer learning for kidney segmentation in ultrasound images with synthesized data. Ultrasonics 2022;122 :106706.35149255
25 Kim J Kim M Kang H Lee K . U-gat-it: unsupervised generative attentional networks with adaptive layer-instance normalization for image-to-image translation. Int Conf Learn Represent 2020.
26 Zhao Y Wu R Dong H . Unpaired image-to-image translation using adversarial consistency loss. Eur Conf Comput Vis 2020;800–815.
27 Park T Efros AA Zhang R Zhu J-Y . Contrastive learning for unpaired image-to-image translation. Eur Conf Comput Vis 2020;319–345.
28 Choi J Kim S Jeong Y Gwon Y Yoon S . Ilvr: conditioning method for denoising diffusion probabilistic models. IEEE/CVF Int Conf Comput Vis 2021;14367–14376.
29 Meng C He Y Song Y , . Sdedit: guided image synthesis and editing with stochastic differential equations. Int Conf Learn Represent 2022.
30 Zhao M Bao F Li C Zhu J . Egsde: unpaired image-to-image translation via energy guided stochastic differential equations. Adv Neural Inf Process Syst 2022;3609–3623.
