
==== Front
Photoacoustics
Photoacoustics
Photoacoustics
2213-5979
Elsevier

S2213-5979(24)00057-0
10.1016/j.pacs.2024.100640
100640
Research Article
A portable photoacoustic microscopy and ultrasound system for rectal cancer imaging
Kou Sitai a
Thakur Sanskar ab
Eltahir Ahmed c
Nie Haolin a
Zhang Yitian ab
Song Andrew a
Hunt Steven R. c
Mutch Matthew G. c
Chapman William C. Jr c
Zhu Quing zhu.q@wustl.edu
ad⁎
a Department of Biomedical Engineering, Washington University in St. Louis, St. Louis, MO 63130, USA
b Imaging Science Program, Washington University in St. Louis, St. Louis, MO 63130, USA
c Department of Surgery, Washington University School of Medicine, St. Louis, MO 63110, USA
d Department of Radiology Washington University School of Medicine, St. Louis, MO 63110, USA
⁎ Correspondence to: Edwin H. Murty Professor of Biomedical Engineering and Radiology, Washington University in St Louis. zhu.q@wustl.edu
14 8 2024
10 2024
14 8 2024
39 10064026 6 2024
4 8 2024
12 8 2024
© 2024 The Authors. Published by Elsevier GmbH.
2024

https://creativecommons.org/licenses/by-nc/4.0/ This is an open access article under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).
Photoacoustic microscopy offers functional information regarding tissue vasculature while ultrasound characterizes tissue structure. Combining these two modalities provides novel clinical applications including response assessment among rectal cancer patients undergoing therapy. We have previously demonstrated the capabilities of a co-registered photoacoustic and ultrasound device in vivo, but multiple challenges limited broad adoption. In this paper, we report significant improvements in an acoustic resolution photoacoustic microscopy and ultrasound (ARPAM/US) system characterized by simulation and phantom study, focusing on resolution, optical coupling, and signal characteristics. In turn, higher in-probe optical coupling efficiency, higher signal-to-noise ratio, higher data throughput, and better stability with minimal maintenance requirements were all accomplished. We applied the system to 19 ex vivo resected colorectal cancer samples and found significantly different signals between normal, cancer, and post-treatment tumor tissues. Finally, we report initial results of the first in vivo imaging study.

Keywords

Acoustic resolution photoacoustic microscopy
Ultrasound
Rectal cancer
Endoscopy
System optimization
ex vivo imaging
in vivo imaging
==== Body
pmc1 Introduction

For the past decade, the incidence of rectal cancer has been increasing for individuals under the age of 50, global incidence of rectal cancer is projected to rise to 1.2 million in 2040 [1], [2]. Modern rectal cancer management is rapidly evolving [3]. First reported in the early 2000s, a watch and wait treatment strategy for select patients has reduced unnecessary surgical resections among selected rectal cancer patients [4], [5], [6]. This approach involves administering radiation and neoadjuvant chemotherapy followed by assessment of tumor response. Patients who respond well to treatment and achieve a complete pathological response (pCR; no residual tumor left) can avoid surgical resection. Thus, accurate assessment of a patient’s clinical response is critical for effective management.

Unfortunately, current standard-of-care tests for post-treatment evaluation – high-resolution magnetic resonance imaging (MRI) and white light endoscopy – have limited resolution and poor capability to differentiate active tumors from post-treatment edema and fibrosis. [7], [8], [9]. An urgent need exists for imaging devices capable of providing accurate and reliable evaluations of patients’ responses to chemoradiation treatment.

Photoacoustic (PA) imaging is an emerging imaging modality that combines optical contrast with ultrasound (US) resolution. The depth of penetration in PA imaging ranges from millimeters to several centimeters depending on the laser power, ultrasound transducer (XDC) frequency, and tissue optical properties [10], [11]. Utilizing acoustic focusing, acoustic resolution photoacoustic microscopy (ARPAM) detects localized PA signals with resolution on the order of 100 µm and penetration depth on the order of 1–2 cm. Co-registered PA and US imaging has shown promise in cancer detection and diagnosis [12], [13].

PA endoscopy imaging, initially implemented in 2009, was performed on animal models to show its imaging capability [14]. Since then, various implementations of photoacoustic endoscopy have been explored, including techniques utilizing all-optical detection, miniature phased arrays, and micro-electromechanical mirrors [15], [16], [17], [18]. Several research groups have successfully demonstrated the potential of PA endoscopy for in vivo imaging of the gastrointestinal tract, but current application is limited to animal models [19], [20], [21]. To test whether ARPAM endoscopy might provide improved response assessment in the post-treatment rectum, we previously developed an ARPAM/US endoscopic imaging system and conducted a pilot study [22]. Our study showcased the system’s potential in discerning complete responders from patients with residual cancers and showed cancerous regions exhibit lower PA signals than normal tissue. Dagheyan et. al found similar initial findings with ex vivo tissue and a benchtop system in 2024 [23].

In this work, we present a second-generation ARPAM/US imaging system suitable for clinical application. It incorporates an optimized optical system and laser coupling, detection electronics, and high data throughput and real-time imaging display. Additionally, the system is portable and can be easily maneuvered in an endoscopy room. To test the system’s performance, we conducted simulations and phantom tests to assess stability, resolution, and signal-to-noise ratio (SNR). We further demonstrated the system’s performance using ex vivo rectal specimens and performed our first in vivo imaging in a patient with treated rectal cancer. To the best of our knowledge, it is the first of this kind suitable for clinical applications in operating rooms and endoscopic imaging suites for a long-term, large-scale clinical trial.

2 Materials and methods

2.1 System and control synchronization

The system, shown in Fig. 1(a) comprises a diode-pumped, Edgewave® electro-optically Q-switched ND:YAG laser operating at 1064 nm, with a 9 ns pulse duration, a 1 kHz pulse repetition rate, and maximum per pulse energy of 10 mJ. The coupling optics (b) is developed and optimized to couple the laser light to an endoscopic imaging probe shown in Fig. 1(e). A Panametrics® US pulser, an on-probe receiving electronic module, and a data acquisition (DAQ) PC with Alazartech® ATS9462 DAQ card are used for both US pulse-echo imaging and PA imaging. A function generator (FG) synchronizes the laser, US pulser, and DAQ receiving circuits for co-registered US and PA imaging. The system is mounted on a 58*64*92 cm 12 U server frame customized with shock-absorbing wheels and complete electrical grounding. The optical components are mounted on a vibration-damped optical breadboard on top of the server frame.Fig. 1 Imaging system construction. (a) Imaging system construction and system components, including a laser head and its power supply, a DAQ PC, a probe, coupling optics, an FG, a US pulse, and power supply and motor driver modules. We mounted all system components on a server frame with an optical breadboard on top, various vibration dampers are used. (b) Shows the optical coupling consisting of a 1064 fundamental laser line mirror, a F125 plano convex lens, an optic window and beam trap, two F-25 plano concave lens, and a custom bare fiber coupler all mounted on a cage system. (c) Prism cage design with half of the aluminum shown transparent, laser coupled from the fiber is reflected by the air-backed prism into the glass rod. (d) Imaging head design with the transducer, housing and drive shaft shown in transparent mode, prism cage is securely mounted in the housing to ensure stable optical coupling. (e) Probe design with the structural parts shown in transparent mode. The amplifier mounted on the middle of the probe handles T/R switching and amplification of the receiving signal. The scanning of the imaging head is achieved by motor rotation driving the drive shaft through a torque coupler, and the rotating fiber clamp relieves fiber torque inside the probe due to rotation.

Fig. 1

The transmission and acquisition system are synchronized via two triggers generated 15.74 µs apart by the FG. The first trigger signal triggers the DAQ and the laser simultaneously. However, the laser has an inherent pulse delay of 850 ns to generate PA signal, and the DAQ starts listening for XDC response 1 µs after the initial trigger. The second trigger signal triggers the US pulser. The A-line data train consists of 2730 points of PA signal followed by 5460 points of US signal, corresponding to 23.4 mm of imaging depth.

2.2 Imaging probe

Shown in Fig. 1(e), the probe consists of a motor and amplifier assembly, a probe handle, an imaging head, and a probe neck we retrofitted using a BK medical® endoscopic ultrasound probe. Two stages of amplifiers were used to improve the system SNR. The first stage amplifier of 26 dB gain was designed in-house with a transmit-receive (T/R) switch to protect the amplifier during transmission. The second stage amplifier was a MiniCircuit® ZX60–43-S+ with 23 dB gain. A 45 MHz low-pass filter was connected between the first and the second stage amplifiers to filter out high-frequency radio frequency (RF) noise. The first stage amplifier was mounted on the drive shaft which rotated as a unit to prevent cable twisting in the front sections of the probe.

Mechanically, a small NEMA 14 motor, through a torque coupler, rotates the imaging head. A silent step ATS 8952 motor driver was used to reduce audible noise and electrical noise of the motor rotation. 3200 micro-steps per rotation ensured the continuous rotation of the motor. Close to the imaging head, shown in Fig. 1(d), a 3D-printed plastic part was force-fitted into the drive shaft to center the optical fiber and seal the drive shaft against water from backflowing and shorting receiving electronics.

The imaging head, designed in Solidworks®, shown in Fig. 1(d), was fabricated with stainless steel. The head housed the US XDC and a prism cage, which uses a 5 mm air-backed 45-degree prism surface to reflect the light, as shown in Fig. 1(c). The fiber is a 910 µm diameter double-cladding multimode fiber from Thorlabs®. A polished 1.8 mm diameter glass rod from McMaster-Carr® was press-fitted into the center of the XDC to guide the light reflected from the prism to imaging targets. The optical coupling efficiency was simulated using Zemanx® and validated with measurements. An acoustic coupling balloon is attached to the front of the probe for coupling sound waves, the imaging head rotates freely inside the balloon.

The US ring XDC was fabricated by Capistrano Labs ® with the specifications of 10 mm outer diameter, 1.8 mm inner diameter, 25 MHz central frequency, and 115 % bandwidth. The XDC’s acoustic properties were also simulated and validated with measurement.

2.3 Optical coupling design

Optical coupling was one of the key issues to guarantee the stability of the system for transporting the system within the clinical space, (see Fig. 1(b)). One 1064 nm laser line fundamental mirrors adjust the laser beam direction for alignment, and A 125 mm plano-convex lens and two 25 mm plano-concave lens comprised a telescope system to reduce the laser diameter from 2 mm to 0.95 mm. This takes advantage of the full fiber aperture while preventing optical focusing inside the fiber. Due to the inherent laser instability at lower powers, we operate the laser at high power and dump excess power to achieve higher laser stability. A thin optical window is used to reduce the laser power by 50 % in the optical path and a beam trap mounted with 3D printed holder absorbs the energy. The entire optical cage system is covered for laser safety (not shown). The measured laser RMS pulse-by-pulse variation at the optimized current setting is 2.1 %. To reduce the complicity of the system, we did not implement pulse-by-pulse monitoring and compensation.

2.4 Optical power simulation and measurement

We used Zemax OpticStudio® software to simulate the imaging system. Due to concerns of laser back reflection, all optics are mounted with the curved surface facing laser entrance, thus sequential simulation was needed to calculate the dimensions for optics mounting. Non-sequential simulation was used to calculate coupling efficiency across the imaging medium.

In the simulation for coupling optics, the simulation is setup with the laser entrance diameter of 2 mm and lenses placed at locations calculated by the sequential simulation. The multimode fiber is simulated as a light pipe with cladding. The diameter of the fiber (910um) and cladding (1000um) is obtained from Thorlabs, the material is modeled as pure silica and fluoride-doped silica as specified in the specifications. The light pipe and cladding are modeled with 3D modeling software and imported to Zemax for simulation, the geometry is modeled with a 180-degree bend with a bending radius of 20 mm for simplicity. In the simulation for the imaging head, the prism is setup in a spherical water medium, and another prism shaped area was defined as air. 1E6 ray tracing was performed to calculate the coupling efficiency for both. A Coherent ® LabMax-Top laser power and energy meter is used to measure the actual output of the system at optical interfaces. These measurement values are compared against simulation results.

2.5 Ultrasound simulation and characterization

We characterized the transducer’s transmission field with simulation and measurement because the transducers have slight differences in geometry and central frequencies due to manufacturing variability. Then simulation parameters were obtained from dimensional measurement of the US XDC, 8.5 mm diameter active area, and 1.8 mm hole, with 25 MHz central frequency and 15 mm focus.

K-wave ultrasound simulation was used to show the acoustic field [24]. Axisymmetric simulation was used for its efficiency in simulating the 2D acoustic transmission field. A simulation step of 10 µm in both the axial and radial direction was used with semi-infinite boundary conditions. We characterized the resolution of the system by linearly translating a 7 µm diameter carbon fiber with a motorized linear motion stage, the carbon fiber is mounted in an C shaped aluminum block and submerged in DI water mixed with 0.4 % intralipid solution. The target was linearly translated over 7 mm range and the PA and US waveforms were recorded. We increased the sampling rate to 0.0225° per step to obtain the resolution of the system.

2.6 System programming and control

The main LabVIEW® control structure of the system is comprised of a producer and consumer loop architecture, which decouples data saving from data generation for improved data streaming. The producer and consumer loops each contains a case structure with a pre-programmed sequence. A queue passes the control information between the producer and consumer loops, but the queue is not used to pass signal data; instead, the DAQ internal memory is used as the data queue to speed up producer loop runtime. And the consumer loop asynchronously manages the system’s data readout, saving, processing, and display. Fig. 2 illustrates this control structure.Fig. 2 System software and control architecture. ARPAM/US system software construction. DAQ PC contains the DAQ and use LabVIEW to interface with hardware and user. A producer and consumer loop architecture is employed, producer loop gets user input, drives the motor and DAQ synchronously to generate data, which is stored in DAQ buffer. And consumer loop reads user control information from producer loop to read data from DAQ buffer asynchronously and take charge of signal processing, saving and display. Three GUI buttons, “start”, “save data?” and “stop” control the program and an emergency stop button can quickly terminate the program.

Fig. 2

The producer loop oversees the data generation of the system, handles user input and real-time update, DAQ acquisition, probe clockwise and anticlockwise rotation. Inside the producer loop, LabVIEW program simultaneously initiates DAQ acquisition by designating a buffer, and starts the analog signal output to the stepper motor driver via a NI PCIE 6251 card, synchronizing motor rotation with data acquisition. The consumer loop manages cases that create the save file, data readout, real-time data processing and display of PA and US images. It also prevents premature termination of the program before data saving completes. An emergency stop to terminate the program is implemented to quickly exit the program without losing data.

Overall, the system achieves automatic control of the scanning for real time data acquisition, processing, display, and saving at a rate of approximately 0.9 Hz with a display delay of 1–2 frames. It is close to the theoretical 1 Hz frame rate limited by the laser pulse repetition frequency. Approximately 1100 PA and US co-registered B-scans are generated for 20 minutes of scanning.

2.7 Contrast agents and gelatin phantom fabrication and imaging

The preparation procedure for the contrast agents used for the gelatin phantom is modified from a previous publication [25]. In summary, glass microspheres with 100 µm diameter are mixed with an equal volume of a hydrophobic dye from American Ink Co®, and the mixture is sonicated at room temperature for 90 minutes. The increased microsphere size allows the contrast agent to offer both acoustic and photoacoustic contrast.

The phantom mixture was produced by suspending 1 %v contrast agent and dissolving 8 %w porcine gelatin in DI water, as well as 1 %v Germall® preservative. The mixture was heated to 40° C and allowed to cool down to 28° C at room temperature before being poured into a 3D-printed tubular mold. The phantom and mold were covered by polyester film to avoid water evaporation and the phantom was allowed to slowly cool down to 4° C in a fridge for 2 hours, after which the phantom was unmolded, an illustration of this process is shown in Fig. 3(a).Fig. 3 Phantom Fabrication and imaging. (a) Phantom preparation procedure. 100 µm glass beads are sonicated with hydrophobic black dye to create the contrast agent, which is mixed with DI water and gelatin to create the phantom mixture. The mixture is poured into a custom phantom mold and allowed to cool to make the tubular phantom. (b) Phantom imaging setup and PA and US images of the phantom. Images of PA and US are equally separated into 10 ROIs and their mean pixel intensity calculated throughout a 30 second imaging sequence.

Fig. 3

For imaging, the phantom is submerged in DI water the ARPAM/US imaging probe is mounted in a clamping mount and the probe imaging head placed into the annule of the phantom for scanning, shown in Fig. 3(b). Using this phantom, we studied the image quality and stability of the system.

2.8 Signal processing pipeline

The signal processing pipeline was developed based on typical ultrasound signal processing procedures with modifications for our specific acoustic and electrical signal characteristics. The raw data from the DAQ is first converted to voltage and FIR filtered, then Hilbert transform-based signal envelop detection is then applied to the voltage signal, followed by log compression and dynamic range adjustments, the image data is plotted using a custom US and PA look-up table, and scan converted to form the radial display. All the signal analysis in this work is based on the FIR-filtered data before Hilbert transform unless specified. The resultant images are then co-registered and scan-converted to radial images for display as shown in Fig. 4.Fig. 4 Optimization of the reconstruction algorithm and processing pipeline. ARPAM/US Processing pipeline and A-line signal. RF data is parsed into PA and US segments apodized and aligned, the RF A-line shows PA and US signal from tissue. The signal envelop is taken to obtain the image data, which is then log-compressed and dynamic range gated. The data is passed through a custom color conversion look-up-table to give PA and US image, the US and PA images are then co-registered, followed by radial conversion to form the radial images representing colorectal tissue scanned.

Fig. 4

To optimize image quality, we performed an exhaustive search of the available frequency bands from 1 to 40 MHz for the tradeoff between SNR and image spot size using the phantom imaging data. The purpose of this filter is to eliminate the low-frequency signals from the amplifier transmit/receive (T/R) switching, low-frequency PA artifacts, and high RF noise coupled from the environment.

2.9 Ex vivo specimen and in vivo patient imaging

All patients eligible and electing to participate in this study signed written informed consent as proved by the Institutional Review Board (IRB# 201707066). From April 2023 to May 2024, a total of 19 colorectal specimens from 19 patients undergoing surgery were imaged (mean age 57, age range 35–77, 7 women, 12 men). Final pathology evaluation revealed pathologic complete response among 3 specimens, while the remainder had viable tumors. Ex vivo rectal tissue specimens were imaged with the setup shown in Fig. 7(a), mimicking in vivo imaging conditions. We selected 6 cancer or pCR regions of interest (ROIs), 6 normal ROIs, and 18 noise ROIs for each ex vivo sample. The RMS (root-mean-square value of PA and US signals) of pre-processed data are analyzed to study the stability of noise and differences between tissue types.

After completion of the ex vivo imaging, we next developed an in vivo imaging protocol. Before standard-of-care surgery, the patient was put under anesthesia, the probe was inserted into the patient's rectum, and the laser was turned on after probe insertion. Laser pulse energy of 3 mJ out of the probe illuminates a tissue area of 12.6 mm2 and the resulting surface optical exposure is 23.87 mJ/cm2. This energy is under the ANSI maximum permissible exposure (MPE) for skin application of 100 mJ/cm2 for a single pulse and an average MPE 47.78 mJ/cm2 for 1 kHz pulse laser at 1024 nm [26]. The probe continuously scans the rectum while the surgeon adjusts the probe insertion location based on real-time image display. Data is continuously displayed and saved.

3 Results

3.1 System characterization

Fig. 5(a) shows the simulated ultrasound transmission field of the 25 MHz transducer, and (b) demonstrates the axial profile and lateral beam pattern, respectively. The acoustic field agrees with typical focused single element XDC with the center hole having pronounced influence in the near field. We used a simple US pulse-echo experiment to characterize the effects on the acoustic field from the glass rod in the center of the XDC. Targets of the test were aluminum blocks submerged in DI water, results are plotted in Fig. 5(b), the signals are shifted slightly for better readability. No significant difference was observed in the pulse-echo signal, indicating the glass rod has minimal to no effect on the ultrasound transmission and receiving fields.Fig. 5 Acoustic simulation and optical coupling of the system. Acoustic simulation and measurement and optical simulation and measurement of the system. (a) 2D axisymmetric acoustic simulation of the 25 MHz XDC, as well as (b) beam patterns and propagation curves. (c) Comparison of the XDC’s pulse echo signal with and without glass rod inserted in the center, showing no difference in acoustic fields. (d) Sequential and non-sequential optical simulation of the coupling optics consisting of a 125 mm plano-convex lens and two 25 mm plano-concave lens, the lenses are separated by 98, 23, and 25 mm as shown in figure, allowing the beam from laser to be focused and fill the numerical aperture of the fiber. (e) Non-sequential simulation of the optical path within the imaging head, simulation is setup in water (not shown) with a prism shaped air backing region. (f) Measured optical power at various system openings compared against simulation, overall coupling efficiency is 38.8 % after the optical window reduces total energy by 50 %.

Fig. 5

In the sequential optical simulation, shown in Fig. 5(d), we manually adjusted the location of the lenses by 1 mm increments to find the optimal location at which the beam entering the fiber was slightly diverging. The optical distances were found to be 98 mm and 23 mm apart respectively between lenses. We confirmed this telescope system consisting of a 125 mm convex lens and two 25 mm plano concave lens reduced the beam from 2 mm diameter to 0.95 mm, allowing 0.4 mm diametric tolerances at the 0.91 mm fiber tip, taking full advantage of the complete fiber aperture.

Zemax simulation evaluated the coupling efficiency. The angle of the glass window was adjusted to 17° to manage the laser power at 50 % without significantly impacting the beam profile at the fiber interface. The coupling efficiencies to the fiber and out of fiber are also evaluated.

The imaging head optical design was simulated to study the coupling efficiency inside the imaging head, and to prevent optical damage to the transducer. The simulation was setup with a collimated laser input coupled into the fiber, exiting the fiber in water, and a glass-air interface reflects the laser into the 1.8 mm diameter glass rod, the 3D simulation profile is illustrated in Fig. 5(c). The simulation shows the coupling efficiency in the imaging head is 83 %.

To confirm the simulation results, optical power was measured at 5 critical system points: output of the laser; input to the fiber; output of the fiber; output of the probe; and output from the balloon. As expected, the optical coupling efficiency before the fiber does not differ from the simulations Fig. 5(d). The differences in optical fiber coupling efficiency are likely due to limitations in manual fiber polishing. Overall, the probe can couple 80 % laser input, and the optical coupling cage system couples 49 %, which is expected since we used the optical window to reduce light by 50 %. The majority of the energy loss happens at the fiber coupling interface, which is also expected since the laser spot is slightly larger than the fiber and no anti-reflection coating is used.

3.2 Carbon fiber phantom imaging for system characterization

We performed carbon fiber phantom imaging with a carbon fiber in an intralipid solution to study the system resolution, temporal and frequency response, and SNR. A single-strand carbon fiber was fixed onto a piece of C-shaped aluminum block for resolution measurements. The aluminum block was secured into a water tank filled with 0.4 % intralipid solution in DI water. The water tank was translated through 7 mm by a 1D motorized linear translation stage. The probe was mounted vertically to image the cross-section of the carbon fiber as shown in Fig. 6(a).Fig. 6 Experimental characterization of the system with phantoms and resolution targets. Experimental characterization of the system (a) imaging setup using 7 µm carbon fiber target in 0.4 % intralipid solution, the water tank is mounted on a 1D motorized translation stage. Stacked PA (b) and US (c) images of carbon fiber and zoomed in PA (d) and US (e) of the smallest focal spot. (f) SNR characterization of the carbon fiber data for PA and US. Time domain signal, signal envelop, and frequency spectrum of US (g) and PA (h) signal from carbon fiber, beam pattern of PA (i) and US (j) signal envelope.

Fig. 6

We reconstructed the images from the resolution measurements and horizontally concatenated them to produce Fig. 6(b) and (c), the PA and US images of carbon fiber. As expected from the US simulation, the focusing is weak in the ultrasound near field. PA intensity is strong throughout the near and focal regions but drops significantly in the far field due to optical scattering in intralipid medium. Images of the smallest focal spot are shown in Fig. 6(d) and (e).

System temporal and frequency responses were evaluated using the data with the smallest focal spot. For the frequency response, US and PA signals are averaged over 3 acquisitions of the carbon fiber at the same location, the frequency spectrum of the raw data is plotted in dB scale in Fig. 6(g) and (h). As expected, the PA signal is broadband, with signal spectrum linearly varying from 0 dB at 2 MHz to −30 dB at 42 MHz; US signal has a center frequency around 25 MHz, with −6 dB band from 17 to 33 MHz.

Axial resolutions of the imaging system were measured from the time domain signal envelopes of the US and PA signals, shown in Fig. 6(g) and (h). The axial resolution of PA and US imaging of carbon fiber was 72 µm and 35 µm respectively. This result is expected since PA signal has lower frequency as seen in the frequency response. The beam pattern of the focal spot is plotted for PA and US in Fig. 6(i) and (j). The lateral resolution of PA and US imaging of carbon fiber, also averaged over 3 acquisitions, was measured at 116 µm and 122 µm respectively, the small difference is likely due to the difference in SNR.

SNR characterization was conducted using the same dataset. An algorithm automatically finds the carbon fiber in the raw data, uses it as the signal region, and finds an adjacent region without signals as the noise region. The RMS value of the signal regions and noise regions are plotted and curve-fitted using a multi-parametric model, shown in Fig. 6(f). The PA signal exhibits a high SNR of 50–53 dB in the near and focal fields and quickly drops off in the far field due to light scattering in intralipid. The US signal has low SNR in the near field, which is expected since the simulation in Fig. 6(a) shows the ultrasound field is very unfocused in the near field. Around focus the US signal has a 28–30 dB SNR, which drops in the far field; again, this agrees with the simulation results. Noteworthily, carbon fiber is a good photoacoustic target but not a good ultrasound reflector, thus the signal amplitude of the ultrasound is much weaker; we expect a higher SNR in phantom and tissue imaging.

3.3 Gelatin phantom imaging for system characterization and processing optimization

Gelatin phantom was submerged in DI water for phantom imaging. An image of the experiment setup is shown in Fig. 3(b). The gelatin phantom image is used to confirm the uniformity of PA and US imaging throughout the 360-degree rotation over a 30-second imaging sequence.

We quantified the variation of US and PA across rotation by dividing the images into 10 ROIs, shown in Fig. 3(b) The US ROI intensity is stable across the image, indicating a more uniform distribution of the US contrast agents. The PA ROI intensity follows roughly a straight line but has more fluctuations, which can also be seen in the PA & US co-registered image. This variation is likely caused by the contrast agents being closer to the probe and therefore getting more laser illumination, which causes a non-uniform distribution as the US and PA contrast are offered by the same contrast agents. The time-dependent variation of mean pixel intensity in the ROIs is also calculated. The US mean variation is 3.5 % and PA is 2.5 %, demonstrating the imaging, phantom is stable throughout the imaging sequence.

One contrast agent spot in the gelatin phantom images was isolated to study the frequency-dependent SNR and 6 dB spot size for optimizing the FIR filter. We went through an exhaustive search of the available frequency band of 0–40 MHz for band-pass filtering and found optimal PA frequency band of 3–30 MHz, US frequency band of 9–22 MHz for processing.

3.4 Ex vivo imaging

ROIs are labeled from pre-processed RF data and compared for 19 ex vivo samples imaged. Supplementary fig.1 shows the variation of PA and US noise ROI RMS intensity. The PA noise is higher than US due XDC artifacts created by laser hitting the transducer surface and RF interference of the signal wire with the laser emission pulses from the laser. The PA noise is still consistently under 4 mV and averages 2.09 mV RMS. US noise is significantly lower, averaging 1.50 mV RMS, and the variation follows that of the PA noise, suggesting that the remaining noise likely comes from RF interference.

Three examples of typical images are shown in Fig. 7(b-d), including P19 normal region, P4 cancer region, and P15 tumor bed region with pCR. The normal tissues show uniform perfusion, especially in mucosa and submucosa region, and scattered vessels in the muscularis and serosa, which is consistent with the histology of the colon and rectum. The cancer US images show clear disruption to tissue layers, also consistent with these tumors. The PA image shows the perfusion within the tumors is greatly reduced, while tissue adjacent to the cancer shows slightly increased hemoglobin distribution. Interestingly, we observed some PA signals inside the cancer, likely corresponding to blood vessels or microcirculation. This observation differs from previous studies, the difference can be attributed to the improved SNR of the ARPAM/US imaging system. From the image of the pCR case, we observe a similar invasion in the US image and the lesion is hypoechoic, but PA shows uniform perfusion similar to that of normal tissue, indicating vascular regrowth and return to normal architecture within the mucosa and submucosa layer.Fig. 7 Ex vivo imaging of colorectal tissue. Ex vivo imaging of colorectal tissue, imaging setup (a). (b) Example of a normal ex vivo scan of a 60-year-old female patient with cancer PA&US co-registered image (left) and US image (right). Normal perfusion can be observed throughout the circumference of the rectal tissue. (c) The patient is a 51-year-old male with adenocarcinoma invading into pericolorectal tissue, which is what we observe in the US image. The PA shows the lesion area has reduced blood perfusion and adjacent tissue has slightly elevated blood perfusion. (d) Example of pCR tissue from a 42-year-old female patient, US image shows the lesion is hypoechoic and invading from mucosa to serosa, the tissue had fibrosis from the pathology. PA image normal perfusion around the legion, which agrees with the pathological observation of regenerative changes. (e) Statistical analysis of ex vivo imaging data, the RMS signal intensity between ROIs of noise, cancer, normal, and pCR tissues are compared with a t-test, all showing statistical significance except comparing normal with pCR for PA images.

Fig. 7

The ROIs from all patients were labeled and separated into normal (N = 19), cancer (N = 16), and pCR (N = 3) categories, and compared the RMS signal intensity in those regions with an unpaired t-test, shown in Fig. 7(e). For US images, all tissue regions are significantly different from each other, which is expected due to morphological differences between tissue types. For PA images, clear differences between noise, cancer, and normal tissue (P << 0.0001), and a statistically significant difference between cancer and pCR tissue (p = 0.0204) are observed. The pCR tissue is not significantly different from normal tissue (P = 0.106), possibly indicating vascular recovery in the tumor bed.

3.5 In vivo imaging

We performed the in vivo imaging in the operating room with our collaborating surgeon controlling the probe (see Fig. 8). The patient, a 63-year-old male with invasive adenocarcinoma involving muscularis propria, had a rectal tumor measuring 1.1 cm by 0.5 cm. From the normal portion of the rectum, Fig. 8(a), we observe a uniform vascular pattern, which is consistent throughout the entire imaging session. Comparing Fig. 8(a) against Fig. 7(d), we see clear similarities between in vivo and ex vivo imaging in which both demonstrate clear tissue layers in US images, and PA images show similar uniform perfusion.Fig. 8 In vivo imaging of colorectal tissue. In vivo US and co-registered PA & US images of (a) proximal normal region and (b) distal cancer region. 63-year-old male with invasive adenocarcinoma involving muscularis propria, the lesion measured 1.1 cm by 0.5 cm in pathology report. PA signal is mostly concentrated in the vasculature rich submucosa, little signal from muscularis propria and scattered vessels in serosa. The cancer shows reduced perfusion in the region. US images show continuous layer structure in normal tissue. Cancer tissue US demonstrates invasion into serosa and disruption of normal tissue layers.

Fig. 8

Tumor imaging, shown in Fig. 8(b), demonstrates clear disruption of tissue layers. The co-registered photoacoustic and ultrasound image shows vascular structures heavily disrupted by the tumor, with adjacent tissues around the tumor bed showing increased vasculature and the tumor bed itself showing reduced blood perfusion. Video 1 shows a 2-minute imaging sequence (speed up 4 times) of this patient. The video shows imaging characteristics as the probe is pulled in and out of the tumor region, a clear transition of normal and cancerous tissues can be easily identified. The surgical pathology confirmed the presence of cancer in the tumor bed.

Supplementary material related to this article can be found online at doi:10.1016/j.pacs.2024.100640.

The following is the Supplementary material related to this article Video S1.Video S1

4 Discussion

In this work, we designed and optimized a compact ARPAM/US system, characterized its performance in phantoms, and demonstrated its capacity in imaging patients treated by radiation and chemotherapy. Various improvements were implemented, including enhancing the mechanical mounting and packaging of the system; using optical simulations in conjunction with computer-aided design to enhance optical stability and reliability; adding on-probe amplifiers and filtering to reduce noise; designing a consistent and reliable data processing pipeline; and programming new triggering schemes and control software to enhance reliability, increase frame rate, and maximize data throughput. To fully assess the system's performance, a custom contrast agent and imaging phantoms were also developed and utilized to characterize the system and optimize data processing.

Overall, through a series of experiments, an improved imaging system and its thorough characterization are presented. Optical coupling simulation and experiment showed the system can couple light more efficiently and reliably. System imaging parameters, such as depth of penetration, SNR, frequency response, and resolution were evaluated. Ex vivo and in vivo imaging showcased the high-quality imaging capabilities of the system across various colorectal tissue types.

In the current system, a hand-held probe has been used for patient study due to surgeons’ familiarity with real-time adjustment of the colorectal scanning probe position with the real-time imaging feedback. 3D scan along the longitudinal direction of the rectum can provide a more comprehensive view of the rectum morphology and vascular structure and potentially enhance the diagnostic capabilities of the ARPAM/US system. We are currently working on an automated 3-D imaging scanning system while giving surgeons the flexibility to perform 2-D scans by hand.

One limitation of our system is the absence of multiple wavelengths for calculating blood oxygen saturation and hemoglobin concentration. This limitation arises from the laser, we are currently experimenting with options to generate a second wavelength to overcome this challenge. Alternatively, another solution would add a laser or switch to a slower OPO tunable laser, but these options will either increase the cost and space to the current system or inevitably reduce the pulse repetition rate for real-time imaging.

The system’s robustness and stability are demonstrated by its successful usage in conducting 19 ex vivo human colorectal sample imaging sessions without the need for major system maintenance over 12 months. Through ex vivo imaging, we quantified discernable signal in cancer regions and found similar PA signals between normal and pCR cases. In our previous pilot ex vivo and in vivo studies [22], [27], we observed decreased PA signal in chemoradiation-treated residual cancerous regions which were mainly caused by treatment-induced scarring and necrosis. These findings were validated from microvessel density (MVD) analysis using immunohistochemistry for CD31 and ERG expression in a small pool of patients with residual cancer, partial response, and pCR [28]. Tumor beds with minimal response to therapy showed large clusters of residual cancer cells with low vascularization. These areas typically showed highly disorganized tissue with obliteration of the layered structure typical of the colon and rectum and lacking the standard arterioles characteristic of the submucosa. Tumor beds with partial response showed regions of treatment response with high vascularization, but most regions with residual viable cancer again showed minimal vascularization at the level of arterioles. Tumor beds of patients with pCR, however, showed a return of the normal, layered structure of the bowel wall with concomitant return of organized arteriolar patterns in the submucosa. The mean MVD of the pCR group is significantly higher than both the partial-responder group (p = 0.0001) and the non-responder group (p < 0.0001). These confirmed findings provide valuable insights into vascular regrowth and cancer in rectal tissue. In the initial patient study reported in this manuscript, we observed a clear difference between normal and cancerous tissues in the patient with residual cancer. This initial result suggests the system’s potential in detecting and differentiating patients with residual cancer from those with complete responses in real-time. More patients are being recruited to further validate the initial results.

The reported portable ARPAM/US imaging system represents significant advancement toward clinical application. The system offers promise for assessing local morphology and functional changes of tumor beds after radiation and chemotherapy for evaluating treatment response and offering optimal surgical management options for rectal cancer patients. Another potential application is to perform serial PAM/US imaging studies during post-treatment surveillance to predict the regrowth of rectal cancer in the setting of a complete clinical response. We anticipate that the dynamic morphologic and vasculature changes of tumor beds over a longer period of time assessed by PAM/US may provide more sensitive detection of tumor regrowth than SOC MRI imaging for patients under non-operative surveillance. In summary, PAM/US may provide surgeons the information they need to answer the most important – and currently most difficult – question while caring for patients with rectal cancer: is radical resection of the rectum needed, or has the medical therapy definitively cured the disease?

Code, Data, and Materials Availability

Simulation and more detailed 3D model as well as bill of material of the system will be available upon reasonable request. Sample data and processing code for the image processing will be available on GitHub (https://github.com/SitaiK/arpamPythonProcessing).

CRediT authorship contribution statement

Yitian Zhang: Methodology, Investigation. Andrew Song: Methodology, Investigation. William C Chapman Jr: Writing – review & editing, Supervision, Project administration, Methodology, Investigation, Funding acquisition, Data curation, Conceptualization. Quing Zhu: Writing – review & editing, Validation, Supervision, Resources, Project administration, Methodology, Investigation, Funding acquisition, Data curation, Conceptualization. Steven R Hunt: Visualization, Validation, Methodology, Investigation, Data curation. Matthew G Mutch: Visualization, Supervision, Project administration, Methodology, Investigation, Funding acquisition, Data curation, Conceptualization. Sanskar Thakur: Writing – review & editing, Visualization, Validation, Methodology, Investigation, Data curation, Conceptualization. Ahmed Eltahir: Writing – review & editing, Visualization, Validation, Methodology, Investigation, Data curation. Sitai Kou: Writing – original draft, Visualization, Validation, Software, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. Haolin Nie: Writing – review & editing, Visualization, Validation, Methodology, Investigation, Data curation.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Dr. Sitai Kou received his BSc degree in bioengineering from the University of Washington, Seattle, Washington, in 2016. He has completed his PhD in July 2024 at Washington University in St. Louis, Missouri, in the biomedical engineering program. His research interests include ultrasound and photoacoustic microscopy and tomography, photoacoustic imaging system development, and quantitative photoacoustic imaging of human tissue.

Sanskar Thakur completed bachelor’s in production engineering at Shivaji University, India, in 2017. He then received his master’s in mechanical engineering from Indiana University-Purdue University, Indianapolis (IUPUI) and worked as a research engineer at Mass General Hospital(MGH). He is Currently Pursuing his PhD in Imaging Science at Washington University in St. Louis, Missouri. His research interests include Photoacoustic imaging, Optical Coherence Tomography, machine learning, and cancer imaging.

Dr. Ahmed Eltahir is a general surgery resident at Washington University in St. Louis. He received his B.S. in Biology from the Massachusetts Institute of Technology and his M.D. from the Johns Hopkins University School of Medicine. His clinical and research interests are the prevention and treatment of colorectal diseases.

Haolin(Taylor) Nie completed her bachelors in biomedical engineering and masters in computer science at Northwestern University, Evanston, Illinois, in 2021. She is currently pursuing her PhD in biomedical engineering at Washington University in St. Louis, Missouri. Her research interests include Photoacoustic imaging, software development, machine learning, and women’s health.

Yitain Zhang completed his bachelor from Oregon State University and MS from Washinton University in St Louis. He is currently pursuing his PhD in Imaging Science at Washington University in St Louis. His research interest is Optical Coherence Tomography.

Andrew Song completed his bachelor from Washinton University in St Louis in 2024. He is pursuing medical school for his next education.

Dr. Steven Hunt is a colorectal surgeon at Washington University in St. Louis. He received his MD degree from Stanford University. His research interests are currently focused on non-operative management of rectal cancer after total neoadjuvant therapy. Dr. Hunt is a member of National Comprehensive Cancer Network committee.

Dr. Matthew Mutch is the chief of colon and rectal surgery of the Surgery Department at Washington University in St. Louis. Dr. Mutch received his MD degree from Washington University in St. Louis. He has been the lead PI at Washington University for several large scale rectal cancer trials such ACSOG Z6051 and ROLAAR.

Dr. Will Chapman Jr is an Assistant Professor of Surgery in the Section of Colon and Rectal Surgery at Washington University in St. Louis, Missouri. He is an active clinician caring for patients with both malignant and benign conditions of the colon, rectum, and anus. Dr. Chapman is actively involved in clinical and translational research focused on rectal cancer imaging, tumor biology, and clinical outcomes.

Dr. Quing Zhu is the Edwin H. Murty Professor of Biomedical Engineering, Washington University in St. Louis. She is also an associate faculty in Radiology, Washington University in St. Louis. Professor Zhu has been named a Fellow of OSA, a Fellow of SPIE, and a Fellow of AIMBE. Her research interests include multi-modality ultrasound, diffuse light, photoacoustic imaging, and Optical Coherence Tomography for breast, ovarian, and colorectal cancer applications.

Appendix A Supplementary material

Supplementary material

Data availability

Data will be made available on request.

Acknowledgments

We thank the funding support of National Institute of Health R01EB034398 and the institutional support of the Siteman® cancer center. And we thank Dr. Konstantin Maslov for the valuable technical consultation on the system development. We also thank study coordinator Asima Badic for screening and consenting patients. We thank Sandra Matteucci for providing editing support.

Appendix A Supplementary data associated with this article can be found in the online version at doi:10.1016/j.pacs.2024.100640.
==== Refs
References

1 Siegel Mph R.L. Colorectal cancer statistics, 2023 CA Cancer J. Clin. vol. 73 3 May 2023 233 254 10.3322/CAAC.21772 36856579
2 Xi Y. Xu P. Global colorectal cancer burden in 2020 and projections to 2040 Transl. Oncol. vol. 14 10 Oct. 2021 101174 10.1016/J.TRANON.2021.101174
3 Biller L.H. Schrag D. Diagnosis and treatment of metastatic colorectal cancer: a review JAMA vol. 325 7 Feb. 2021 669 685 10.1001/JAMA.2021.0106 33591350
4 Dossa F. Chesney T.R. Acuna S.A. Baxter N.N. A watch-and-wait approach for locally advanced rectal cancer after a clinical complete response following neoadjuvant chemoradiation: a systematic review and meta-analysis Lancet Gastroenterol. Hepatol. vol. 2 7 Jul. 2017 501 513 10.1016/S2468-1253(17)30074-2 28479372
5 Bahadoer R.R. Watch and wait after a clinical complete response in rectal cancer patients younger than 50 years Br. J. Surg. vol. 109 1 Dec. 2021 114 120 10.1093/BJS/ZNAB372 34738104
6 Habr-Gama A. Watch and wait approach following extended neoadjuvant chemoradiation for distal rectal cancer: are we getting closer to anal cancer management? Dis. Colon Rectum vol. 56 10 Oct. 2013 1109 1117 10.1097/DCR.0B013E3182A25C4E 24022527
7 Nahas S.C. Pathologic complete response in rectal cancer: can we detect It? Lessons learned from a proposed randomized trial of watch-and-wait treatment of rectal cancer Dis. Colon Rectum vol. 59 4 Apr. 2016 255 263 10.1097/DCR.0000000000000558 26953983
8 Liu S. Can endorectal ultrasound, MRI, and mucosa integrity accurately predict the complete response for mid-low rectal cancer after preoperative chemoradiation? A prospective observational study from a single medical center Dis. Colon Rectum vol. 61 8 2018 903 910 10.1097/DCR.0000000000001135 29944579
9 van der Sande M.E. Response assessment after (chemo)radiotherapy for rectal cancer: why are we missing complete responses with MRI and endoscopy? Eur. J. Surg. Oncol. vol. 45 6 Jun. 2019 1011 1017 10.1016/J.EJSO.2018.11.019 30528891
10 Lihong V. Wang and H. Wu, Biomedical Optics: Principles and Imaging - Lihong V. Wang, Hsin-i Wu - Google Books. 2007. Accessed: Feb. 11, 2021. [Online]. Available: https://books.google.com/books?hl=en&lr=&id=EJeQ0hAB76gC&oi=fnd&pg=PR3&dq=lv+wang+bio+optics&ots=k9egmStVFX&sig=O_9750KGnfTPeZOqBnlOITM97v8#v=onepage&q&f=false.
11 Xu M. Wang L.V. Photoacoustic imaging in biomedicine Review of Scientific Instruments vol. 77 2006 American Institute of PhysicsAIP 041101 10.1063/1.2195024
12 Yao J. Wang L.V. Photoacoustic microscopy Laser Photon Rev. vol. 7 5 Sep. 2013 758 778 10.1002/lpor.201200060
13 Das D. Sharma A. Rajendran P. Pramanik M. Another decade of photoacoustic imaging Phys. Med Biol. vol. 66 5 Feb. 2021 05TR01 10.1088/1361-6560/ABD669
14 Zhou Q. Yang J.-M. Wang L.V. Shung K.K. Maslov K. Yang H.-C. Photoacoustic endoscopy Opt. Lett., Vol. 34, Issue 10, pp. 1591-1593 vol. 34 10 May 2009 1591 1593 10.1364/OL.34.001591
15 Li Y. Lu G. Zhou Q. Chen Z. Advances in endoscopic photoacoustic imaging Photonics 2021, Vol. 8, Page 281 vol. 8 7 Jul. 2021 281 10.3390/PHOTONICS8070281
16 Basij M. Miniaturized phased-array ultrasound and photoacoustic endoscopic imaging system Photoacoustics vol. 15 Sep. 2019 100139 10.1016/J.PACS.2019.100139
17 Ansari R. Zhang E.Z. Desjardins A.E. Beard P.C. All-optical forward-viewing photoacoustic probe for high-resolution 3D endoscopy Light.: Sci. Appl. 2018 7:1 vol. 7 1 Oct. 2018 1 9 10.1038/s41377-018-0070-5
18 Guo H. Xi L. Song C. Xie H. Photoacoustic endomicroscopy based on a MEMS scanning mirror Opt. Lett., Vol. 42, Issue 22, pp. 4615-4618 vol. 42 22 Nov. 2017 4615 4618 10.1364/OL.42.004615
19 Yang J.M. Simultaneous functional photoacoustic and ultrasonic endoscopy of internal organs in vivo Nat. Med. 2012 18:8 vol. 18 8 Jul. 2012 1297 1302 10.1038/nm.2823
20 Xu G. Characterizing intestinal strictures of Crohn’s disease in vivo by endoscopic photoacoustic imaging Biomed. Opt. Express, Vol. 10, Issue 5, pp. 2542-2555 vol. 10 5 May 2019 2542 2555 10.1364/BOE.10.002542
21 Guo T. Xiong K. Zhang Z. Li L. Yang S. In vivo anatomical imaging of colorectum by tens-of-micron-resolved photoacoustic/ultrasonic endoscope Appl. Phys. Lett. vol. 118 15 Apr. 2021 153702 10.1063/5.0049855/39939
22 Leng X. Assessing rectal cancer treatment response using coregistered endorectal photoacoustic and us imaging paired with deep learning,” Radiology vol. 299 2 Mar. 2021 349 358 10.1148/RADIOL.2021202208/ASSET/IMAGES/LARGE/RADIOL.2021202208.FIG6.JPEG 33754826
23 A. Ghanbarzadeh-Dagheyan et al., “Photoacoustic imaging of colorectal cancer in ex vivo samples: a preliminary study,” https://doi.org/10.1117/12.3002767, vol. 12842, pp. 233–238, Mar. 2024, 10.1117/12.3002767.
24 B.E. Treeby and B.T. Cox, “k-Wave: MATLAB toolbox for the simulation and reconstruction of photoacoustic wave fields,” 2010, 10.1117/1.3360308.
25 S. Kou, X. Leng, H. Luo, W.C.C. Jr, M. Mutch, and Q. Zhu, “Acoustic resolution photoacoustic doppler flowmetry for assessment of patient rectal cancer blood perfusion,” https://doi.org/10.1117/12.2578882, vol. 11642, p. 116420U, Mar. 2021, 10.1117/12.2578882.
26 “ANSI Z136.1–2022: Safe Use of Lasers - ANSI Blog.” Accessed: Jun. 03, 2024. [Online]. Available: https://blog.ansi.org/ansi-z136–1-2022-safe-use-of-lasers/.
27 Leng X. Chapman W. Jr Rao B. Nandy S. Chen R. Rais R. Gonzalez I. Zhou Q. Chatterjee D. Mutch M. Zhu Q. Feasibility of co-registered ultrasound and acoustic-resolution photoacoustic imaging of human colorectal cancer Biomed. Opt. Express 9 11 2018 Oct 3 5159 5172 10.1364/BOE.9.005159 PMID: 30460120; PMCID: PMC6238928. 30460120
28 A. Eltahir, H. Nie, S. Kou, C. Marolt, P. Navale, M. Mutch, W. Chapman Jr, Q. Zhu, Histologic and Photoacoustic Evaluation of Rectal Cancer after Neoadjuvant Chemoradiation Using Microvascular Density. Colorectal Disease, revision submitted.
