
==== Front
PNAS Nexus
PNAS Nexus
pnasnexus
PNAS Nexus
2752-6542
Oxford University Press US

10.1093/pnasnexus/pgae323
pgae323
Biological, Health, and Medical Sciences
AcademicSubjects/MED00010
AcademicSubjects/SCI00010
AcademicSubjects/SOC00010
PNAS_Nexus/cell-bio
PNAS_Nexus/biophys-phys
Mantis: High-throughput 4D imaging and analysis of the molecular and physical architecture of cells
https://orcid.org/0000-0001-6389-0213
Ivanov Ivan E Chan Zuckerberg Biohub San Francisco, San Francisco, CA 94158, USA

https://orcid.org/0000-0002-1016-2447
Hirata-Miyasaki Eduardo Chan Zuckerberg Biohub San Francisco, San Francisco, CA 94158, USA

https://orcid.org/0000-0002-3033-674X
Chandler Talon Chan Zuckerberg Biohub San Francisco, San Francisco, CA 94158, USA

Cheloor-Kovilakam Rasmi Department of Pharmaceutical Chemistry, University of California, San Francisco, San Francisco, CA 94158, USA

https://orcid.org/0000-0001-7482-1299
Liu Ziwen Chan Zuckerberg Biohub San Francisco, San Francisco, CA 94158, USA

https://orcid.org/0000-0002-0926-1480
Pradeep Soorya Chan Zuckerberg Biohub San Francisco, San Francisco, CA 94158, USA

https://orcid.org/0000-0002-8345-8847
Liu Chad Chan Zuckerberg Biohub San Francisco, San Francisco, CA 94158, USA

Bhave Madhura Chan Zuckerberg Biohub San Francisco, San Francisco, CA 94158, USA

https://orcid.org/0000-0002-5551-0417
Khadka Sudip Chan Zuckerberg Biohub San Francisco, San Francisco, CA 94158, USA

Arias Carolina Chan Zuckerberg Biohub San Francisco, San Francisco, CA 94158, USA

https://orcid.org/0000-0002-4881-405X
Leonetti Manuel D Chan Zuckerberg Biohub San Francisco, San Francisco, CA 94158, USA

https://orcid.org/0000-0003-1704-4141
Huang Bo Chan Zuckerberg Biohub San Francisco, San Francisco, CA 94158, USA
Department of Pharmaceutical Chemistry, University of California, San Francisco, San Francisco, CA 94158, USA

https://orcid.org/0000-0002-2542-3582
Mehta Shalin B Chan Zuckerberg Biohub San Francisco, San Francisco, CA 94158, USA

Bahar Ivet Editor
To whom correspondence should be addressed: Email: ivan.ivanov@czbiohub.org (I.E.I.); Email: bo.huang@ucsf.edu (B.H.); Email: shalin.mehta@czbiohub.org (S.B.M.)
I.E.I., E.H.-M., T.C., and R.C.-K. contributed equally to this work.

Competing Interest: The authors declare no competing interests.

9 2024
09 8 2024
09 8 2024
3 9 pgae32326 2 2024
17 7 2024
12 9 2024
© The Author(s) 2024. Published by Oxford University Press on behalf of National Academy of Sciences.
2024
https://creativecommons.org/licenses/by-nc/4.0/ This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial License (https://creativecommons.org/licenses/by-nc/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact reprints@oup.com for reprints and translation rights for reprints. All other permissions can be obtained through our RightsLink service via the Permissions link on the article page on our site—for further information please contact journals.permissions@oup.com.

Abstract

High-throughput dynamic imaging of cells and organelles is essential for understanding complex cellular responses. We report Mantis, a high-throughput 4D microscope that integrates two complementary, gentle, live-cell imaging technologies: remote-refocus label-free microscopy and oblique light-sheet fluorescence microscopy. Additionally, we report shrimPy (Smart High-throughput Robust Imaging and Measurement in Python), an open-source software for high-throughput imaging, deconvolution, and single-cell phenotyping of 4D data. Using Mantis and shrimPy, we achieved high-content correlative imaging of molecular dynamics and the physical architecture of 20 cell lines every 15 min over 7.5 h. This platform also facilitated detailed measurements of the impacts of viral infection on the architecture of host cells and host proteins. The Mantis platform can enable high-throughput profiling of intracellular dynamics, long-term imaging and analysis of cellular responses to perturbations, and live-cell optical screens to dissect gene regulatory networks.

3D dynamic imaging
correlative imaging
smart microscopy
high-throughput screens
viral infection
Chan Zuckerberg Biohub Chan Zuckerberg Biohub National Institute of Health 10.13039/100000002 R01GM131641 Chan Zuckerberg Biohub
==== Body
pmcSignificance Statement

Understanding the dynamics and interactions of cellular components is crucial for biological research and drug discovery. Current dynamic fluorescence microscopy methods can only image a few fluorescent labels, providing a limited view of these complex processes. We developed Mantis, a high-throughput 4D microscope that maps interactions among components of dynamic cell systems. Mantis combines light-sheet fluorescence imaging of multiple fluorophores with quantitative label-free microscopy and is complemented by Smart High-throughput Robust Imaging and Measurement in Python, our open-source software for high-throughput data acquisition and high-performance analysis. Mantis enabled simultaneous 3D time-lapse imaging of 20 cell lines and quantitative analysis of responses to perturbations like viral infection at single-cell resolution. This approach can accelerate the analysis of cellular dynamics and image-based drug discovery.

Introduction

Several open problems in cell biology and drug discovery require high-throughput methods to measure, predict, and model the dynamic interactions among cells, organelles, and proteins. Multiplexed imaging of cellular morphology and machine learning is increasingly used to predict the effects of genetic and chemical perturbations (1–3) from changes in the cellular architecture. High-throughput live-cell imaging, combined with deep learning, has enabled systematic analysis of the dynamic mechanisms that underpin healthy (4–6) and diseased (7–9) states of cells.

Correlative imaging of the 3D architecture of cells and constituent components over time (4D imaging) can accelerate discovery and dissection of cellular processes. However, 4D imaging of more than three organelles with multispectral fluorescence microscopy (10) remains challenging, because engineering cells with multiple fluorescent proteins is labor intensive and the wide emission spectra of fluorescent proteins limit the number of labels that can be imaged simultaneously. Correlative label-free and fluorescence microscopy is a viable strategy to mitigate the longstanding multiplexing bottleneck in 4D microscopy, because label-free imaging captures many cellular structures at once. Several cellular landmarks, e.g. nuclei, cell membrane, and nucleoli, scatter sufficient visible light and can be consistently visualized with label-free imaging. Several organelles, e.g. cytoskeleton, endoplasmic reticulum (ER), and Golgi, as well as individual proteins need to be labeled with fluorophores for consistent visualization as they do not scatter sufficient visible light that can be measured in live cells. Correlative label-free and fluorescence imaging systems have recently been reported (4, 11–14) for 3D imaging of organelles, cells, and tissues.

Building on these advancements, we have developed an automated microscope, named “Mantis,” that synergizes light-sheet and label-free microscopy in multiwell plates. Light-sheet and label-free microscopy are both gentle on live samples, as they do not require high light doses that can cause phototoxicity. We multiplex oblique light-sheet fluorescence microscopy (15–19) with remote-refocus (20, 21) quantitative label-free imaging with phase and polarization (QLIPP) using distinct wavelengths. Mantis enables 4D imaging of three or more fluorescent (i.e. extrinsic) labels and three physical properties of phase, retardance, and orientation as described in Ref. (4) at high-speed and in parallel. The large field of view of the microscope and the combination of two imaging modalities provide rich phenotypic data on the dynamics of specific molecules in the context of the cell architecture. The name “Mantis” is inspired by the high-dimensional vision and the quick reflexes of the mantis shrimp.

Automated, robust, and configurable acquisition software is essential for 4D imaging, especially in high-throughput format. High-throughput correlative microscopes can produce tens of terabytes of data per day. Parallelized, configurable, and reproducible analysis is essential to leverage the statistical patterns captured with these large datasets. We integrate high-throughput acquisition and high-performance computing, streamlining the process of imaging and profiling at high resolution up to a million cells in a single experiment. The acquisition and analysis engine is implemented in Python and is available as an open-source repository shrimPy: Smart High-throughput Robust Imaging and Measurement in Python on GitHub (22). shrimPy streamlines calibration of the optical path, deconvolution of the specimen properties from the acquired intensities, and segmentation of single nuclei and cells. Instead of experimentally staining nuclei and cell membranes, we rely on virtual staining of quantitative phase images (4, 23). Virtual staining of nuclei and membrane frees up fluorescence channels for imaging of proteins and other organelles, improves the imaging throughput, and counteracts the loss of fluorescence due to photobleaching or stochastic labeling. The shrimPy analysis engine uses our recent robust models for joint virtual staining of nuclei and membrane (23).

We demonstrate the performance and applications of Mantis by reporting 3D tracking of organelles and parallel imaging of 20 cell lines expressing distinct fluorescently tagged intracellular reporters—in the context of cellular landmarks that are imaged without labels. We illustrate joint virtual staining of nuclei and membrane, and accurate single-cell segmentation in multiple cell lines. Gentle imaging with Mantis enables analysis of the temporal dynamics of the cell morphology and the localization of key host proteins in response to viral infection. Used together, these tools enable high-content analysis of the dynamics of intracellular reporters in the context of the global morphology of the cell.

Results

Multichannel 4D imaging of cell dynamics

The Mantis microscope is a synergistic combination of oblique light-sheet fluorescence microscopy and remote-refocus QLIPP as illustrated in Fig. 1a and detailed in Materials and methods. The integration of two gentle imaging modalities through wavelength multiplexing enables simultaneous imaging of both the physical and molecular compositions of dynamic biological samples at high throughput in multiwell plates. The label-free measurements enable gentle imaging of many organelles in terms of the distribution of their density and biomolecular orientation. Quantitative label-free imaging provides key landmarks for subsequent segmentation and preserves the fluorescence spectrum for labeled proteins and organelles.

Fig. 1. Overview of the Mantis microscope. a) Schematic of the light path and data acquisition pipeline. Mantis enables high-throughput measurements of the physical and molecular composition of dynamic biological samples in multiwell plates by combining remote-refocus label-free microscopy and oblique light-sheet fluorescence microscopy. Landmark organelles such as nuclei and membranes are virtually stained from label-free images. b) Images of A549 cells in different channels. Fluorescent and virtually stained organelles can be captured in the context of the overall cell architecture. Left to right: labeled mitochondria (TOMM20-GFP) and lysosomes (LysoTracker), label-free measurements of phase and molecular orientation, and nuclei and plasma membrane virtually stained from the phase channel. Top row shows XZ orthogonal projections of data presented in the bottom row at the location indicated by the dotted line. c) Correlative imaging of physical and molecular composition enables 4D imaging of multiple organelles in parallel. Left: overlay of mitochondria (magenta), lysosomes (green), nuclei (white), and membrane (orange); right: overlay of phase (grayscale) and orientation (color). d) Time-lapse imaging enables tracking of organelles. Line segments show the movement of lysosomes in 3D (depth encoded in color) over time. Circles show in-focus spots, and dots show out-of-focus spots.

Long-term imaging of intracellular dynamics and image-based screens in a multiwell plate format are key applications of the Mantis platform. We chose oblique light-sheet fluorescence microscopy as it does not impose additional sample mounting requirements, reduces phototoxicity by only illuminating the plane in the sample that is being imaged, and enables fast volumetric imaging by scanning a galvo mirror (16, 18). The principle of remote-refocus microscopy (15, 20, 21) underpins this design—the oblique light sheet and detection perpendicular to the light sheet are implemented in the remote volume. To enable fast correlative label-free imaging, Mantis speeds up the axial scanning in label-free phase and polarization microscopy (4) by an order of magnitude using a remote-refocus architecture.

The primary objective of the microscope is a 100× 1.35NA silicone immersion lens (O1, Fig. 1a) and is shared by the label-free and fluorescence arms. The remote-refocus path in both arms is implemented using 40× 0.95NA air objective (O2 and O4) and suitable tube lenses to achieve magnification of 1.4×, equal to the ratio of refractive indices of silicone oil and air, as dictated by the principles of remote refocus (20, 21). The remote volume in the fluorescence arm is imaged with a “Snouty” objective (O3), which is perpendicular to the oblique light sheet (17). The remote-refocus objective (O4) in the label-free arm is reused to magnify the remote volume after reflection from a scan mirror. A detailed optical schematic and CAD models of the microscope are available in Fig. 1—Supplementary S1, Fig. 1—Supplementary S2, and Fig. 1—Supplementary S3. In this specific configuration, the Mantis microscope enables volumetric imaging of over ∼15 μm depth without moving the primary objective or the sample because axial scanning is carried out entirely in the remote-refocus arms. The current light path can be configured to image up to 60 μm deep volumes by changing the angle between light sheet and coverslip, and the corresponding angle between the imaging axes of O2 and O3.

Building a microscope that achieves high resolution, high light efficiency, and minimal polarization distortions required the development of the following optical alignment procedures and optical modules: (i) a method for attaching a glass coverslip to the O2 and O4 objectives to compensate for spherical aberrations (Fig. 1—Supplementary S4); (ii) a polarized-light analyzer cube that is insensitive to birefringence of the dichroic beam splitter (Fig. 1—Supplementary S5); (iii) a polarization-based label-free remote-refocus path that maximizes light throughput (Fig. 1—Supplementary S6); (iv) a procedure for calibrating the 3D point-spread function (PSF) of the light-sheet path using beads distributed in 3D agarose gel (Fig. 1—Supplementary Note 1). These procedures correct the majority of the optical aberrations, enable the diagnosis of the alignment, and provide the calibration data needed to deconvolve any residual aberrations. These procedures play a critical role in enabling high-throughput dynamic imaging with diffraction-limited resolution over multiple days, and facilitate re-alignment as needed, typically once a month.

We illustrate the channels of information that can be acquired with Mantis using A549 cells (Figs. 1b and 1—Supplementary Movie 1). We visualized mitochondria and lysosomes by tagging TOMM20 with GFP and staining the cells with LysoTracker Deep Red. The phase channel in the same field of view reports the spatial distribution of biomolecular density and visualizes the nuclei, nucleoli, plasma membrane, and other dense organelles. The orientation channel reports the orientation of angular order (4) among biomolecules and visualizes the plasma membrane and perinuclear fibers that appear similar to vimentin filaments previously reported in this cell line at high confluence (24–26). Lastly, we used virtual staining to predict nuclei and plasma membrane from the phase image (see Virtual staining and segmentation section). Virtual staining of these landmark molecular markers can enable segmentation and single-cell analysis using existing segmentation models built for fluorescence images (23). All channels of information can be acquired in 3D—in Fig. 1b, we show orthogonal slices through the cell body with distinct morphological structures evident throughout the volume.

We overlay fluorescence and label-free images in Figs. 1c and 1—Supplementary Movie 2. We show the entire 150 × 150 µm field of view of the Mantis microscope in all channels of information in Fig. 1—Supplementary S7. We acquired these data at a rate of two volumes per minute, which enabled us to do 3D tracking of lysosomes (Figs. 1d and 1—Supplementary Movie 3).

High-throughput acquisition and analysis

Robust high-content imaging of multiple proteins in the context of the global cell morphology required the development of a dedicated acquisition and analysis engine (Fig. 2) that we named shrimPy (22).

Fig. 2. Automation for high-throughput data acquisition and image processing. a) The shrimPy acquisition engine synchronizes data collection from the two arms of the microscope, acquiring images over time and positions, and is responsible for autofocus and autoexposure. Raw label-free and light-sheet PTCZYX datasets (P: position; T: time; C: channel; Z, Y, and X: three spatial dimensions) are converted to Zarr format for input to the shrimPy analysis engine. Light-sheet data are deconvolved and deskewed, label-free data undergo phase and birefringence reconstruction, and cellular landmarks are predicted using virtual staining of phase images. The processing of the data is accelerated by a high-performance computing (HPC) cluster. Metadata is generated and traced through each step of the pipeline. Data originating from the two arms of the microscope are registered to generate correlative multimodal datasets. b) Deskewing of light-sheet fluorescence volumes. Top left: an oblique light sheet illuminates the sample and defines the raw fluorescence sampling coordinates; after deskewing, we show data in Cartesian coordinates aligned with the objective optical axis. Middle and right: raw and deskewed 3D maximum intensity projections of Argolight sphere target. Bottom: raw and deskewed slices through a representative bead point-spread function. c) Reconstruction of label-free volumes. Top: maximum intensity projections of phase and retardance reconstructions of the Argolight sphere target. Bottom: quantitative phase reconstructions of 100, 250, and 350 nm high glass features; a linear relationship between feature height and reconstructed phase is seen. d) Registration of label-free and fluorescence volumes. Data from different modalities can be registered in 3D by incorporating fiducial markers visible in all channels, by virtual staining of structures also visible in the fluorescence channels, or by applying filtering operations (e.g. Sobel filter) to both datasets to maximize their mutual information. e) Axial stabilization of the label-free and light-sheet volumes. The light-sheet arm is stabilized by a hardware autofocus feedback loop. The label-free arm is stabilized computationally postacquisition by subtracting the focus drift estimated from all positions at a given time point (see Materials and methods).

The shrimPy acquisition engine orchestrates parallel data acquisition from the label-free and light-sheet arms and carries out smart microscopy tasks, such as autofocus and autoexposure. The two arms of the microscope run on two instances of Micro-Manager (27, 28), which we control using the Pycro-Manager Python bridge (29). The shrimPy acquisition engine coordinates their synchronous operation using hardware triggering (Fig. 2—Supplementary S1; see Microscope automation section).

The acquired multidimensional raw datasets are processed by the shrimPy analysis engine to generate registered multimodal data that can be used for phenotyping (Fig. 2a). Raw data are first converted into the OME-Zarr format (30), which enables efficient parallel processing of multiple time points and positions. As described below, discrete data volumes then undergo deskewing of fluorescence channels, reconstruction of phase and orientation, registration, and virtual staining. Parallel processing is accelerated using our high-performance computing cluster (Fig. 2—Supplementary S2).

Light-sheet volumes are acquired in a skewed frame of reference due to the oblique illumination and detection (Figs. 2b and 2—Supplementary S3) and need to be deskewed for visualization in the frame of reference of the imaging chamber and for downstream analysis alongside the label-free data. Figure 2b shows the transformation of a fluorescent sphere target (Argolight) between skewed (here SCAN—TILT—COVERSLIP) coordinates and deskewed (X—Y—Z) coordinates. After deskewing, the target assumes a spherical shape as expected (Fig. 2—Supplementary Movie 1). We also show images of subresolution fluorescent beads in both coordinate systems. These data demonstrate optical resolution of 290 nm in SCAN, 260 nm in COVERSLIP, and 290 nm in TILT; or 290 nm in X, 260 nm in Y, and 680 nm in Z coordinates (see Fig. 2—Supplementary S4 and Fig. 2—Supplementary Movie 2). We improve our resolution and contrast further by applying a bead-based deconvolution routine (see Fig. 2—Supplementary S5), where we average beads to measure the microscope PSF, apply a Tikhonov-regularized least-squares deconvolution, and then deskew the results.

Phase (optical path length), retardance (difference of optical path length between the two symmetry axes of the structure), and orientation (orientation of the axis along which the biomolecular density is higher) are reconstructed from raw brightfield or polarized light images as described earlier (4). The microscope was calibrated for polarized light imaging and the acquired data reconstructed using the recOrder plugin (31) for napari. Figure 2c shows phase and retardance reconstruction of the same Argolight sphere. The target has a different refractive index than its surrounding medium and is weakly birefringent, showing signal in our label-free detection channels. Figure 2—Supplementary S6 further shows projections of the sphere in the three principal planes in each of the three contrast modes. To our knowledge, our work reports quantitative imaging with phase and polarization using remote-refocus acquisition for the first time. In Fig. 2c, glass features of increasing height were imaged to demonstrate that the measured phase increases linearly with the height of features, i.e. with increasing optical path length, as expected. We quantify the transverse (XY) spatial resolution for phase imaging to <400 nm (Fig. 2—Supplementary S7)—note that as in fluorescence imaging, the ability to resolve objects depends on both their density and the detection signal-to-noise ratio, here governed by the difference in refractive index between the objects and their surrounding medium.

Registration between the label-free and fluorescence volumes acquired on the two arms of the microscope is critical for correlative data analysis. Although volume registration can be precalibrated using targets such as Argolight or fluorescent beads embedded in agarose gel, the registration between the two arms can drift over days. However, the registration remains stable during the time required to acquire all positions. To enable robust registration from the acquired volumes, we have developed the following strategies (Fig. 2d): (i) We include fiducial markers such as large fluorescent microspheres that produce strong signals in both the label-free and fluorescence channels as one position in the multiposition acquisition. We used this approach to register data shown in Fig. 1. (ii) In some experiments, fiducial markers are not needed, because molecular markers present in the fluorescence channel can be virtually stained (4, 23) from phase volumes. In these cases, nuclei or plasma membranes are predicted from phase volumes, and the 3D similarity transformation matrix is computed by maximizing mutual information between the experimental and virtual stain (Fig. 2—Supplementary S9). (iii) In experiments where the fluorescence channel encodes membranous organelles, applying image processing filters (e.g. edge detection) to both modalities suffices to achieve sufficient mutual information for registration.

Enabling long-term experiments requires compensating mechanical drift between the image planes of the four objectives of the Mantis microscope (Fig. 2e). The primary objective (O1) can be kept in focus using the built-in Nikon Perfect Focus System. The drift between the O2 and O3 objectives in the light-sheet remote-refocus arm (see Fig. 1a) leads to misalignment of the illumination and detection planes, which leads to substantial degradation of image quality. To maintain the alignment between the O2 and O3 objectives, an image-based autofocus method (Fig. 2—Supplementary S8) is implemented in the shrimPy acquisition engine (Fig. 2e, fluorescence trace). Axial drift between the remote-refocusing objective (O4) and the scan mirror was also observed in the label-free arm. However, the drift in the label-free arm leads to imaging of different depths of the sample rather than a loss of image contrast. Therefore, the drift in the label-free arm is stabilized as a postprocessing step (Fig. 2e, compare phase traces). A hardware image-based autofocus method may also be developed in this arm of the microscope to reduce the need for imaging outside of the axial region of interest.

Live-cell phenotyping

The shrimPy acquisition and analysis engine enables high-throughput correlative measurements of physical and molecular architecture in a multiwell plate format over several hours. Figure 3 shows phase and fluorescence images of 20 cell lines from the OpenCell library (32) expressing a unique endogenously tagged protein and a CAAX-mScarlet reporter of the plasma membrane. Volumetric imaging of fluorescently tagged proteins (Fig. 3—Supplementary Movie 1 and Fig. 3—Supplementary Movie 2) spanning multiple organelles (nuclei, nucleoli, Golgi, ER, endosomes, lysosomes, cytoplasm, stress granules) illustrates the ability of Mantis and shrimPy to map the localization of diverse targets with single-cell resolution.

Fig. 3. High-throughput imaging using the Mantis microscope. Images of 20 OpenCell targets with overlaid phase (grayscale), CAAX-mScarlet labeled membrane (magenta), and split-mNeonGreen2 tagged molecular marker (green) channels. Fluorescent channels are shown as a maximum intensity projection over a 1.1-μm z-section of the sample.

Live-cell phenotyping typically requires segmentation of nuclei and cytoplasm in order to parse the heterogeneity of cellular responses to perturbations. We leverage virtual staining (23, 33) to enable image-based phenotyping with single-cell resolution (Fig. 4). Recently, generalist models for segmenting nuclei (34–36) and membrane (36, 37) of diverse cell types have been developed. They perform well for fluorescence images but require substantial human annotation to adapt to label-free datasets (38). Optimizing these models to segment label-free images would require onerous human annotation effort, especially for 3D segmentation. Virtual staining bypasses the need for human annotations, instead using molecular labels for annotation. We leveraged our recently published robust virtual staining model VSCyto3D (23), and CellPose (36) for single-cell segmentation (see Virtual staining and segmentation section).

Fig. 4. Virtual staining and instance segmentation of nuclei and membrane. a) Comparison of membrane stained with CAAX-mScarlet (magenta, bottom-left half of FOVs) and virtually stained membrane (orange, top-right half of FOVs) in SEC61B and GOLGA2 cell lines (green) indicates reliable virtual staining. Virtually stained nuclei (blue) provide an additional channel of information. b) Virtual staining rescues missing membrane labels. Bounding boxes show cells in the GOLGA2 line that do not express the CAAX-mScarlet label, which is rescued by virtual staining of the membrane. c) Segmentation of virtually stained nuclei and membrane facilitates single-cell analysis of the localization and expression of fluorescently tagged proteins. Contours of the nuclear and membrane masks are shown in cyan and yellow, respectively. d) Examples of virtual staining-based segmentation of nuclei and membrane of the cell lines in the OpenCell library.

VSCyto3D model led to reliable prediction of membrane and nuclei as illustrated in Figs. 4a and 4—Supplementary Movie 1. Note that the VSCyto3D model was trained on deconvolved widefield HEK293T volumes, and generalized to HEK293T volumes acquired with the label-free arm of the Mantis microscope. The model required fine-tuning with 100 FOVs to generalize to A549 data shown in Fig. 1. In some cell lines, a fraction of cells had lost their fluorescent membrane marker. Experimental and virtually stained images of one such field of view (FOV) are shown in Fig. 4b. The virtual staining model, which learns correlations between the two imaging modalities, rescued the missing label. By the same argument, virtual staining models cannot learn uncorrelated noise. Therefore, virtually stained fluorescence images are intrinsically denoised. Thus, virtual staining relaxes the experimental constraints on cell-line engineering and live-cell imaging.

Segmentation of virtually stained nuclei and membranes is shown in Fig. 4c and d for five cell lines in which proteins with diverse localizations are labeled: GOLGA2 (localized to Golgi), NPM1 (localized to nucleoli), SEC61B (localized to ER), POLR1A (RNA Polymerase, localized to nucleoli), and RAB11A (localized to recycling endosomes). To confirm the accuracy of segmentation of virtually stained nuclei and membrane, we imaged the HIST2H2BE cell line expressing fluorescently labeled histone and plasma membrane markers. As shown in Fig. 4—Supplementary S1, the segmentations obtained from the experimental label and virtual staining show high correspondence as measured by common metrics used to evaluate instance segmentation (see Virtual staining and segmentation section).

The modular pipeline consisting of reconstruction, virtual staining, and segmentation, described above enables the analysis of intracellular dynamics across multiple cell lines and perturbations with single-cell resolution.

Analysis of viral infection dynamics

The speed and gentleness of the Mantis microscope enable molecular and morphological profiling of cells over long time periods, even when subjected to perturbations that induce cell stress. To assess this capability, we imaged cells from the OpenCell library that were infected with the common cold human coronavirus hCoV-OC43 (OC43). We chose OC43 as a model virus due to its disease relevance and experimental ease of use (39–41), and we chose to image from 20 to 50 h postinfection (hpi) because cells undergo large molecular changes during those times as measured by comprehensive organellar immunoprecipitation-mass spectrometry (42). In-depth follow-up of these molecular changes, however, remains challenging due to the phototoxicity induced by live-cell imaging using confocal microscopy.

Mantis enables long-term imaging with minimal phototoxicity, and thus allows close monitoring of protein dynamics in response to infection and the identification of image-based viral infection sensors. Given that coronaviruses use the ER to replicate, which leads to eventual activation of the unfolded protein response pathway (40, 41), we monitored protein folding chaperons. We discovered a substantial and consistent change in the localization of the protein folding chaperon HSP90AB1 during infection. We virtually stained cell membrane and nuclei, and segmented cells using CellPose (see Fig. 4 and Materials and methods). This approach allows image-based profiling of changes in protein localization, cell morphology, and cell density with single-cell resolution.

Figure 5 shows the impact of OC43 infection on the cell morphology and the localization of HSP90AB1. The phase images show that infected HSP90AB1 cells are more condensed and have sharper variations in dry mass (Fig. 5a, 40 hpi), relative to the mock condition. This observation is further supported by a decrease in cell number and an increase in the phase interquartile range (IQR), which demonstrates condensation of cell mass that is typically found in distressed or dying cells (Figs. 5b and 5—Supplementary Movie 1). In the fluorescence channel, we observed increased protein condensation and formation of puncta, which we quantified by measuring the skewness of the fluorescence intensity distribution in the cell cytoplasm and the number of puncta per cell (see Calculation of phenotypic features section). To our knowledge, condensation of HSP90AB1 in infected cells has not been directly observed before. The phase IQR and fluorescence skewness measurements are positively correlated and allow clustring of infected and uninfected cells through straightforward Gaussian mixture model using single-cell measurements from a short time window (Fig. 5—Supplementary S1). The clustering algorithm enables classification of infection state of single cells from the phase IQR and fluorescence skewness features across the infection time course (Figs. 5c and 5—Supplementary Movie 2). These data indicate that quantitative phase images and their features may enable label-free readout of the cell's state of infection, enabling high-throughput quantitative analysis of how organelles respond to infection.

Fig. 5. Single-cell phenotyping of OC43 infection. a) Infection state prediction in images of phase (bottom row) and fluorescence (top row, maximum intensity projection over 4.1 µm z-section) in HSP90AB1 control cells (left, mock infection) and cells infected with OC43 β-coronavirus (right, MOI 1) at 24 and 40 hpi. Cell classified as uninfected and infected are marked with blue and red nuclear boundaries, respectively. b) Measurements of phenotypic markers showing changes over the time course of infection. Membrane and nuclei of single cells were segmented after virtual staining as in Fig. 4. The membrane mask was used to calculate the number of cells, cell confluency, cell area (in µm2), the IQR in phase (in milliradians), and the skewness in fluorescence intensity. We further segmented and counted the number of large HSP90AB1 aggregates from fluorescence images (see Materials and methods) and show the average number of puncta per cell over time. c) Classification of infected and uninfected population of cells in mock and infected conditions using Gaussian mixture model. Uninfected cell population is covered by the blue Gaussian ellipse and marked by “+” marker and infected population by orange ellipse and “o” markers.

In addition to the gradual increase in fluorescence skewness and phase IQR, multiple fast changes are observed (Fig. 5b). These changes may be due to shifts in the infection dynamics or measurement uncertainty. The normalized phase IQR of the uninfected subpopulation in the multiplicity of infection (MOI) 1 condition (Fig. 5c) is slightly different from the normalized phase IQR of the uninfected cells. This change may represent stress response in uninfected cells or experimental batch effects across two wells. Disentangling biological heterogeneity from measurement heterogeneity is an important area of future investigation.

Discussion

We described an automated imaging and analysis platform for 4D high-throughput imaging of molecular components and physical properties of cells and organelles. The Mantis imaging system consists of wavelength-multiplexed oblique light-sheet fluorescence microscopy and remote-refocus quantitative label-free microscopy. The shrimPy acquisition and analysis engine enables high-throughput imaging and profiling of intracellular dynamics. We demonstrate the spatial resolution and contrast with test targets, the temporal resolution with dynamic imaging of A549 cells, and the high-throughput imaging capabilities by following 20 cell lines from the OpenCell library over time. Correlative data obtained with Mantis and shrimPy can be leveraged for simultaneous virtual staining of nuclei and membrane. Virtual staining and instance segmentation of nuclei and membrane enabled single-cell analysis and relaxed the experimental constraints on cell line engineering and live-cell imaging. This platform enabled long-term imaging and profiling of changes in protein localization and cell morphology under stress induced by viral infection. Instrument design specifications as well as software for automation, image reconstruction, and virtual staining are shared via open-source repositories. Next, we discuss how we elected to balance the design tradeoffs, promising directions for subsequent technological developments, and key applications of the platform.

We currently acquire fluorescence channels sequentially. The oblique plane illumination geometry maps the oblique slice through the sample volume to a short strip of the camera sensor. By adding an image splitter before the camera that maps fluorescence channels to separate areas of the sensor, two or three emission channels can be imaged in parallel, improving the throughput and temporal resolution. Furthermore, the light-sheet and label-free arms acquire data at the same volumetric rate at present. However, the photon budget in the label-free arm is sufficient to operate it 2–5× faster than the light-sheet arm with few enhancements in automation. By using fast-switching liquid crystal devices (43) to modulate the polarization state of illumination light and improved synchronization sequences, we can drive the components of the label-free arm faster than the light-sheet arm. Lastly, we opted for a silicone immersion objective to minimize spherical aberrations when imaging deep into biological specimens. We can integrate recent design improvements (44) in remote-refocus microscopy for aberration-free imaging with silicon or air immersion with small adjustments to the light path.

The tight integration between acquisition and image analysis creates opportunities for improving the quality and utility of image data with computational microscopy, without increasing the complexity of the hardware. We have reported online alignment of two objectives in the detection path of the light-sheet arm that was essential for keeping the acquisition plane aligned with the illuminated plane of the sample. We can extend the approach to online label-free selection of fields of view with sufficient cell density and long-term tracking of collective cell dynamics.

Although high-throughput analysis is essential for iterative pursuit of biological questions, it remains significantly more challenging than high-throughput imaging. We have made algorithmic and engineering design choices in an attempt to make high-throughput analysis as tractable as high-throughput imaging. We opted for the OME-Zarr format throughout our image reconstruction and analysis pipeline to enable parallel processing of the data and efficient visualization with N-dimensional visualization tools, such as napari (45). All the steps in our analysis workflow are written such that they can be scaled up on a HPC cluster. We developed focus estimation algorithms, PSF calibration algorithms, 3D registration algorithms, and 3D deconvolution algorithms. We integrated 3D virtual staining and 2D instance segmentation of cells as the first step in single-cell analysis. The 3D segmentation and tracking of biological structures remain an exciting open opportunity. We reported long-term imaging, single-cell segmentation, and image-based profiling of infected cells. These image-based phenotypes can now be leveraged to identify label-free markers of infection.

The state-of-the-art imaging and analysis throughput of the Mantis platform creates new opportunities for systematic mapping and analysis of how diseases reprogram cells. For example, we are pursuing systematic mapping of how viral infections or genetic perturbations reprogram organelles by leveraging a subset of fluorescently tagged organelles from the OpenCell library. These dynamic measurements, parsed with suitable machine learning models that leverage temporal dynamics, can clarify mechanisms of disease and lead to informative markers for drug screens.

The throughput of the Mantis platform can be adapted for several other applications. For example, we and others have shown that label-free imaging of antibody-stained (4) and hematoxylin and eosin (H&E) –stained (4, 14) tissue sections can improve the reliability and speed of imaging the tissue architecture. The high-speed correlative imaging capabilities of Mantis platform can accelerate digital pathology studies. Another exciting application is correlative analysis of the dynamics of molecules, organelles, and cells with subsecond temporal resolution. Correlative imaging of signaling dynamics, cell division, and cell differentiation is valuable for addressing open questions in developmental and cell biology. This adaptation can enable study of outstanding questions in how the biological function at higher spatial scales emerges from interactions at finer scales.

Conclusion

We report measurements of molecular dynamics in the context of biomolecular density and orientation using Mantis, a 4D correlative imaging system that integrates label-free and light-sheet microscopy. In combination with an integrated acquisition and analysis engine (shrimPy), Mantis enables scalable analysis of image data with subcellular resolution. The Mantis platform enables longitudinal 3D imaging of multiple cell lines expressing multiple molecular markers in the context of the cellular morphology. We demonstrate the ability to track 3D dynamics of organelles in live cells with high temporal resolution. We also demonstrate long-term imaging and quantitative profiling of changes in the protein localization and cell morphology of infected cells. We anticipate that our approach will enable systematic mapping and analysis of biological processes that govern the health and disease states of cells in diverse biological systems.

Materials and methods

Microscope layout

The Mantis microscope is built around a Nikon Ti2 microscope body. Label-free imaging is performed at 450 nm, and longer wavelengths are used for fluorescence excitation and detection. The two detection arms of the microscope are separated using a dichroic beamsplitter positioned under the objective. The light-sheet remote-refocus arm is built following the design in Sapoznik et al. (18). The remote-refocus arm for label-free microscopy was designed by combining concepts described in Guo et al. (4) and Botcherby et al. (20). A detailed optical schematic is provided in Fig. 1—Supplementary S1, and the mechanical layout of the optical paths is given in Fig. 1—Supplementary S2 and Fig. 2—Supplementary S3. For birefringence imaging, the sample was illuminated with circularly polarized light using a universal polarizer (46) positioned near the back-focal plane of the microscope condenser; when only phase imaging was performed, the sample was illuminated with unpolarized light. For inspection of the sample before imaging, we used the widefield epi-illumination and side detection port built into the microscope body. The primary (O1) objective was kept at a fixed position, and the desired focal plane was selected by moving the sample axially using a piezo plate on the microscope stage or using custom spacers and sample holders when larger translation was needed.

Microscope automation

Data acquisition is accomplished using custom Python software (see Fig. 2a) (22). The shrimPy acquisition engine coordinated parallel acquisition from the two remote-refocus arms of the microscope, collecting data over time and positions, and performed smart microscopy tasks such as autofocus and autoexposure. Each of the remote-refocus arms of the microscope was controlled by an independent instance of Micro-Manager (27, 28) and Pycro-Manager (29), collecting data over channels and z-slices. The two parallel acquisitions were synchronized using trigger pulses generated by a National Instruments DAQ card (NI cDAQ-9178). The exposure pulses of each camera were routed to a dedicated TriggerScope (AVR Optics) which changed the state of hardware associated with that acquisition (see Fig. 2—Supplementary S1).

Sample preparation and imaging

The Argolight target used in this study was model Argo-SIM. The quantitative phase target was purchased from Benchmark Technologies and immersed in water for imaging. To measure the microscope PSF 0.1 μm FluoSpheres beads (Thermo Fisher Scientific, F8803) embedded in 0.5% low melting point agarose (Sigma-Aldrich, A2576) dissolved in 50% (w/w) glycerol in water (47) were excited at 488 nm and imaged using a 525/45-nm bandpass filter.

CRISPR/Cas9 was used to generate endogenously tagged A549 TOMM20-GFP cell line (Fig. 1). Briefly, wild-type A549 cells (ATCC CCL-185) were electroporated with a mixture of Streptococcus pyogenes Cas9 protein (Macrolab), sgRNA targeting the protein of interest (PMID: 35271311 (32), GAGCTTGGCTGAAGATGATG, IDT) and a double-stranded homology-directed repair donor (42) using Amaxa 96-well shuttle nucleofector (Lonza) according to the manufacturer's protocol. Cells were allowed to recover in media with 1 µM nedisertib (M3814; Selleckchem # S8586) for 2 days. Electroporated cells were then expanded and sorted using SONY FACS to sort GFP-positive cells.

HEK293T cell lines from the OpenCell (32) library were labeled with H2B-mIFP and CAAX-mScarlet using standard lentivirus transduction protocols.

Engineered HEK293T and A549 cells were cultured in 37 °C and 5% CO2 and maintained between 20 and 90% confluency. Cells were routinely grown in DMEM with GlutaMAX (Thermo Fisher Scientific, 10566024), 10% fetal bovine serum (Omega Scientific, FB-11), and 100 U/mL penicillin/streptomycin (Thermo Fisher Scientific, 15140163). For imaging, HEK293T cells were seeded on 96-well glass-bottom plates (Greiner Bio One, 655891) coated with 50 µg/mL fibronectin (Corning, 356008, following manufacturer's protocol); A549 cells were seeded on 12-well glass-bottom plates (Cellvis, P12-1.5H-N).

Prior to imaging, the microscope PSF was benchmarked as described in Fig. 1—Supplementary Note 1, and the O1 correction collar was adjusted to minimize the PSF full width at half-maximum. Optionally, we collected datasets of immobilized beads and fluorescein solution for fluorescence deconvolution and flatfield correction.

Cells were imaged in DMEM media without phenol red (Thermo Fisher Scientific, 21063029) supplemented with 10% fetal bovine serum, 5 mM L-glutamine (Thermo Fisher Scientific, 25-030-081), and 100 U/mL penicillin/streptomycin. Cells were maintained at 37 °C and 5% CO2 using an OkoLab stage-top environmental chamber (H301-K-FRAME). Prior to imaging, the cell culture media was supplemented with ProLong Live antifade reagent (Thermo Fisher Scientific, P36975) at 1:66 ratio. A549 cells were stained with 100 nM LysoTracker Deep Red (Thermo Fisher Scientific, L12492). We used polydimethylsiloxane oil with 12500 cSt viscosity (MicroLubrol) as immersion medium for the primary (O1) objective as it provided better coating of the sample chambers. A549 cells were imaged using elliptically polarized light with swing of 0.05 waves as described in (4); HEK293T cells were imaged using unpolarized illumination to reconstruct the quantitative phase.

Virus propagation and infection

OC43 (ATCC, VR-1558) was propagated in Huh7.5.1 cells at 34 °C. To determine viral titers, 800,000 BHK-T7 cells per well were seeded in 6-well plates for 24 h at 34 °C. Virus stocks were then 10-fold serially diluted and applied to cells for 2 h at 34 °C. Media was then replaced with DMEM containing 1.2% Avicel RC-591. Infected cells were incubated for 6 days at 34 °C, then fixed with 4% formaldehyde, and stained with crystal violet for plaque counting. All experiments were performed in a biosafety level 2 laboratory. Aliquots of OC43 stored in −80 °C were used to infect HEK293T cells with a target MOI of 1. One day prior to infection, 8,000 HEK293T cells were seeded on 96-well glass-bottom plates (Greiner Bio One, 655891) coated with 50 µg/mL fibronectin (Corning, 356008, following manufacturer's protocol) in DMEM with GlutaMAX (Thermo Fisher Scientific, 10566024), 10% fetal bovine serum (Omega Scientific, FB-11), and 100 U/mL penicillin/streptomycin (Thermo Fisher Scientific, 15140163). On the day of infection, media was replenished with the same growth media with or without OC43 (viruses were thawed on ice). For MOI calculation, cells were assumed to have tripled. The final volume in a 96-well plate well was 100 µL.

Image processing

Raw fluorescence data were deskewed by applying an affine transformation with trilinear interpolation resampling (Fig. 2—Supplementary S3). The affine transformation was parameterized by two quantities: the ratio of the object-space pixel width to the object-space scan step and the light-sheet tilt angle. These parameters were estimated in three different ways: (i) for Fig. 1 and the beads in Fig. 2b, we used a bead sample imaged before and after known stage motions along the X and Z axes, (ii) for the Argolight target in Fig. 2b, we chose the affine transformation that restored the spherical shape of the target (Fig. 2—Supplementary S6), and (iii) for the remaining figures, we used the transformation estimated from the Argolight target followed by an axial rescaling to account for the mismatched RI = 1.52 index of refraction. Imaging into an index-mismatched target like the Argolight sphere leads to depth-dependent axial scaling (48), so we empirically measured our axial rescaling factor to be ∼1.3. Following deskewing, every three axial slices were averaged, improving SNR without losing resolution (Fig. 2—Supplementary S3).

Optionally, we applied a bead-based deconvolution to our fluorescence data to improve resolution and contrast (Fig. 2—Supplementary S5). As a calibration step, we acquired a volume of beads, found bead patches, and averaged these patches into a single PSF. We then used the averaged PSF to apply a Tikhonov-regularized least-squares deconvolution to our data before deskewing. We show deconvolved and deskewed results in Fig. 1, Fig. 1—Supplementary Movie 1, Fig. 1—Supplementary Movie 2, Fig. 1—Supplementary Movie 3, and Fig. 2—Supplementary S5; elsewhere, we show results without deconvolution. We find our simple bead-averaging method for estimated PSFs performs well on the datasets we tested, but we expect performance to degrade in very dim and sparse samples where inverse modeling techniques are more suitable (49).

Label-free data were reconstructed using recOrder (31). We generated a physics-informed model of the label-free image formation process, providing linear mappings between object properties (phase, retardance, and orientation) and the measured image intensities (4). Retardance and orientation were estimated by applying a pseudoinverse algorithm to five intensity measurements acquired under different polarized illuminations. For A549 data shown in Fig. 1, retardance and orientation were further processed using a 3 × 3 × 5 median filter. Phase was estimated using a Tikhonov-regularized least-squares inverse algorithm applied to brightfield volumes.

Label-free and deskewed fluorescence volumes were registered by choosing the 3D similarity transformation (translation, rotation, and scaling) that maximized the mutual information between virtually stained nuclei or membrane and the corresponding fluorescent target using the Advanced Normalization Tools (ANTS) library (50). The optimization algorithm was initialized with a coarse manually chosen transformation (Fig. 2—Supplementary S9). The resulting transformation was then applied to all label-free volumes in the dataset.

We used a stabilization procedure to eliminate undesired motion in the registered image volumes over time. Label-free volumes were stabilized axially by finding in-focus slices by maximizing the transverse mid-band power for each time point and position, averaging over positions, then applying the averaged shift to each volume (Fig. 2e).

Virtual staining and segmentation

The virtual staining of nuclei and cell membrane from phase was done using the VSCyto3D model (23, 33) fine-tuned using the mantis A549 cells. The fine-tuning dataset was composed of 100 FOVs of (9, 147, and 124 µm) ZYX volumes or (40,985,835) pixels. The model checkpoint with the lowest validation loss value was used for prediction and evaluation. Phase images were normalized to zero median and unit IQR prior to inference. The volumes were predicted in five z-slice sliding windows, and then blended by computing the mean of all windows. We share the models we have used for virtual staining and instance segmentation of nuclei and membrane via our GitHub repository, VisCy (33).

The nuclei and membranes of HEK293T cells were segmented using CellPose, a generalist algorithm for cellular segmentation. The 2D segmentation of the membrane was done using the built-in “cyto2” CellPose model without modifications by providing both the membrane and the nucleus as input channels. The 2D segmentation of the nucleus was done by extending the built-in CellPose “nuclei” model. Our model, “CP_20220902_NuclFL,” uses additional manual annotations on fluorescence images of HEK293T and A549 cells with Hoecht stain (51).

We confirmed the performance of virtual staining models for the segmentation tasks by comparing the instance segmentations of the experimentally labeled nuclei and cell membrane and the virtually stained nuclei and cell membrane (Fig. 4—Supplementary S1). We used a cell line labeled with HIST2H2BE-mNeonGreen and CAAX-mScarlet. We computed DICE coefficient per FOV to assess the degree of overlap between segmented nuclei and segmented cell membrane. We also computed average precision of detecting instances of nuclei or cells for each FOV. In some FOVs, the average precision was found to be low, which turned out to be the FOVs in which some cells were missing the HIST2H2BE-mNeonGreen or CAAX-mScarlet label. Subsequently, these models were applied to segment virtually stained nuclei and membrane and perform single-cell analysis (Figs. 4 and 5).

Calculation of phenotypic features

Phenotypic features shown in Fig. 5 were calculated based on the cell membrane mask and image quantities in the phase or fluorescence channels. The number of cells was calculated as the total count of membrane masks in the nine FOVs acquired for each condition. The confluency was calculated as the fraction of image size covered by membrane masks. Cell area was calculated on a per-cell basis using the membrane mask and averaged across all cells. Fluorescence skewness and phase IQR were similarly calculated on a per-cell basis from the respective channels and averaged across all cells. Puncta in the fluorescence images were detected by intensity thresholding, erosion of the binary mask to exclude very small regions, and applying a determinant of Hessian blob detector using scikit-image.

Infection state classification

Cell infection state, as shown in Fig. 5, was predicted using a Gaussian mixture model. Single-cell HSP90AB1 skew and phase IQR were computed from 4.1 μm maximum intensity projections of the HSP90AB channel and a single slice from phase volumes. Single-cell information was achieved by combining the cell segmentation information for each FOV of imaging, and each condition of imaging.

The Gaussian mixture model was trained on normalized HSP90AB1 skew and normalized phase IQR data from the MOI 1 condition between 41.5 and 43.5 hpi. Gaussian fitting on infected and uninfected cell populations in mock and MOI 1 conditions was performed at the training window (Fig. S5). The infection class was predicted for every cell on images from the training window to visually validate the results.

The model was applied to time indices 24 and 40 hpi (Fig. 5) on mock and MOI 1 conditions. Gaussian fitting of the infected and uninfected populations was also performed on both conditions at both time points. Single cells were labeled as infected or uninfected based on the output of the Gaussian mixture model.

Tracking

The tracking of lysosomes, shown in Fig. 1, was performed using TrackMate 7 in Fiji (52). We used a Laplacian of Gaussians spot detector and the Simple LAP tracker to generate lysosome trajectories.

Supplementary Material

pgae323_Supplementary_Data

Acknowledgments

The authors thank Shivanshi Vaid, Grace Yun, and Vincent Turon-Lagot, Chan Zuckerberg Biohub San Francisco, for preparing cell samples. They thank Rachel Banks for contributing the segmentation models used for segmenting experimentally and virtually stained nuclei and membranes. The authors thank Sandra Schmid for her careful reading of the manuscript and her critical feedback.

Supplementary Material

Supplementary material is available at PNAS Nexus online.

Funding

This research was funded by the Chan Zuckerberg Initiative through the Chan Zuckerberg Biohub, San Francisco. I.E.I., E.H.-M., T.C., Z.L., C.L., M.B., S.K., C.A., M.D.L., and S.B.M. are supported by the intramural program of the Chan Zuckerberg Biohub, San Francisco. B.H. receives support from National Institute of Health (R01GM131641) and is a Chan Zuckerberg Biohub, San Francisco Investigator.

Preprints

This manuscript was posted as a preprint at http://doi.org/10.1101/2023.12.19.572435.

Data Availability

Datasets reported in this manuscript are available from Bioimage Archive at https://doi.org/10.6019/S–BIAD1287. The software used for collecting and processing these datasets is available via our GitHub repository, shrimPy. Models for virtual staining and segmentation of nuclei and membrane are available via our GitHub repository, VisCy.
==== Refs
References

1 Carpenter  AE, Sabatini  DM. 2004. Systematic genome-wide screens of gene function. Nat Rev Genet. 5 :11–22.14708012
2 Feldman  D, et al  2019. Optical pooled screens in human cells. Cell. 179 :787–799.e17.31626775
3 Sivanandan  S, et al  2023 A pooled cell painting CRISPR screening platform enables de novo inference of gene function by self-supervised deep learning. biorxiv 553051. 10.1101/2023.08.13.553051, preprint: not peer reviewed.
4 Guo  S-M, et al  2020. Revealing architectural order with quantitative label-free imaging and deep learning. Elife. 9 :e55502.32716843
5 Kobayashi  H, Cheveralls  KC, Leonetti  MD, Royer  LA. 2022. Self-supervised deep learning encodes high-resolution features of protein subcellular localization. Nat Methods. 19 :995–1003.35879608
6 Viana  MP, et al  2023. Integrated intracellular organization and its variations in human iPS cells. Nature. 613 :345–354.36599983
7 Zaritsky  A, et al  2021. Interpretable deep learning uncovers cellular properties in label-free live cell images that are predictive of highly metastatic melanoma. Cell Systems. 12 (7 ):733–747.e6.34077708
8 Saunders  N, et al  2024. Dynamic label-free analysis of SARS-CoV-2 infection reveals virus-induced subcellular remodeling. Nat Commun. 15 :4996.38862527
9 Petkidis  A, Andriasyan  V, Greber  UF. 2023. Machine learning for cross-scale microscopy of viruses. Cell Rep Methods. 3 :100557.37751685
10 Valm  AM, et al  2017. Applying systems-level spectral imaging and analysis to reveal the organelle interactome. Nature. 546 :162–167.28538724
11 Ivanov  IE, et al  2022. Correlative imaging of the spatio-angular dynamics of biological systems with multimodal instant polarization microscope. Biomed Opt Express. 13 :3102–3119.35774313
12 Schlüßler  R, et al  2022. Correlative all-optical quantification of mass density and mechanics of subcellular compartments with fluorescence specificity. Elife. 11 :e68490.35001870
13 Kandel  ME, et al  2020. Phase imaging with computational specificity (PICS) for measuring dry mass changes in sub-cellular compartments. Nat Commun. 11 :6256.33288761
14 Yeh  L-H, et al  2024. Permittivity tensor imaging: modular label-free imaging of 3D dry mass and 3D orientation at high resolution. Nat Methods. 21 :1257–1274.38890427
15 Dunsby  C . 2008. Optically sectioned imaging by oblique plane microscopy. Opt Express. 16 :20306–20316.19065169
16 Yang  B, et al  2019. Epi-illumination SPIM for volumetric imaging with high spatial-temporal resolution. Nat Methods. 16 :501–504.31061492
17 Millett-Sikking  A, Andrew  A. 2019. High NA single-objective light-sheet. Zenodo. 10.5281/zenodo.3376243.
18 Sapoznik  E, et al  2020. A versatile oblique plane microscope for large-scale and high-resolution imaging of subcellular dynamics. Elife. 9 :e57681.33179596
19 Yang  B, et al  2022. Daxi—high-resolution, large imaging volume and multi-view single-objective light-sheet microscopy. Nat Methods. 19 :461–469.35314838
20 Botcherby  EJ, Juškaitis  R, Booth  MJ, Wilson  T. 2008. An optical technique for remote focusing in microscopy. Opt Commun. 281 :880–887.
21 Millett-Sikking  A, Thayer  NH, Bohnert  A, York  AG. 2018. Remote refocus enables class-leading spatiotemporal resolution in 4D optical microscopy. Zenodo. 10.5281/zenodo.1146084.
22 Ivanov  IE, Hirata-Miyasaki  E, Chandler  T, Mehta  SB. shrimPy: smart high-throughput robust imaging & measurement in Python [Computer software]. https://github.com/czbiohub-sf/shrimPy.
23 Liu  Z, et al  3 June 2024. Robust virtual staining of landmark organelles. biorxiv 596901. 10.1101/2024.05.31.596901, preprint: not peer reviewed.
24 Litwiniec  A, Gackowska  L, Helmin-Basa  A, Zuryń  A, Grzanka  A. 2013. Low-dose etoposide-treatment induces endoreplication and cell death accompanied by cytoskeletal alterations in A549 cells: does the response involve senescence? The possible role of vimentin. Cancer Cell Int. 13 :9.23383739
25 Gagat  M, et al  2018. The effect of piperlongumine on endothelial and lung adenocarcinoma cells with regulated expression of profilin-1. OncoTargets Ther. 11 :8275–8292.
26 Tirino  V, et al  2013. TGF-β1 exposure induces epithelial to mesenchymal transition both in CSCs and non-CSCs of the A549 cell line, leading to an increase of migration ability in the CD133+ A549 cell fraction. Cell Death Dis. 4 :e620.23640462
27 Edelstein  AD, et al  2014. Advanced methods of microscope control using μManager software. J Biol Methods. 1 :e10.25606571
28 Edelstein  A, Amodaj  N, Hoover  K, Vale  R, Stuurman  N. 2010. Computer control of microscopes using µmanager. Curr Protoc Mol Biol. Chapter 14 :Unit14.20.
29 Pinkard  H, et al  2021. Pycro-manager: open-source software for customized and reproducible microscope control. Nat Methods. 18 :226–228.33674797
30 Moore  J, et al  2023. OME-Zarr: a cloud-optimized bioimaging file format with international community support. Histochem. Cell Biol. 160 :223–251.37428210
31 Chandler  T, et al recOrder: 3D quantitative label-free imaging with phase and polarization [Computer software]. https://github.com/mehta-lab/recOrder.
32 Cho  NH, et al  2022. OpenCell: endogenous tagging for the cartography of human cellular organization. Science. 375 :eabi6983.35271311
33 Liu  Z, et al VisCy: computer vision models for single-cell phenotyping [Computer software]. https://github.com/mehta-lab/viscy.
34 Hollandi  R, et al  2020. nucleAIzer: a parameter-free deep learning framework for nucleus segmentation using image style transfer. Cell Syst. 10 :453–458.e6.34222682
35 Schmidt  U, Weigert  M, Broaddus  C, Myers  G. 2018. Cell detection with star-convex polygons. In: Frangi  AF, Schnabel  JA, Davatzikos  C, Alberola-López  C, Fichtinger  G, editors. Medical image computing and computer assisted intervention—MICCAI 2018. Cham: Springer International Publishing. p. 265–273. Lecture Notes in Computer Science.
36 Pachitariu  M, Stringer  C. 2022. Cellpose 2.0: how to train your own model. Nat Methods. 19 :1634–1641.36344832
37 Sheridan  A, et al  2023. Local shape descriptors for neuron segmentation. Nat Methods. 20 :295–303.36585455
38 Edlund  C, et al  2021. LIVECell—a large-scale dataset for label-free live cell segmentation. Nat Methods. 18 :1038–1045.34462594
39 Kim  MI, Lee  C. 2023. Human Coronavirus OC43 as a low-risk model to study COVID-19. Viruses. 15 :578.36851792
40 Prestes  EB, Bruno  JCP, Travassos  LH, Carneiro  LAM. 2021. The unfolded protein response and autophagy on the crossroads of Coronaviruses infections. Front Cell Infect Microbiol. 11 :668034.33996638
41 Liu  DX, Liang  JQ, Fung  TS. 2021. Human coronavirus-229E, -OC43, -NL63, and -HKU1 (Coronaviridae). In: Bamford  DH, Zuckerman  M, editors. Encyclopedia of virology. 4th ed. Academic Press. p. 428–440.
42 Hein  MY, et al  18 December 2023. Global organelle profiling reveals subcellular localization and remodeling at proteome scale. bioRxiv 572249. 10.1101/2023.12.18.572249, preprint: not peer reviewed.
43 Huang  CY, Fung  RX, Lin  YG, Hsieh  CT. 2007. Fast switching of polymer-stabilized liquid crystal pi cells. Appl Phys Lett. 90 :171918.
44 Millett-Sikking  A . 2022. Any immersion remote refocus (AIRR) microscopy. Zenodo. 10.5281/zenodo.7425705.
45 Sofroniew  N, et al napari: a multi-dimensional image viewer for Python [Computer software]. https://github.com/napari/napari.
46 Mehta  SB, Shribak  M, Oldenbourg  R. 2013. Polarized light imaging of birefringence and diattenuation at high resolution and high sensitivity. J Opt. 15 (9 ):094007.
47 Takamura  K, Fischer  H, Morrow  NR. 2012. Physical properties of aqueous glycerol solutions. J Pet Sci Eng.  98–99 :50–60.
48 Loginov  SV, Boltje  DB, Hensgens  MNF, Hoogenboom  JP, van der Wee  EB. 2024. Depth-dependent scaling of axial distances in light microscopy. Optica. 11 :553–568.
49 Liu  S, et al  2024. Universal inverse modeling of point spread functions for SMLM localization and microscope characterization. Nat Methods. 21 :1082–1093.38831208
50 Avants  BB, Tustison  N, Song  G. 2009. Advanced normalization tools (ANTS). Insight J. 2 :1–35.
51 Liu  Z, Hirata-Miyasaki  E, Pradeep  S, Mehta  SB. 2023. Virtual staining and segmentation of nuclei and membrane from quantitative phase. Zenodo. 10.5281/zenodo.10403605.
52 Ershov  D, et al  2022. TrackMate 7: integrating state-of-the-art segmentation algorithms into tracking pipelines. Nat Methods. 19 :829–832.35654950
