
==== Front
Biophys Rep (N Y)
Biophys Rep (N Y)
Biophysical Reports
2667-0747
Elsevier

S2667-0747(24)00032-6
10.1016/j.bpr.2024.100173
100173
Article
Advanced analysis of single-molecule spectroscopic data
Botha Joshua L. 1
van Heerden Bertus 123
Krüger Tjaart P.J. tjaart.kruger@up.ac.za
123∗
1 Department of Physics, University of Pretoria, Pretoria, Gauteng, South Africa
2 Forestry and Agricultural Biotechnology Institute (FABI), University of Pretoria, Pretoria, Gauteng, South Africa
3 National Institute of Theoretical and Computational Sciences (NITheCS), South Africa
∗ Corresponding author tjaart.kruger@up.ac.za
02 8 2024
11 9 2024
02 8 2024
4 3 1001736 5 2024
25 7 2024
© 2024 The Author(s)
2024
https://creativecommons.org/licenses/by-nc-nd/4.0/ This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
We present Full SMS, a multipurpose graphical user interface (GUI)-based software package for analyzing single-molecule spectroscopy (SMS) data. SMS typically delivers multiparameter data—such as fluorescence brightness, lifetime, and spectra—of molecular- or nanometer-scale particles such as single dye molecules, quantum dots, or fluorescently labeled biological macromolecules. Full SMS allows an unbiased statistical analysis of fluorescence brightness through level resolution and clustering, analysis of fluorescence lifetimes through decay fitting, as well as the calculation of second-order correlation functions and the display of fluorescence spectra and raster-scan images. Additional features include extensive data filtering options, a custom HDF5-based file format, and flexible data export options. The software is open source and written in Python but GUI based so it may be used without any programming knowledge. A multiprocess architecture was employed for computational efficiency. The software is also designed to be easily extendable to include additional import data types and analysis capabilities.

Graphical abstract

Editor: Jorg Enderlein.
==== Body
pmcWhy it matters

Single-molecule spectroscopy unveils hidden dynamics and properties of biomolecules and other molecular-scale systems with extraordinary selectivity and sensitivity. This technique typically involves multiparameter fluorescence data of, for example, fluorescence brightness, lifetime, and spectra. Simultaneous unbiased analysis of these diverse data sets is challenging, and no fully open-source, unified tool exists to do this. Full SMS is such a tool. It combines powerful multiparameter analysis capabilities with a user-friendly graphical interface while being open source and written in Python. The software is easy to use without any programming knowledge, while being easily extendable and adaptable to additional use cases.

Introduction

Single molecules were measured for the first time by Moerner and Kador (based on absorption) (1) and Orrit and Bernard (based on fluorescence) (2), working at ultracold temperature. Shortly thereafter, room temperature detection of single chromophores was pioneered by the group of Keller (3,4,5), after demonstrating the same approach to a single multichromophoric light-harvesting protein complex back in 1987 (6). Since then, single-molecule spectroscopy (SMS) has developed rapidly. Also called “single-particle spectroscopy” when not applied to strictly one molecule, SMS is applicable to nanoscale systems containing one or a small number of quantum emitters. The exquisite level of selectivity and sensitivity of SMS techniques gives access to numerous properties of nanoscale emitters that are usually masked in conventional ensemble-averaging techniques, thereby offering a broad scope of applications in physics, chemistry, and biology (7). For example, time-dependent SMS measurements provide information about the dynamics and kinetics of nanoscale systems and can resolve time-dependent processes without the need to synchronize these processes in a large collection of identical molecules. The biological applications of SMS are particularly vast and include observations of protein conformational dynamics (8), enzyme reactions (9,10), transcription in single DNA molecules (11,12), light-harvesting complexes switching between different functional states (13,14,15,16), changes in the oligomeric states of macromolecules (17), and changes in the diffusive states of nanoparticles (18). Real-time 3D tracking of biological (macro)molecules allows SMS to be performed with high spatiotemporal resolution in live cells (19). Applications in nonbiological settings include the use of quantum dots (20), single organic molecules (21), and nitrogen-vacancy centers in diamond (22) in photonic quantum technologies and quantum sensing.

Two typical features exhibited in the photon emission from individual nanoscale emitters are photon antibunching and fluorescence intermittency. These phenomena are used to verify the presence of a single quantum emitter but have also found interesting applications. Photon antibunching can be used to count the number of fluorescent labels in confocal microscopy, investigate multiple coupled emitters (by analyzing higher-order photon correlations), and quantify photophysical processes through the width of the antibunching dip (23). While fluorescence intermittency (often called “blinking”) tends to be a hindrance in most applications, it provides a source of information to improve the spatial resolution beyond the diffraction limit (24,25) and, for light-harvesting complexes, is a useful means to study photoprotection (26,27).

SMS is a powerful investigation technique, as the measured data contain a rich amount of information. Numerous approaches have been developed during the past three decades to give access to diverse types of dynamic information about various kinds of nanoscaled systems. However, since the developments have been done across different research groups, each measurement type often requires a different data analysis program, which necessitates the analysis of each measurement type to take place typically in isolation from the rest. In addition, different software packages usually require different file formats and often lack versatility, having been designed for a particular application. As a result, it usually takes a great amount of effort to obtain a multiparameter SMS data set through the combination of different existing software packages.

Although numerous open-source programs give access to their scripts, it is cumbersome to adapt those scripts to enable other applications and is only a viable solution for experienced programmers. Commercial software is also limited in its application, incompatible with other hardware types, and expensive.

There is, therefore, a need for generalized, open-access analysis suites to facilitate and standardize the analysis of multiparameter SMS data. Some work in developing such software has been done, and examples include the Multiparameter Fluorescence Detection (MFD) software from the group of Seidel (28), PAM from the group of Lamb (29), Globals from the group of Gratton (30), iSMS from the group of Birkedal (31), and a change-point analysis (CPA) toolbox from the group of Koenderink (32). A general-purpose file format for SMS and other photon-by-photon data has also been developed, namely Photon-HDF5 (33).

Here, we present Full SMS, a single workspace where data from multiple concurrent measurements of the same subject can be viewed and analyzed. Unlike the MFD, PAM, Globals, and iSMS software mentioned above, our software is largely based on the analysis of intensity time traces using a statistically robust CPA. CPA, along with subsequent clustering of the intensity levels and fluorescence lifetime fitting, is a powerful approach to analyzing fluorescence intermittency and identifying the different emissive states of nanoscale emitters. Full SMS is also entirely written in Python and thus is free to use and extend, setting it apart from MFD and Globals, which are closed source, while PAM and iSMS are open source but written in MATLAB, which is proprietary software. Unlike the toolbox developed by Palstra and Koenderink, our software is based on a graphical user interface (GUI). This has several advantages over simple scripts or command-line programs, the main ones being that easy visual exploration of the data is enabled and that users need very little technical knowledge to get started using the software. In this regard, our software is similar to Glotaran (34,35), a widely used tool for analyzing time-resolved spectra. Additional advantages of Full SMS over the toolbox of Palstra and Koenderink are the display of raster-scan images and spectral time traces, and automated trimming of traces as well as built-in data filtering based on intensity or lifetime distributions. We also developed a custom file format based on the HDF5 format and similar to Photon-HDF5, but which includes specific additional measurement capabilities suitable to measurements of single, immobilized particles. Full SMS is available at http://github.com/BioPhysicsUP/Full_SMS.

Materials and methods

Experimental setup

As described elsewhere (36), a pulsed supercontinuum laser (Fianium, SC400-4-PP) with a repetition rate of 40 MHz was used. The desired central wavelength of excitation depended on the sample (see below) and was determined by an acousto-optic tunable filter (Crystal Technology, Fremont, CA, USA). A combination of a linear polarizer (LPVISB050-MP, Thorlabs, Newton, NJ, USA) and a quarter-wave plate (λ/4 485-630, Achromatic Retarder, Edmund Optics, Barrington, NJ, USA) produced near-circularly polarized laser pulses that were subsequently passed through a spatial filter. The beam was reflected by a dichroic mirror (see below) and focused into a near-diffraction limited laser spot by a 1.45 numerical aperture oil-immersion objective (Plan-Fluor Apo λ 100×, Nikon, Tokyo, Japan). Samples were mounted on a three-axis piezo nanopositioning stage (LPS200, Mad City Labs, Madison, WI, USA). The fluorescence was collected by the same objective and focused onto a 100-μm pinhole to filter out light not originating from within the focal plane. An appropriate fluorescence filter was used to improve the signal/background ratio further depending on the fluorescence wavelength of the sample (see below). The emitted light was split into two beams, 70% of which was reflected and focused onto a single-photon avalanche photodiode (PD-050-CTE, Micro Photon Devices, Bolzano, Italy IRF ∼128 ps) that generated a series of electronic pulses, which in turn was measured by a time-correlated single-photon counting (TCSPC) module (SPC-130-EM, Becker & Hickl, Berlin, Germany). The remaining 30% was transmitted to measure the fluorescence spectra by dispersing the light using a grating (GR25-0608, 600/mm, 750 nm blaze, Thorlabs, Newton, NJ, USA) and thereafter focusing the light onto an electron-multiplying charge-coupled device camera (EMCCD) (iXon3, Andor, Belfast, UK) with an integration time of 1 s.

Samples

Alexa Fluor 647 carboxylic acid (Thermo Fisher Scientific, Waltham, MA, USA) was diluted to ∼25 pM in 6 mM 2-(N-morpholino)ethanesulfonic acid (MES) buffer (pH 7) containing 4% (w/w) poly(vinyl alcohol), and spin coated onto a glass coverslip. The excitation power and wavelength were 200 nW and 633 nm, respectively. The dichroic mirror and fluorescence filter were FF650-Di01-25x36 (Semrock, Rochester, NY, USA) and FELH0650 (Thorlabs, Newton, NJ, USA), respectively.

Qdot 605 (carboxylic acid conjugate) (Thermo Fisher Scientific, Waltham, MA, USA) was diluted to ∼80 pM in 10 mM MES buffer (pH 7) containing 0.2 mM MgCl2 and 0.05% (w/v) Tween-20. A small droplet was applied to a coverslip treated with poly-L-lysine and another coverslip was placed on top. The excitation power and wavelength were 140 nW and 488 nm, respectively. The dichroic mirror and fluorescence filter were 605dcxt (Chroma Technology, Bellows Falls, VT, USA) and 600LPF (Edmund Optics, Barrington, NJ, USA), respectively.

Light-harvesting complex II (LHCII) from spinach, isolated using the protocol in (37), was diluted to ∼3 pM in 20 mM 4-(2-hydroxyethyl)-1-piperazineethanesulfonic acid (HEPES) buffer (pH 8) containing 0.03% (w/v) N-dodecyl-α-D-maltoside and 1 mM MgCl2. A small droplet was applied to a coverslip treated with poly-L-lysine, and the coverslip was placed in a custom sample chamber filled with an additional buffer-detergent-salt solution. To remove oxygen and prolong measurement times, a glucose/glucose oxidase/catalase mixture was used along with flushing of the sample chamber with N2 gas. The excitation power and wavelength were 119 nW and 633 nm, respectively. The dichroic mirror and fluorescence filter were TX660 and ET665lp, respectively (both from Chroma Technology, Bellows Falls, VT, USA).

Results and discussion

Full SMS is a GUI-based application that allows users to open, view, and perform analyses of multiple types of SMS measurements. Written in Python, it uses a Python binding of the Qt software suite for the GUI functionality, which produces a cross-platform application. A multiprocess architecture was used to allow for parallel computation operations, which is notoriously challenging in Python due to the so-called global interpreter lock, but it is crucial to allow for the quick analysis of large data sets. The measurements are stored in the HDF5 format, developed by the HDF Group, which allows for efficient storage, a hierarchical-relational grouping of measurements with rich meta-data capabilities.

All the analyses described below as well as subsequent exporting of the results can be performed fully within the GUI, with no programming knowledge needed from the user. Fig. S1 (supporting material) shows a screenshot of the main window of the GUI.

File format

Full SMS is designed to analyze data sets stored in the HDF5 format, with a specific custom structure. This means that any data not in this format has to be converted before it can be opened. Currently, the software has built-in conversion tools for converting PicoQuant’s .pt3 format, as well as plain text .csv files, containing time-tagged time-resolved (TTTR) photon data, along with plain text spectral time traces. In principle, a wide range of data formats can be converted; however, a fundamental requirement is that the intensity data consist of photon arrival times and not prebinned traces or traces obtained by gated detection (see intensity-level resolution). Currently, up to two photon channels are supported. Our software provides for the display of raster-scan images and spectra; however, these are not required to be present in the data set for Full SMS to open it.

Analysis

We first give a summary of the analyses and tools that the software provides and follow this with a discussion of the main features, using mainly the data from one measured particle as a stepwise illustration of the analysis capabilities. For detailed information, including practical usage instructions, see the user guide at https://up-biophysics-sms.readthedocs.io/en/latest/. As the primary use case of the software involves point measurements on individual particles, each individual measurement set is called a “particle” in our software, as well as in the discussion below (this is also reflected in the name Full SMS).

Many kinds of analyses can be performed on SMS data, depending on the type(s) of measurements made. A predominant SMS measurement modality is fluorescence intensity (also called “brightness”), either directly by measuring the photon flux within predetermined time windows or indirectly by measuring photon arrival times and then calculating the intensity in time bins (see intensity-level resolution). The latter, called TCSPC, typically provides a statistical distribution of photon arrival times relative to the moment of excitation of the sample by a pulsed light source, a method commonly known as TTTR data collection. Further analysis allows the resolved intensity levels to be grouped by employing a clustering algorithm, and the statistically most probable number of states to be determined (see intensity-level grouping). Full SMS additionally enables globally grouping over the resolved intensity levels from multiple measurements to resolve underlying states with greater confidence (see intensity-level grouping). Full SMS can analyze TCSPC data to extract fluorescence lifetimes for the whole measurement, but also for each resolved intensity level or intensity group (see fluorescence decay fitting). It can also calculate the second-order photon correlation function g(2)(τ) (see second-order correlation function).

Additional features, available in separate tabs, are data filtering options (see data filtering), the display of measured fluorescence spectral sequences and raster-scan images, and options for exporting the analyzed data (see display of spectra, display of raster-scan images, and data export). Raster-scan images are useful for the spatial mapping of the measured particles in concurrent imaging modalities or simply to locate individual particles before measurement.

The software also provides several other features (see additional tools), such as data format conversion, photon burst detection, and the capability to define regions of interest (i.e., trimming of intensity traces) for each measurement individually, either manually or procedurally, by defining intensity and temporal thresholds.

Intensity-level resolution

Analysis of an emitter’s intensity trace is useful not only to assess its intrinsic brightness but especially to investigate time-dependent processes giving rise to fluorescence intensity fluctuations, which may result from static or dynamic quenching, photobleaching, or other photophysical changes. Among these, fluorescence intermittency, a telltale sign of a single emitter, is demonstrated by abrupt and reversible decreases in the photon emission rate for periods anywhere between submilliseconds to tens of minutes (38,39). To correctly analyze intensity fluctuations, the points at which the intensity changes occur must be identified. An additional benefit of such a capability is to filter out unwanted segments of the intensity trace, for example, those corresponding to photobleaching.

Identifying the points at which a change in the photon statistics occurs is typically done using CPA—an approach to locate abrupt changes in a time-dependent variable such as emission intensity (40,41). This is commonly used to analyze fluorescence intermittency, as in this work, but can also be used, e.g., to detect changes in the diffusion behavior of a single particle (42). CPA can be done using time-binned intensity traces or directly using the photon arrival times. Working in the intensity regime (the first approach) is often done to simplify the analysis workflow, and suitable CPA algorithms have been developed (43,44,45). The choice of bin size is crucial in this case as it determines the temporal resolution with which dynamics can be resolved. However, data binning inevitably introduces a bias in the analysis (40,41,46,47). This bias is especially relevant when the dynamics of the intensity fluctuations occur at a rate comparable with the chosen bin size.

Change-point detection in the temporal regime avoids the bias introduced by data binning by working with the individual photon arrival times. This enables the optimal amount of information retrieved from a photon stream, limited only by the shot noise. We opted for the CPA method introduced in (40) because of its independence of a physical model underlying the photon statistics. This method compares the duration between each pair of consecutive photon detections to the durations between neighboring pairs of photon detections and assigns a change point to the instant when a sustained abrupt change in these durations occurs. A chosen confidence value determines a statistical criterion for each potential change point that must be met before acceptance. The higher the confidence value, the more stringent the criteria that need to be met, and the less sensitive the change-point detection becomes. In addition, each change point has an associated temporal error region, defined by the first and last detected photons that pass the statistical test. The resulting change points separate the measurement into segments.

After loading a measured data set, an intensity trace is displayed in the first tab of Full SMS with a corresponding intensity histogram on the right (see Fig. S2). In this tab, CPA can be performed on the data. The screenshot in Fig. 1 is an example of a measurement of single Alexa Fluor 647 (hereafter called Alexa) dye molecules—a common dye used for microscopic chromophore labeling—that was subjected to CPA. The green trace represents the binned intensities calculated from the measured time series, whereas the black trace represents the resolved levels. It is worth noting that the CPA intensity level resolution is independent of the bin size used for the intensity representation, as the CPA operates on the underlying time series. An additional functionality is the display of two-channel data (Fig. S3). Currently, the two channels are analyzed separately, except for second-order correlation analysis (see second-order correlation function).Figure 1 Screenshot of a resolved fluorescence brightness trace of a single Alexa Fluor 647 dye molecule, with trimmed region of interest (ROI) shown in green. See text as well as the user guide for details.

Fluorescence decay fitting

To extract fluorescence lifetime information, the photon arrival times relative to the excitation laser pulse are histogrammed and fitted in an iterative reconvolution minimization procedure, which is based on either the least-squares (LS) (48) or maximum-likelihood (ML) (49) approaches. The approaches give identical results for high photon counts, and in such cases LS is preferred since it is more computationally efficient and robust. For low photon counts, ML should be used since it correctly accounts for Poisson noise (49). Lifetime fitting is performed within the Lifetime tab of the software (Fig. 2). A fit can be performed on an entire experimental trace or individual resolved levels using only the photons from the level. After performing the grouping, a fit can also be done for each level group, as each intensity state is typically associated with a specific lifetime. This is especially powerful for dim levels or data with very fast switching where the individually fitted levels do not contain many photons from which to extract a lifetime.Figure 2 Screenshot of an example lifetime fit corresponding to a single resolved brightness level of the Alexa trace shown in Fig. 1. Fitting residuals are shown in the lowest pane and fitting results are on the right. See text and user guide for details.

An interactive dialog is used to choose the parameters for the fitting (Fig. 3). Multiexponential fitting with up to three components is supported, and either a measured or simulated Gaussian IRF can be used, and decay and IRF background can be manually specified or determined automatically. After performing the fit(s), the results are shown in the Lifetime tab (Fig. 2). The fits are shown along with the residuals, if enabled, and the fitting parameters and goodness-of-fit statistics are shown in the right pane. To evaluate the goodness-of-fit, we use the reduced χ2 value (only for LS) and the Durbin-Watson (DW) parameter (50) (for both LS and ML). The latter measures autocorrelation in the residuals, which is more sensitive to minor fitting errors than χ2. Our code also provides tools to automatically identify suitable boundaries for curve-fitting, which is important for evaluating the fit using the DW parameter.Figure 3 Screenshot of the user-definable parameters for the lifetime fitting, showing the decay histogram for the Alexa trace in Fig. 1. For details, see the user guide.

Data filtering

In the Filtering tab of the software (Fig. 4), the data can be filtered after level resolution, grouping, and lifetime fitting. The parameters used for filtering are the number of photons, intensity, average fitted lifetime, DW parameter of the fit, χ2 of the fit, and fitted IRF shift for each level or group. A histogram of each parameter or a two-dimensional (2D) scatterplot of two filter values can be plotted to aid in determining appropriate filter parameters (as shown in Fig. 4). Examples include filtering out levels with less than a certain number of photons (e.g., 100) or with lifetime or intensity values that are outliers. The filtering can be done for individual particles, a selection, or the entire data set.Figure 4 Data filtering example for Alexa, showing a scatterplot of intensity and lifetime values for grouped levels. The data are filtered to include only groups with more than 100 photons. The various filtering criteria are mentioned in the main text. Detailed information is given in the user guide.

Intensity-level grouping

The information gained from CPA can be significantly increased by statistically associating the resolved intensity levels with one another. A grouping or clustering of this kind has multiple uses. In the first place, it serves as a tool to resolve underlying states in the system being studied (40). An added advantage is performing fluorescence decay fitting of levels with low numbers of detected photons. The efficacy of fluorescence decay fitting (discussed in fluorescence decay fitting) depends, among other factors, on the number of data points used. Investigating states that emit very few photons, therefore, often proves to be problematic. By grouping several of these dark or dim states, a successful fit is possible where it would otherwise involve a large uncertainty.

The clustering method implemented in Full SMS is a mixed model of an agglomerative hierarchical clustering (AHC) method and expectation maximization (EM) clustering method (40,45,51). AHC starts by considering each intensity level as a group of its own. It then merges the pairs of groups that maximize a log-likelihood ratio merit function calculated using the number of photons within and the total duration of the two candidate groups. In other words, pairs of groups with the smallest intensity difference will be combined. This merging of group pairs is repeated until all the groups have been merged into a single group comprising all levels.

One of the limitations of AHC is that the outcome is highly dependent on the initial state as well as outliers, and once two groups have been merged, the decision is not challenged. For this reason, the outcome of the AHC—the combined levels corresponding to each number of possible groups—is used as an initial state for more advanced EM clustering, which serves to optimize the intensity-level groupings. In the EM algorithm, the log-likelihood that each group would have a certain resulting intensity is calculated given the number of photons in the group and the group’s total duration as defined by the Poisson probability density function. After the total probability is calculated, new probabilities for each level relating to each group are calculated.

Neither the AHC nor EM clustering method predicts the most likely number of groups. The latter is done using the Bayesian information criterion (BIC) (40). Full SMS provides an interactive tool to visualize the groups identified in each AHC-EM grouping scheme along with the corresponding BIC value.

The screenshot in Fig. 5 is an example of intensity-level grouping of data from an Alexa measurement. The top pane shows the group intensity as dotted lines and the group boundaries as yellow lines. The resulting intensity “bands” are indicated by alternating light-blue and white bands. The BIC values for each grouping scheme are shown in the bottom pane where the best grouping scheme is indicated as a green circle, representing the statistically most likely number of states, which corresponds to the largest BIC value. The option that is currently applied is indicated by the red outline, but the user can apply different solutions by clicking on any of the other circles. In this example, the best solution is also applied. After performing the grouping, the groups can also be viewed in the Intensity tab (Fig. S4).Figure 5 Screenshot of an intensity-level grouping of the Alexa intensity trace in Fig. 1. See text and user guide for details.

To demonstrate the advantage of grouping levels when fitting fluorescence lifetimes, the fitting results of three data sets are compared, as shown in Fig. 6. (Note that this figure is not a direct output from Full SMS but was created by using exported data.) The three samples for this case study are Alexa (see above), Qdot 605, and LHCII. Qdot 605 is a commercially available type of CdSe/ZnS core-shell quantum dot and LHCII is the main light-harvesting complex of plants, a pigment-protein complex containing several energetically strongly connected chromophores that are responsible for absorbing photons that provide the necessary energy to drive the initial photochemical processes in the photosynthetic apparatus of higher plants. The first column of these panels (A, D, and G) shows the distributions of the numbers of photons used in each fitted level for both the nongrouped and grouped cases. As the total number of levels between the nongrouped and grouped cases differs, the y axis was normalized to facilitate comparison. The second and third columns show 2D distributions of the weighted average of the fitted lifetimes and the level intensity for each level, with the second column showing nongrouped levels, and the third column showing grouped levels. It is clear from panels A, D, and G that the number of photons used for fitting the lifetimes in the grouped levels is significantly larger than when no grouping was done. The advantage of performing lifetime fitting on grouped levels is particularly evident for the Qdot 605 and LHCII data, which show the expected predominantly linear relationships between lifetime and intensity (panels F and I). A linear relationship is expected when the change in intensity is due to a change in the nonradiative decay rate (52,53,54). This linear relationship is significantly broadened in the Alexa data (panel C), which can be explained by the heterogeneous distribution of the molecular orientations relative to the elliptically polarized excitation light resulting in varying absorption cross sections and, consequently, altered fluorescence intensities.Figure 6 Lifetime fitting results of three data sets: Alexa Fluor 647 dye, Qdot 605 (CdSe/ZnS quantum dots), and light-harvesting complex II (LHCII). The first column (panels A, D, and G) shows the normalized distributions of the numbers of photons used in each fitted level for both the nongrouped and grouped cases. The second (panels B, E, and H) and third (panels C, F, and I) columns depict the relationship between the weighted average of the fitted lifetimes for each level and the corresponding level intensity, where the second column is for the nongrouped levels and the third column is for the grouped levels. The red lines denote linear fits through the origin.

While this grouping method already offers several advantages, the resolved intensity levels of each trace are grouped in isolation from other traces, which could hamper the successful resolution of the underlying states. For example, not all states are likely accessed in each measurement. In Full SMS, a global analysis can be done where each particle’s region of interest (ROI) (i.e., trimmed trace—see below) is appended into a single data set and then grouped, as shown in Fig. S5. Systematic intensity variations that may have resulted, for example, from focal drift during measurements, will negatively affect the efficacy of a global grouping analysis. In a case where the lifetime and intensity are linearly correlated (like for Qdot 605 and LHCII in Fig. 6) the user has the option to perform an intensity normalization, which will greatly improve the result of such an analysis. A sufficiently large data set is likely to resolve true states but this operation is computationally expensive.

Second-order correlation function

Single-quantum emitters exhibit so-called antibunching of photons, as seen in the second-order correlation function (g(2)(τ)) of photon arrival times (55). This is calculated as a cross correlation between two individual photon channels from detectors in a Hanbury Brown-Twiss configuration since the dead time of a detector precludes the measurement of coincident photons by a single detector. The Hanbury Brown-Twiss configuration consists of a beamsplitter dividing a stream of photons between two detectors connected to a coincidence unit and was its original application to measure the sizes of stars (56) before finding use in quantum optics. A perfect single emitter will never emit two photons at the same time, giving a value of g(2)(0)=0. The number of independent emitters, 1/(1−g(2)(0)), is the reciprocal of the value of the “antibunching dip,” 1−g(2)(0). Full SMS can calculate g(2)(τ) for two-channel data with an adjustable time window and bin size.

We demonstrate the analysis of photon antibunching using simulated data obtained from the code of Palstra and Koenderink (32) and converted to the Full SMS HDF5 format (Fig. 7). In the Antibunching tab of the software, g(2)(τ) can be calculated over an adjustable time window and bin size. The sum over all selected particles’ g(2)(τ) is shown in the top pane, which is useful in the case of low-intensity or short measurements where antibunching is not readily apparent in a single particle’s correlation histogram. Note that, while the demonstration shows a simulation for pulsed excitation, the calculation of g(2)(τ) after continuous-wave excitation is identical and can, therefore, also be done by Full SMS.Figure 7 Antibunching analysis using second-order photon correlation histogram (g(2)(τ)). The bottom pane shows the current particle’s correlation histogram, while the top pane shows the sum of correlations over all the selected particles, indicating clear evidence of photon antibunching through the absence of a zero-delay peak.

Display of spectra

In the Spectra tab of the software (Fig. 8), spectral time traces can be viewed, which are displayed as a 2D color map of the photon counts (intensity) as a function of the bin time and wavelength on the x and y axes, respectively. The color scale can be adjusted for optimal contrast by setting the upper and lower thresholds as well as the relative intensities of set colors along the scale. A histogram of pixel intensities is shown to assist in this regard. An ROI (separate from the “trace ROI”—see below) can be defined, allowing a plot of intensity as a function of time or wavelength to be displayed.Figure 8 Example of a spectral measurement of a single LHCII pigment-protein complex, showing a full spectral time trace (top) and temporally summed spectrum of the selected ROI (bottom).

Display of raster-scan images

The Raster Scan tab of the software (Fig. S6) displays raster-scan images. In SMS experiments, these images are mostly used to locate particles to be measured. Therefore, the functionality of this tab is currently limited to displaying images and indicating the position of the currently selected particle. The images can, however, easily be exported if further image analysis or processing is required.

Data export

In the Export tab of the software (Fig. 9), the analyzed data can be exported for further analysis. Binned intensity traces, resolved and grouped levels, and lifetime fitting results, can all be exported as plain text, or in the form of a parquet file (a commonly used open-source column-oriented binary data storage format maintained by Apache), which enables more convenient downstream processing for users familiar with Pandas, a Python library for data manipulation and analysis, which can easily read and load parquet files. Apart from this final export, it is also possible to save the current state of the analysis to continue working on it at a later time. This includes the binned intensity, resolved, grouped and fitted levels, as well as the current particle selection.Figure 9 Exporting tab of the software. The needed data can be selected, which can then be exported for the current, selected, or all particles.

Additional tools

In addition to the previously mentioned advantages of Full SMS, some additional tools have been developed. These tools add significant functionality to the analysis or enhance the user experience by automating common tasks.

Measurement data conversion

The native format of measurement data for which this software suite has been designed (HDF5) is custom in that the particular structure and naming used were arbitrary choices that needed to be made. As mentioned above, Full SMS uses a custom HDF5 file format—custom in the sense that the specific grouping and naming of data inside the hierarchical format are customized. Therefore, if measurements are made in a format that differs from this, either in file format or structure, it will be necessary to convert the original data into the format that Full SMS can read. This can be done ad hoc or built into the software. As an example of the latter, the software includes built-in conversion tools to convert measurements (with the appropriate structure) in either PicoQuant’s .pt3 format or plain text .csv format to the required .h5 format. As an example of the former, we include two scripts to convert publicly available data from the Koenderink and Schlau-Cohen groups (57,58), available on the GitHub repository under src/conversion scripts. These also act as examples for future contributors of how such a conversion can be done. Our software can also easily be extended to allow the analysis of Photon-HDF5 data.

Trace ROI

When performing SMS measurements, depending on the nature of the sample being studied, it is almost certain that a single particle cannot be measured for an indefinitely long time. This is usually due to photodamage, as the excitation intensities are typically large. Particles often bleach during a measurement run, and therefore it is often valuable to select a subsection of the measurement for analysis, excluding the data originating from a bleached particle (i.e., trimming the end of the intensity trace). However, in cases where data sets contain many measurements, applying some user-defined criteria to determine the ROI for each measurement individually is often a tedious task. To aid this task, a tool has been included to automate the application of an ROI over the entire data set (Fig. S7). The user defines a lower intensity threshold and a lower duration threshold. Resolved levels are considered until the last one that satisfies both thresholds. Each data set is analyzed from back to front, and the point (if any) at which both of these thresholds are broken is set to be the end of the ROI. An example is shown in Fig. 1, where the ROI is shown by the green shaded area, excluding the presumably bleached state at the end of the trace.

Photon burst detection

In certain types of SMS measurements, very short-lived spikes in intensity are recorded, a phenomenon that may be caused by cosmic rays or unwanted fluorescent particles diffusing through the focal volume. In most cases, such spikes skew the analysis and are therefore better to remove. Full SMS provides the user with two methods of identifying and removing photon bursts (Fig. S8). The first is based on the standard deviation (σ) of the mean intensity of each resolved level. A multiple of σ, as chosen by the user, above the mean intensity is used as a threshold definition. Alternatively, the user can define a manual intensity threshold. In either case, if any levels exceed the threshold, they are identified as potential photon bursts and can be removed.

Application architecture

General architecture

The program was written completely in Python, using a package called PyQt5 to interface the Qt GUI framework. The Qt GUI framework was chosen as it has rich features, works across all platforms (Windows/Mac/Linux) and allows rapid development of GUI windows using a GUI builder (Qt Designer). However, the primary complexity of adding a GUI is not in the design of the windows but in the underlying framework that supports it.

Full SMS was designed using object-oriented programming and makes use of a controller architecture, in which several class instances are responsible for separate aspects of the program. The interaction with the GUI, as well as the functional implementations of the analysis, are handled by several different controller classes. For example, the intensity controller is responsible for converting time series to binned intensity traces and interacting with the GUI to display the selected intensity trace. However, much of the advanced functionality that is used within the controllers is implemented in separate modules. For example, the CPA code is found within its own module but is called from within the intensity controller. In this manner, the code is organized into logical sections.

Multiprocessing

One of the critical current limitations of the Python language, as implemented by CPython (the most common implementation of Python written in C), is that it makes use of a global interpreter lock (GIL). The GIL is a necessary restriction that allows only one thread to execute code for a single Python process, effectively limiting the CPU core usage to a single core. Multithreading is still possible by allowing a single process to switch between threads rapidly. In the case of a GUI, this allows the interface to remain responsive to the user, even if a separate thread is running to, e.g., perform an analysis. However, the GIL prevents effective usage of modern computing power when performing analyses that allow for parallelization and would benefit from multiple CPU cores (The limitation described here is still the case as of Python version 3.12, but is planned to be removed in some future release.)

Several methods have been developed to achieve multicore processing, which, in most cases, involve starting multiple Python processes. The primary complexity is then the communication between processes.

The approach that the Full SMS software uses is as follows. Once the user interacts with the GUI, a thread is started to perform the analysis while allowing the GUI to remain reactive. The analysis thread then uses Python’s built-in multiprocessing module to start several Python processes, which will here be referred to as worker processes. The number of worker processes created is automatically scaled to the number of CPU cores the system has available. The use of queues achieves communication between processes. Queues are populated by the analysis thread with pairings of copies of a function with serialized versions of the data the function should be applied to. The functions and serialized data are loaded from the queue by the worker processes and executed. Afterward, the results are added to another queue in the form of serialized objects. The unfortunate consequence is that the parent process must consolidate the returned results into the original data structures. This can be done easily for simple objects, but for complex objects, as is the case for most of the objects in this software, great care needs to be taken to integrate all the relevant parts of the returned object. On completion of all the parts of the analysis, the parent process terminates the worker processes and the thread finishes.

Distribution and compatibility

The repository for Full SMS is publicly hosted on GitHub and can be accessed at http://github.com/BioPhysicsUP/Full_SMS. The easiest way to run the software is via a Windows installer that can be downloaded from the repository releases page. This installer does not have any prerequisites but is currently only available for Windows. Instructions for running from source can be found in the GitHub README. Detailed documentation of the software, including how to contribute, is available at https://up-biophysics-sms.readthedocs.io/en/latest/index.html. The data used for this paper can be downloaded at https://doi.org/10.5281/zenodo.12150465.

Conclusion

We describe a new GUI-based application for advanced analysis of single-molecule spectroscopic data. It uses a custom HDF5 file format that is uniquely suited to multiparameter measurements on single particles. The software is user-friendly, requiring no programming knowledge to use, although also being fully open source and easily extendable. It allows the analysis of photon-by-photon data to extract fluorescence intensity change points and groups the resulting intensity levels using AHC. It also allows fitting fluorescence lifetimes of individual and grouped levels, the calculation of the second-order intensity correlation function, data filtering options, and a few additional features.

Full SMS is highly suited to further extensions. An extension of the multichannel functionality to include multichannel change-point detection (59) would be useful for analyzing smFRET data. The CPA functionality could be further extended to allow for analyzing binned data. For the grouping analysis, relative likelihood values can be added as calculated from the BIC (60) for easier interpretation. The lifetime functionality could be extended to allow fitting with more than three exponential components and with decay models other than a multiexponential, such as lifetime distributions. Functionality could be added to analyze spectra (using fitting or phasors) and for analyzing fluorescence correlation spectroscopy and real-time feedback-driven single-particle tracking (19) data. This will allow correlation of lifetime, spectral and diffusion properties. Adding the ability to interface directly with other data formats, such as Photon-HDF5 (33), would enable more groups to start using the software immediately.

Author contributions

All authors conceived the study. J.L.B. and B.v.H. wrote the software, performed the experiments, analyzed the data, and wrote the original draft. T.P.J.K. acquired the funding, supervised the research, administered the project, and reviewed and edited the manuscript. All authors read and approved the final manuscript.

Supporting material

Document S1. Figures S1–S8

Document S2. Article plus supporting material

Acknowledgments

We thank the UP Biophysics group members for testing the software and suggesting improvements. We gratefully acknowledge Michal Gwizdala for the LHCII isolation. J.L.B. was supported by the Vrije Universiteit Amsterdam–NRF Desmond Tutu Programme (grant no. 99413 ). B.v.H. was supported by the 10.13039/501100001321 National Research Foundation (NRF), South Africa (grant nos. 115463 and 120387 ), the South African Academy for Science and Art, National Institute for Theoretical and Computational Sciences (NITheCS), South Africa, and the Fulbright Programme. T.P.J.K. acknowledges funding from the 10.13039/501100001321 NRF (grant nos. 87990 , 94107 , 109302 , 110983 , 112085 , 120387 , and 137973 ), the Photonics Initiative of South Africa, the Rental Pool Programme of the Council for Scientific and Industrial Research’s Photonics Centre, South Africa, and the University of Pretoria’s Research Development Programme, Strategic Research Funding, and Institutional Research Theme on Energy.

Declaration of interests

The authors declare no conflict of interest.

Joshua L. Botha and Bertus van Heerden contributed equally to this work.

Supplemental information can be found online at https://doi.org/10.1016/j.bpr.2024.100173.
==== Refs
References

1 Moerner W.E. Kador L. Optical Detection and Spectroscopy of Single Molecules in a Solid Phys. Rev. Lett. 62 1989 2535 2538 10040013
2 Orrit M. Bernard J. Single Pentacene Molecules Detected by Fluorescence Excitation in a P-Terphenyl Crystal Phys. Rev. Lett. 65 1990 2716 2719 10042674
3 Brooks Shera E. Seitzinger N.K. Soper S.A. Detection of Single Fluorescent Molecules Chem. Phys. Lett. 174 1990 553 557
4 Soper S.A. Shera E.B. Keller R.A. Single-Molecule Detection of Rhodamine 6G in Ethanolic Solutions Using Continuous Wave Laser Excitation Anal. Chem. 63 1991 432 437
5 Soper S.A. Davis L.M. Shera E.B. Detection and Identification of Single Molecules in Solution J. Opt. Soc. Am. B 9 1992 1761 1769
6 Nguyen D.C. Keller R.A. Martin J.C. Detection of Single Molecules of Phycoerythrin in Hydrodynamically Focused Flows by Laser-Induced Fluorescence Anal. Chem. 59 1987 2158 2161 19886672
7 Moerner W.E. A Dozen Years of Single-Molecule Spectroscopy in Physics, Chemistry, and Biophysics J. Phys. Chem. B 106 2002 910 927
8 Mazal H. Haran G. Single-Molecule FRET Methods to Study the Dynamics of Proteins at Work Curr. Opin. Biomed. Eng. 12 2019 8 17 31989063
9 Ha T. Ting A.Y. Weiss S. Single-Molecule Fluorescence Spectroscopy of Enzyme Conformational Dynamics and Cleavage Mechanism Proc. Natl. Acad. Sci. USA 96 1999 893 898 9927664
10 Jiang Y. Douglas N.R. Moerner W.E. Sensing Cooperativity in ATP Hydrolysis for Single Multisubunit Enzymes in Solution Proc. Natl. Acad. Sci. USA 108 2011 16962 16967 21896715
11 Kapanidis A.N. Margeat E. Ebright R.H. Initial Transcription by RNA Polymerase Proceeds through a DNA-scrunching Mechanism Science 314 2006 1144 1147 17110578
12 Hou S. Exell J. Welsher K. Real-Time 3D Single Molecule Tracking Nat. Commun. 11 2020 3607 32680983
13 Krüger T.P.J. Malý P. Van Grondelle R. How Reduced Excitonic Coupling Enhances Light Harvesting in the Main Photosynthetic Antennae of Diatoms Proc. Natl. Acad. Sci. USA 114 2017 E11063 E11071 29229806
14 Kondo T. Chen W.J. Schlau-Cohen G.S. Single-Molecule Fluorescence Spectroscopy of Photosynthetic Systems Chem. Rev. 117 2017 860 898 28072519
15 Gwizdala M. Botha J.L. Krüger T.P.J. Switching an Individual Phycobilisome off and On J. Phys. Chem. Lett. 9 2018 2426 2432 29688018
16 Gruber J.M. Malý P. Grondelle R.v. From Isolated Light-Harvesting Complexes to the Thylakoid Membrane: A Single-Molecule Perspective Nanophotonics 7 2018 81 92
17 Tuson H.H. Biteen J.S. Unveiling the Inner Workings of Live Bacteria Using Super-Resolution Microscopy Anal. Chem. 87 2015 42 63 25380480
18 Welsher K. Yang H. Multi-Resolution 3D Visualization of the Early Stages of Cellular Uptake of Peptide-Coated Nanoparticles Nat. Nanotechnol. 9 2014 198 203 24561356
19 van Heerden B. Krüger T.P.J. Theoretical Comparison of Real-Time Feedback-Driven Single-Particle Tracking Techniques J. Chem. Phys. 157 2022 084111
20 García de Arquer F.P. Talapin D.V. Sargent E.H. Semiconductor Quantum Dots: Technological Progress and Future Challenges Science 373 2021 eaaz8541
21 Toninelli C. Gerhardt I. Orrit M. Single Organic Molecules for Photonic Quantum Technologies Nat. Mater. 20 2021 1615 1628 33972762
22 Zhang T. Pramanik G. Chu Z. Toward Quantitative Bio-Sensing with Nitrogen–Vacancy Center in Diamond ACS Sens. 6 2021 2077 2107 34038091
23 Lubin G. Oron D. Yallapragada V.J. Photon Correlations in Spectroscopy and Microscopy ACS Photonics 9 2022 2891 2904
24 Heilemann M. van de Linde S. Sauer M. Subdiffraction-Resolution Fluorescence Imaging with Conventional Fluorescent Probes Angew. Chem. Int. Ed. 47 2008 6172 6176
25 van de Linde S. Wolter S. Sauer M. Single-Molecule Photoswitching and Localization Aust. J. Chem. 64 2011 503 511
26 Krüger T.P.J. Ilioaia C. Van Grondelle R. Controlled Disorder in Plant Light-Harvesting Complex II Explains Its Photoprotective Role Biophys. J. 102 2012 2669 2676 22713583
27 Gwizdala M. Berera R. Krüger T.P.J. Controlling Light Harvesting with Light J. Am. Chem. Soc. 138 2016 11616 11622 27546794
28 Seidel, C. A. Universität Düsseldorf: Software Package for MFD, FCS and MFIS. https://www.mpc.hhu.de/en/software/mfd-fcs-and-mfis.
29 Schrimpf W. Barth A. Lamb D.C. PAM: A Framework for Integrated Analysis of Imaging, Single-Molecule, and Ensemble Fluorescence Data Biophys. J. 114 2018 1518 1528 29642023
30 Gratton, E. Globals Software - Laboratory for Fluorescence Dynamics. https://www.lfd.uci.edu/globals/.
31 Preus S. Noer S.L. Birkedal V. iSMS: Single-Molecule FRET Microscopy Software Nat. Methods 12 2015 593 594 26125588
32 Palstra I.M. Koenderink A.F. A Python Toolbox for Unbiased Statistical Analysis of Fluorescence Intermittency of Multilevel Emitters J. Phys. Chem. C 125 2021 12050 12060
33 Ingargiola A. Laurence T. Michalet X. Photon-HDF5: An Open File Format for Timestamp-Based Single-Molecule Fluorescence Experiments Biophys. J. 110 2016 26 33 26745406
34 Snellenburg J.J. Laptenok S.P. Stokkum I.H.M.v. Glotaran: A Java-based Graphical User Interface for the R Package TIMP J. Stat. Software 49 2012 1 22
35 van Stokkum I.H.M. Weißenborn J. Snellenburg J.J. Pyglotaran: A Lego-like Python Framework for Global and Target Analysis of Time-Resolved Spectra Photochem. Photobiol. Sci. 22 2023 2413 2431 37523126
36 Kyeyune F. Botha J.L. Krüger T.P.J. Strong Plasmonic Fluorescence Enhancement of Individual Plant Light-Harvesting Complexes Nanoscale 11 2019 15139 15146 31372623
37 Xu P. Tian L. Croce R. Molecular Insights into Zeaxanthin-dependent Quenching in Higher Plants Sci. Rep. 5 2015 13679 26323786
38 Basché T. Fluorescence Intensity Fluctuations of Single Atoms, Molecules and Nanoparticles J. Lumin. 76–77 1998 263 269
39 Frantsuzov P. Kuno M. Marcus R.A. Universal Emission Intermittency in Quantum Dots, Nanorods and Nanowires Nat. Phys. 4 2008 519 522
40 Watkins L.P. Yang H. Detection of Intensity Change Points in Time-Resolved Single-Molecule Measurements J. Phys. Chem. B 109 2005 617 628 16851054
41 Ensign D.L. Pande V.S. Bayesian Detection of Intensity Changes in Single Molecule and Molecular Dynamics Trajectories J. Phys. Chem. B 114 2010 280 292 20000829
42 Yin S. Song N. Yang H. Detection of Velocity and Diffusion Coefficient Change Points in Single-Particle Trajectories Biophys. J. 115 2018 217 229 29241585
43 Krüger T.P.J. Ilioaia C. Van Grondelle R. Fluorescence Intermittency from the Main Plant Light-Harvesting Complex: Sensitivity to the Local Environment J. Phys. Chem. B 115 2011 5083 5095 21452801
44 Song N. Yang H. Parallelization of Change Point Detection J. Phys. Chem. A 121 2017 5100 5109 28616980
45 Li H. Yang H. Statistical Learning of Discrete States in Time Series J. Phys. Chem. B 123 2019 689 701 30632755
46 Hoogenboom J.P. den Otter W.K. Offerhaus H.L. Accurate and Unbiased Estimation of Power-Law Exponents from Single-Emitter Blinking Data J. Chem. Phys. 125 2006 204713
47 Houel J. Doan Q.T. Kulzer F. Autocorrelation Analysis for the Unbiased Determination of Power-Law Exponents in Single-Quantum-Dot Blinking ACS Nano 9 2015 886 893 25549009
48 Grinvald A. Steinberg I.Z. On the Analysis of Fluorescence Decay Kinetics by the Method of Least-Squares Anal. Biochem. 59 1974 583 598 4838786
49 Bajzer Ž. Therneau T.M. Prendergast F.G. Maximum Likelihood Method for the Analysis of Time-Resolved Fluorescence Decay Curves Eur. Biophys. J. 20 1991 247 262
50 Durbin J. Watson G.S. Testing for Serial Correlation in Least Squares Regression. II Biometrika 38 1951 159 178 14848121
51 Ward J.H. Hierarchical Grouping to Optimize an Objective Function J. Am. Stat. Assoc. 58 1963 236 244
52 Schlau-Cohen G.S. Yang H.-Y. Moerner W.E. Single-Molecule Identification of Quenched and Unquenched States of LHCII J. Phys. Chem. Lett. 6 2015 860 867 26262664
53 Tinnefeld P. Herten D.-P. Sauer M. Photophysical Dynamics of Single Molecules Studied by Spectrally-Resolved Fluorescence Lifetime Imaging Microscopy (SFLIM) J. Phys. Chem. A 105 2001 7989 8003
54 Weston K.D. Dyck M. Sauer M. Measuring the Number of Independent Emitters in Single-Molecule Fluorescence Images and Trajectories Using Coincident Photons Anal. Chem. 74 2002 5342 5349 12403591
55 Mandel L. Wolf E. Optical Coherence and Quantum Optics 1995 Cambridge University Press Cambridge
56 Brown R.H. Twiss R.Q. Correlation between Photons in Two Coherent Beams of Light Nature 177 1956 27 29
57 Palstra I.M. de Buy Wenniger I.M. Koenderink A.F. Intermittency of CsPbBr3 Perovskite Quantum Dots Analyzed by an Unbiased Statistical Analysis J. Phys. Chem. C 125 2021 12061 12072
58 Troiano J.M. Perozeni F. Schlau-Cohen G.S. Identification of Distinct pH- and Zeaxanthin-Dependent Quenching in LHCSR3 from Chlamydomonas Reinhardtii Elife 10 2021 e60383
59 Wilson H. Wang Q. ABEL-FRET: Tether-Free Single-Molecule FRET with Hydrodynamic Profiling Nat. Methods 18 2021 816 820 34127856
60 Burnham K.P. Anderson D.R. Burnham K.P. Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach 2nd ed edition 2002 Springer New York
