
==== Front
Int J Pharm X
Int J Pharm X
International Journal of Pharmaceutics: X
2590-1567
Elsevier

S2590-1567(24)00046-X
10.1016/j.ijpx.2024.100274
100274
Review Paper
An examination of process models and model risk frameworks for pharmaceutical manufacturing
O'Connor Thomas F. thomas.oconnor@fda.hhs.gov
a⁎
Chatterjee Sharmista a
Lam Johnny b
Hernán Pérez de la Ossa Dolores c
Martinez-Peyrat Leticia de
Hoefnagel Marcel H.N. ef
Fisher Adam C. a
a Food and Drug Administration, Center for Drug Evaluation and Research, Silver Spring, MD 20993, United States
b Food and Drug Administration, Center for Biologics Evaluation and Research, Silver Spring, MD 20993, United States
c European Medicines Agency, Human Division, Domenico Scarlattilaan 6, 1083 HS, the Netherlands
d French National Agency for Medicines and Health Products Safety, F-93285, Saint-Denis, France
e Quality Innovation Group (QIG), European Medicines Agency (EMA), Amsterdam, the Netherlands.
f CBG-MEB (Medicines Evaluation Board), Utrecht, the Netherlands
⁎ Corresponding author. thomas.oconnor@fda.hhs.gov
08 8 2024
12 2024
08 8 2024
8 10027431 5 2024
5 8 2024
5 8 2024
Published by Elsevier B.V.
2024

https://creativecommons.org/licenses/by-nc-nd/4.0/ This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
Process models are a growing tool for pharmaceutical manufacturing process design and control. The Industry 4.0 paradigm promises to increase the amount of data available to understand manufacturing processes. Tools such as Artificial Intelligence (AI) might accelerate process development and allow better predictions of process trajectories. Several examples of process improvements realized through the application of process models have been shown in lyophilization, chromatography, fluid bed drying, bioreactor control, continuous direct compression, and wet granulation. An important consideration of implementing a process model is determining the impact of the model on the quality of the product and the risks associated with model maintenance over the product lifecycle. Several regulatory documents address risk-based considerations for process models. This work discusses existing risk-based frameworks for model validation and lifecycle maintenance that could aid the adoption of process models in pharmaceutical manufacturing. Hypothetical case studies illustrate the implications of applying a model risk framework to facilitate model validation and lifecycle maintenance in the manufacture of pharmaceuticals and biological products.

Graphical Abstract

Unlabelled Image

Keywords

Pharmaceuticals
Process models
Model risk assessment
Model validation
Model lifecycle maintenance
==== Body
pmc1 Introduction

In 2002, FDA laid the foundation for implementation of a modern, risk-based pharmaceutical quality assessment (U.S. Food and Drug Administration, 2004). Part of FDA's initiative encouraged manufacturers to use the latest scientific advances in pharmaceutical manufacturing technology throughout the lifecycle of a product to improve the efficiency of developing and manufacturing drugs. This has also been encouraged by EU as part of the EU directive 2001/83 and various CHMP (Committee for Medicinal Products for Human Use) guidelines e.g., CHMP guideline on manufacture of the finished dosage form (EU CHMP, 2015). As scientific and engineering knowledge about pharmaceutical manufacturing has grown, the use of models to aid process development, enhance process control and forecast future process and product quality outcomes has increased. The types of models being developed have also evolved. Models can be broadly categorized into three main types (Kourti et al., 2014).• Mechanistic models (also known as first principles models or theoretical models) are mathematical representations of physical, chemical, or biological phenomena which drive processes.

• Empirical models build on available data which may be collected with intent (causal) or available as historical process data. Artificial Intelligence (AI) based models that are mainly based on heuristics are categorized as empirical models.

• Hybrid models bridge theoretical knowledge with available data sets to generate a working representation of a process. An AI model (Executive Order 14110 of October 30, 2023) coupled with a mechanistic model is an example of a hybrid model. (See Fig. 1.)

Fig. 1 Hypothetical model risk assessment of process model case studies using the ASME V&V 40 framework. The y-axis captures decision consequences, the significance of an adverse outcome that could result from an incorrect decision, with increasing consequences moving up along the y-axis. The x-axis captures model influence, the contribution of the model to the decision relative to other available data, with increasing influences moving to the right along the x-axis. The blue circles denote the result of the hypothetical risk assessment for each case study. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 1

The type of model to apply to a process can depend on the modeling objective, available data, and scientific understanding. Models have been implemented for developing and controlling processes and submitted in regulatory submissions. Such models include, for example but are not limited to, multivariate response surface models to characterize a process design space, chemometric models as part of Process Analytical Technology (PAT) for process monitoring and product release (Chatterjee et al., 2017), and residence time distribution (RTD) models to monitor material traceability, set feeder limits and aid in diversion of non-conforming material in continuous manufacturing processes for solid oral products.

Pharmaceutical manufacturing is moving towards the Industry 4.0 paradigm, which is characterized by a high degree of digital connectivity and in which modeling might be an integral component of process monitoring and control (Arden et al., 2021). The move towards Industry 4.0 increases the availability of plant-wide information and data-rich processes, that can, for example, enable model-based process design and scale-up, process monitoring and fault detection, and advanced process control. This has opened the possibility of using models to create a digital twin of the manufacturing process, which can serve as a virtual representation of the process that mimics its behavior.

Even with the expanded use of process models, stakeholders report challenges with the development, implementation, verification/validation, registration, and lifecycle management of models (BioPhorum, 2021). Regulatory policy documents provide some information to support the development, validation, and submission of models; such documents include: (a) the International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use (ICH) Quality Implementation Working Group Points to Consider (R2), Dec 2011 (ICH quality implementation working group - points to consider (R2) - ICH-endorsed guide for ICH Q8/Q9/Q10 implementation, 2011a), (b) FDA guidance for industry on Development and Submission of Near Infrared Analytical Procedures (FDA NIR guidance) (U.S. Food and Drug, Administration Center for Drug Evaluation and Research, 2021a), (c) the EMA guideline on the use of near infrared spectroscopy by the pharmaceutical industry and the data requirements for new submissions and variations (EMA CHMP, 2014a), (d) the EMA Preliminary QIG Considerations regarding Pharmaceutical Process Models (EMA Preliminary QIG Considerations regarding Pharmaceutical 5 Process Models, 2024), and (e) ICH Q13: Continuous Manufacturing of Drug Substances and Drug Products (ICH, 2023). However, other risk frameworks consider the model's risk and may prove helpful in guiding the development, validation, and maintenance of process models, such as American Society Mechanical Engineers Verification and Validation 40 standard. This paper provides: (i) a review of case studies from literature on the application of process models for pharmaceutical process development, monitoring, and control, (ii) a review of existing risk frameworks for process models, and (iii) a speculative discussion of the application of a risk-based framework to several process model case studies.

2 Applications of process models for pharmaceutical manufacturing

Process models are increasingly utilized to accelerate and improve process design, scale-up, site transfer, process monitoring, and process control. The following examples illustrate how process models have been applied across different types of pharmaceutical manufacturing processes.

2.1 Lyophilization process design and scale-up

Lyophilization is a common processing operation to increase the shelf-life of labile drug products. Primary drying is the rate-limiting step of the lyophilization process. The optimum primary drying cycle depends upon formulation, and primary container, and often requires an iterative and resource-intensive experimental approach for process design optimization, and scale-up. Technical runs to manage the risk during the scale-up of biological products can cost up to $1MM/run for material, facility, resources, and testing (Tchessalov et al., 2021a). Hybrid models have been developed and published for the primary drying phase of lyophilization based on fundamental heat and mass transfer equations (Tchessalov et al., 2021b). A modeling workflow described by an industry consortium consisted of a non-steady state one-dimensional primary drying model, determination of vial heat transfer coefficients, determination of mass transfer coefficients, and establishment of equipment limitations. This model also includes an empirical parameter to represent the difference between primary drying observed in laboratory and commercial manufacturing environments, needed due to differences in supercooling. Industry has reported several cases where primary drying models have been used to support development and scale up (Tchessalov et al., 2021c; Zhu et al., 2018). In one such case, a company was able to optimize lyophilization cycle parameters shelf temperature, chamber pressure, and primary drying time by simulating multiple process conditions. The model built on small-scale experiments and predicted the process conditions for the scaled-up commercial scale equipment. The model predictions were confirmed with a single technical run, and the optimized process reduced the commercial drying cycle time from 80 to 42 h (Tchessalov et al., 2021d).

2.2 Chromatography process design and scale-up

In the biopharmaceutical industry, the development and characterization of chromatography processes are typically based on statistical models derived from experimental studies. The resulting models might fail to predict, for example, non-linear behavior in preparative chromatography with complex protein feed streams and can be limited to the conditions used to develop the model. The fundamentals of mechanistic models for ion-exchange chromatography for proteins are well-established and the application of these models continues to advance for complex proteins such as monoclonal antibodies (Rischawy et al., 2019a). In such, transport dispersive models describe mass transfer and steric mass action models describe protein adsorption and can account for variation in salt concentration and pH, which are typically required for industrial applications. Hybrid models combining a mechanistic isotherm with data driven parameters might also be used to account for the complex interactions between the resin and protein (Ding et al., 2023). Advances in the modeling of protein adsorption may even be able to predict non-ideal peak shapes observed at high protein concentrations by accounting for protein-protein interactions (Kumar et al., 2021). One of the modeling challenges for industrial applications (e.g., pH-dependent multicomponent steric mass action isotherm models is the large number of parameters that need to be estimated. Inverse calibration approaches allow for the estimation of unknown parameters but can lead to ill conditioning that can result in wide ranges for estimated parameters. Calibration approaches have been proposed in the literature to address this issue by reducing model uncertainty and increasing the model's predictive power (Rischawy et al., 2019b; Saleh et al., 2020a). In one industrial application, the process model approach was used to identify critical process parameters for a cation exchange chromatography (CEX) of a bispecific monoclonal antibody (mAb). CEX is a typical polishing step that is relatively work-intensive and often the most critical unit operation to meet product quality goals. Model-based process development approaches can be especially impactful for complex monoclonal antibody formats, for which conditions from previous process development programs may not apply (Rischawy et al., 2019c). In another application of using mechanistic modeling to scale-up CEX process steps (Saleh et al., 2020b), the model was a root cause investigation tool that offered improvements over typically used experimental scale-down models by capturing effects of bed height, loading density, feed composition, and mobile phase properties (Saleh et al., 2020c).

2.3 Continuous direct compression process design

Flowsheet models for continuous manufacturing processes not only incorporate models of the individual unit operations, but also account for interactions between unit operations. Academic, industry, and regulatory groups have published examples of flowsheet models of continuous manufacturing processes for solid oral drug products (Moreno-Benito et al., 2022a; Tian et al., 2019; Galbraith et al., 2019; Garcia-Munoz et al., 2017; Wang et al., 2017; Rogers et al., 2014). In one reported case, a flowsheet model of a continuous direct compression line for a solid drug product was developed, which combined: (i) mass balances, (ii) RTD models to account for composition dynamics, and (iii) empirical models to predict blend and tablet properties (Moreno-Benito et al., 2022b). The connection between the RTD and the flowsheet model with operating conditions was based on Discrete Element Models (DEM), which is based on the numerical simulation of the motion and forces on each particle individually. This allows the RTD to adapt with changes in the process during a simulation. Additionally, the flowsheet's empirical model obtained from process data captures changes in bulk density of intermediate blends as a function of process variables and material properties. This bulk density has a significant impact on quality attributes like tablet mass and hardness. The flowsheet model was used to optimize the design space and de-risk product and process development considering both steady state operation and the dynamic behavior of the process. To do this, simulations were used: (i) to conduct a virtual design of experiments for a range of operating conditions at steady state operation, (ii) to assess the impact of high and low frequency composition disturbances, and (iii) to perform sensitivity analysis of material properties and operating conditions on quality attributes. Simulations were performed with different probability functions to describe the most appropriate operating ranges, uncertainty levels, and potential variability in the inputs. The model results were used for optimizing the formulation and process design, while reducing the number of runs necessary in experimental campaigns. This led to active pharmaceutical ingredient (API) savings, fewer resource requirements, and faster process development.

2.4 Continuous direct compression process monitoring and control

Process models can also be used as part of the control strategy for process monitoring and control. An RTD-based process model was developed and validated for monitoring API concentration in a continuous direct compression control strategy. The process model was based on a tank-in-series RTD model with empirical equations that related process input parameters to the output mean residence time (Hurley et al., 2022a). The process model was validated for GMP manufacturing and was able to achieve a prediction error of 1.4% for tablet API concentration as measured by high performance liquid chromatography. The model was incorporated into the control strategy, which involved updating product rejection limits to incorporate different sources of error (Hurley et al., 2022b). Another approach for implementing a soft sensor for monitoring API concentration utilized a flowsheet of mechanistic models based on mass balance operations for each unit operation. Fault detection logic was incorporated to identify certain operations events and re-adjust the model accordingly. The output of the model was the API concentration leaving each unit operation and confidence bounds for the estimate (Kamyar et al., 2021a). The model was validated for use in a commercial manufacturing process; across a wide range of operating conditions and material properties the prediction error was less than 1.5% at steady state. During dynamic operations the measured API concentration fell within the model prediction confidence limits, suggesting the confidence limits could be used as a conservative measure of concentration during these periods (Kamyar et al., 2021b). Cogoni et al., demonstrated how the mechanistic soft sensor can be combined with near infrared (NIR) spectroscopy to predict API concentration for a continuous direct compression process (Cogoni et al., 2021a). NIR measurements are often sensitive enough to sample physical properties caused by raw materials or process conditions, thus incorporating process and physicochemical knowledge can increase the robustness of the on-line API concentration estimate. This hybrid approach showed improved accuracy, while retaining precision for perturbation detection, when compared to offline analytical measurements (Cogoni et al., 2021b).

2.5 Fluid bed drying process control

Soft sensors generate a signal from software instead of hardware. Soft sensors use available material attribute and process parameter measurements to predict attributes that are not measured by a physical sensor. Once soft sensors are developed, there is the potential to incorporate them into a model as part of process control. For example, Lauri Pla et al., developed a soft sensor for online prediction of moisture content in fluid bed dryers. The soft sensor was a hybrid model where mass and energy balances provided the form of the model, and model parameters were calibrated from experimental data (Lauri Pla et al., 2018a). The model used real-time temperatures, air flow, and humidity data as inputs (Lauri Pla et al., 2018b). The model has been incorporated into an advanced process control application wherein the moisture prediction is used to adjust inlet air temperature and flow to maintain an optimized drying path during the process (Huang et al., 2020a). The advanced process control application, when implemented for a commercial manufacturing process, reduced fluid bed drying cycle times by 20% and reduced variability in tablet weight and hardness by 50% and 30%, respectively (Huang et al., 2020b).

2.6 Bioreactor process control

Process intensification efforts have improved bioreactor cell densities, product output, and process efficiency. Intensified processes have a greater nutrient demand, in particular for glucose and other metabolites that can impact yield and quality. Glucose is essential for cell metabolism and if not maintained within the target ranges can negatively impact cell viability and product quality attributes (Liu, 2015). The glucose concentration in the bioreactor can be measured in real-time using Raman spectroscopy (Craven et al., 2014). Gibbons et al., reported that a Raman-based feedback control of glucose concentration in a fed-batch bioreactor increased product titer by 25% and improved the glycation profile for a CHO cell line in development (Gibbons et al., 2023). An example of a biotechnology application came from Rashedi et al., who developed an advanced process control application to maximize cell growth and biotherapeutic production while maintaining product quality in a fed-batch bioprocess (Rashedi et al., 2022a). The process model was based on glucose mass balance coupled with a linear empirical model that predicts the future states of the bioreactor based on the previous state and the glucose level. The advanced process control application as part of the overall control strategy that included controls for pH, dissolved oxygen, and temperature increased yield and resulted in fewer protein impurities over the current recipe-based control strategy approach. As a result of these improvements the advanced process control application had the potential to reduce production costs by 5% (Rashedi et al., 2022b). Metabolic and glycosylation models have also been used for media design and bioreactor optimization (Reddy et al., 2023).

2.7 Continuous wet granulation process monitoring and continuous improvement

For highly automated pharmaceutical manufacturing systems that have many sensors connected to various unit operations to collect in-process information, large volumes of data are generated during routine operation. In such cases, multivariate statistical process control tools can be employed to gain process understanding and to monitor and control the process. In one example, multivariate statistical process modeling by means of chemometric methods was used to monitor a continuous wet granulation tableting process (Zomer et al., 2018a). Models were developed for each of the different units that make up the continuous tableting line, from material feeding and granulation up to tablet compression. The models predicted dynamics of each system during routine operation. The models can detect the beginning of process issues such as i) prolonged offsets in material feeding during granulation; ii) variable inlet air conditions and filters occlusion in the dryer leading to uneven discharge of material for down-stream processing; iii) filters occlusion and tear during transfer; and iv) offsets and variability in materials discharge prior to blending. This information was used to make corrective actions in real time when needed. In addition, the information was leveraged retroactively to optimize the process as a part of continual improvement (Zomer et al., 2018b).

2.8 Cell therapy products product and process design

Comprehensive multi-omics characterization (including various analytical methods such as mass cytometry, transcriptomics, metabolomics, lipidomics, and secretomics) of various related cell-based products could be used as multivariate inputs/predictors and correlated to patient clinical outcomes/responses (Zylberberg et al., 2017; Torres-Garcia et al., 2021a). De-identified patient information and clinical outcomes pre- and post-treatment may be used to improve the correlative models further towards identifying cell therapy product-type-specific critical quality attributes (CQAs) that could prove useful for quality-by-design implementation in cell manufacturing (Lipsitz et al., 2016; Toye et al., 2021). Several data analysis approaches have been proposed to develop predictive models between multi-omics characterization data and clinical outcomes for such cell therapies, which include linear/nonlinear regression, canonical correlation analysis, and supervised or unsupervised machine learning algorithms (e.g., principal component analysis) (Torres-Garcia et al., 2021b; Yeago et al., 2023; Yon et al., 2022; Xu et al., 2022a). As an example, recent efforts leveraged an image-based machine learning detection model to automate the quantification of the quality of the immunological synapse between the chimeric antigen receptor T cells (CAR-T cells) and the tumor antigen on glass-supported planar lipid bilayer platforms. Using some patient samples from clinical trials, it was found that Machine Learning (ML)-quantified CAR-T immunological synapse quality data correlated with clinical responders and non-responders (Xu et al., 2022b).

Research grants reviewed by the Center for Biologics Evaluation and Research (CBER) have reported preliminary supporting information from developed predictive models using canonical correlation analysis (CCA) and nonlinear, symbolic regression using single-cell gene expression level of various cell-based products as inputs/predictors and their immunosuppressive bioactivity as output/performance (Van Grouw et al., 2023a). Additionally, some early phase clinical studies have indicated the utility of additional characterization of cellular products in facilitating product and process design through data-driven modeling (Roy et al., 2024). Such additional product characterization may include viability, apoptosis profile, single-cell RNA transcriptomics, broad spectrum lipidomics-metabolomics (mass spec), single-cell mass cytometry, and performance in cell-based immunosuppression assays (Van Grouw et al., 2023b) (Doron et al., 2020; Srinivasan et al., 2022a; Mautner et al., 2023).

3 Risk-based frameworks for process model validation and lifecycle maintenance

The previous section described a range of examples of process models being used to accelerate and improve process design, scale-up, site transfer, process monitoring, and process control. Even with these published examples, it is recognized there is an opportunity to more broadly adopt process models to achieve these benefits. As the science of pharmaceutical and biological manufacturing continues to develop, this section describes risk-based frameworks for model validation and lifecycle maintenance that can support moving these advancements into practice.

3.1 Existing model risk frameworks

The ICH Quality Implementation Working Group (IWG) established the principle that the level of oversight for a model should be commensurate with the level of risk to product quality and classified models as high, medium, or low impact (ICH quality implementation working group - points to consider (R2) - ICH-endorsed guide for ICH Q8/Q9/Q10 implementation, 2011b). Based on this principle, models for process design or product development can be considered low impact. Models for process monitoring are typically classified as medium impact, as there are additional mechanisms beyond the model to assess product quality such as traditional release tests. Models that are used for predicting a quality attribute for release of the drug product are generally considered high impact as these models can be a significant indicator of product quality (e.g., models that support real time release testing). This approach can be considered a one-dimensional approach for assessing model risk, as the risk is determined solely based on the model's influence on product quality decisions.

An alternative approach for determining model risk is described in the American Society Mechanical Engineers Verification and Validation 40 (ASME V&V 40) standard (Assessing the credibility of computational modeling through verification and validation: Application to medical devices V&V 40, 2018a). Per this standard, model risk is the possibility that the model may lead to a false/incorrect conclusion about process performance that results in an adverse outcome. Assessing model risk begins with defining the specific context of use for the model to determine the model influence. The context of use is a statement that describes the role of the model in relation to other data to address a specific question of interest. Some examples of context of use:• A crystallization population balance model used to enhance the understanding of process conditions on crystal size and yield during process development

• A RTD model that is used for orchestrating diversion of non-conforming material

• A near infrared spectroscopy procedure with chemometric calibration model used to measure tablet assay for product release

In the ASME standard, risk can then be assessed using a two-dimensional approach that combines model influence with decision consequence. Model influence is the contribution of the model to the decision relative to other available evidence. Decision consequence is the significance of an adverse outcome that could result from an incorrect decision, i.e., the model's impact on product quality (Fig. 1).

3.2 Risk-based model verification and validation

A process model is a mathematical description of an element of the physical manufacturing process. The objective of verification is to ensure that the mathematical model is implemented correctly and then accurately solved. Validation is the process of assessing the degree to which the model is an appropriate representation of the physical system. Therefore, validation activities are principally concerned with demonstrating the correctness of the underlying model assumptions and the degree to which sensitivities and uncertainties of the computational model are understood (Assessing the credibility of computational modeling through verification and validation: Application to medical devices V&V 40, 2018b). The evaluation of sensitivities aims at determining the degree to which the computational model outputs are sensitive to the model inputs. Assessing uncertainties helps quantify the degree to which known or assumed uncertainties in the model inputs are propagated to uncertainties in the simulation results.

In general, the extent of model validation activities is based on model risk. Validation is generally demonstrated by comparing the model predictions with experimental data measured using a reference method. Therefore, appropriate validation activities require attention to both the computational model and the experimental data, along with an appropriately rigorous evaluation of the model results. The ASME V&V 40 describes a framework for connecting model verification and validation activities to model risk for medical device applications (Assessing the credibility of computational modeling through verification and validation: Application to medical devices V&V 40, 2018c). In addition, the Center for Devices and Radiological Health at the FDA published a draft guidance on assessing the credibility of computational models in medical device submissions based on the ASME standard (Assessing the Credibility of Computational Modeling and Simulation in Medical Device Submissions: Guidance for Industry and Food and Drug Administration Staff, 2023).

The scope of the ASME V&V 40 standard encompasses physics-based computational models for medical device applications; however, the framework is general enough that it might be applied in other fields such as pharmaceutical manufacturing. For example, CDER staff published a white paper showing how the risk-informed credibility assessment framework described in the ASME V&V 40 standard can be applied to physiologically-based pharmacokinetic models used for model-informed drug development (Kuemmel et al., 2019a). CDER scientists have applied the ASME framework to the validation of process development models (Liu et al., 2020). The ASME V&V 40 standard has also been applied to a variety of empirical, mechanistic, and machine learning models used in biopharmaceutical manufacturing (Bideault et al., 2021).

There were some challenges noted in the application of the ASME V&V 40 framework to other disciplines. For example, different terminology is used in various disciplines to describe model validation. This might lead to a misunderstanding of terms like validation and verification across disciplines (e.g., across computational science and regulatory science communities) (Kuemmel et al., 2019b; Shepard et al., 2015). Another challenge is translating certain validation activities described in the standard to empirical models such as the assessment of model form and model inputs. For example, machine learning models may estimate the relationships between input and output variables without the modeler programing a specific set of equations or model form. In these cases, the assumptions are contained in the data used to develop the empirical model. Thus, for empirical models considerations for the training and testing data such as the representativeness of the data for the context of use need to be incorporated. Efforts are underway to extend this framework to Computational Modeling for Advanced Manufacturing (VVUQ 50), Artificial Intelligence and Machine Learning (VVUQ 70), and Pharmaceutical Products (VVUQ80) (The American Society of Mechanical Engineers, 2024).

There have been efforts to define good modeling or simulation practice (Erdemir et al., 2020; Rischawy et al., 2019d), to address model development and validation challenges. FDA has noted there is opportunity to establish Good Simulation Practice to foster harmonization across the FDA and with international regulatory bodies (U.S. Food and Drug Administration, 2021). Good modeling practices may include, for example, starting with a precise definition of the problem to be modeled and the relevant performance requirements, discussion of assumptions, then carrying out sensitivity and uncertainty analysis with the final model, and finally documentation of all the performed activities. These practices in general correspond with model credibility factors outlined in the ASME V&V 40 standard.

Similar efforts are underway for the development of Good Machine Learning Practices (GMLP) for AI models. Stakeholder feedback to a Center for Devices and Radiological Health (CDRH) discussion paper provided strong support for the idea and importance of GMLP (U.S. Food and Drug Administration, 2019). In 2021, FDA in partnership with Health Canada and MHRA published 10 guiding principles for good machine learning practice for medical device development (U.S. Food, and Drug Administration, Health Canada, 2021). FDA is engaged with multiple organizations on the development of GMLP as applied to software as a medical device (SaMD) application. Further assessment and engagement is warranted to understand how these efforts can inform application of AI models to pharmaceutical manufacturing.

3.3 Risk-based model maintenance

Management of changes to models that are used to support the control strategy of a medicinal product is an integral element of the product lifecycle. Model maintenance can be defined as a set of planned activities over the product's lifecycle to monitor and maintain the model's performance to continually ensure its suitability for its intended purpose. After a model has been successfully implemented as a component of the overall control strategy, it is necessary to periodically evaluate the performance of the model to ensure it remains fit for purpose. As new information becomes available over the lifecycle of the product, updates to the model can be made as required. For example, some data-centric models are highly sensitive to variation in model inputs (e.g., incoming raw material properties).

A risk-based approach for model maintenance can be designed to account for the importance of the model in the control strategy and its potential to affect product quality. Clear metrics for model updates can be established depending on the impact of the model. For example, some types of empirical models calculate diagnostics (e.g., Hotelling T-squared) that assess how the new input data compares with the data used to develop the model. Such diagnostic metrics can be used to support model maintenance. Model maintenance information might include risk-based frequency of comparing model prediction with the reference method, triggers for model updates, and the approach for model recalibration. Periodic assessment of model predictions can enable a performance-based maintenance approach in which model outputs are specified (e.g., acceptance criteria for prediction errors and model bias rather than focusing on model parameters) (Q12 Technical and Regulatory Considerations for Pharmaceutical Product Lifecycle Management Guidance for Industry, 2021).

The model maintenance approach is dependent on the type of model. The impact of changes on process models can be determined based on two factors (U.S. Food and Drug, Administration Center for Drug Evaluation and Research, 2021b):• Impact of the change on model's performance

• Impact of the change on product quality

In general, the level of oversight by a manufacturer for a model over its lifecycle is dependent on its context of use and model risk, as well as compliance with phase appropriate GMP. The model risk can be considered in the assessment of the impact of the change on product quality. Scientific understanding of the model along with development and manufacturing data can be used to assess the impact of the change on the model's performance. The overall assessment of the impact of the change can inform what validation activities, if any, need to be conducted to ensure the process model remains suitable for the intended use. The manufacturer then documents the assessment and the final determination on if and how the model will be updated. It is to be noted, that this publication does not cover considerations for regulatory notification of model updates, made throughout the product lifecycle.

4 Model risk assessment case studies

Above, we hypothesized that the ASME V&V 40 standard might be applied in pharmaceutical manufacturing settings. Here we apply the ASME framework to hypothetical pharmaceutical manufacturing case studies, based on known uses of process models in literature, and examine the determination of risk. Per the ASME V&V 40, we address some aspects that might be relevant to establish risk-informed credibility of pharmaceutical process models, such as the model's question of interest, context of use, and risk. We also discuss the model's maintenance.

4.1 Example 1

NIR Models: NIR for at-line measurement of active content of a tablet to predict tablet content uniformity data for real time release testing (RTRT) and to trigger diversion of non-conforming tablets.

4.1.1 Background on question of interest

An NIR model, which is an integral part of an NIR procedure, is a mathematical expression that describes how the NIR spectral data, obtained by directly interrogating samples, are related to the property-of-interest. These NIR models are chemometric models which use a multivariate approach to characterize the relationship between the spectral variation in the calibration set and the sample's characteristics (e.g., the sample's active ingredient concentration). NIR models can be either qualitative or quantitative.

4.1.2 Context of use

In the pharmaceutical industry, there are many examples of the implementation of in-line or at-line NIR for monitoring blend uniformity or tablet assay. In this context, NIR is used for at-line measurement of active content of tablet to predict tablet content uniformity data that is used for real time release testing (RTRT) and to trigger diversion of non-conforming tablets.

4.1.3 Determination of model risk

NIR model for measuring content uniformity.• Model Influence: High

The model is an integral component of the control strategy as it is used to support RTRT and to detect non-conforming product and initiate tablet diversion.

• Decision Consequences: High

The model is used to predict a CQA of the final drug product. Model prediction is used for product release.

The final overall model risk might be high.

4.1.4 Model maintenance

Section VIII of FDA NIR guidance and section 7 of the EMA CHMP NIR guidance and its annex provide general recommendations for risk assessment of changes to NIR analytical procedures during lifecycle management (U.S. Food and Drug, Administration Center for Drug Evaluation and Research, 2021c; EMA CHMP, 2014b). As described in the guidance, periodic evaluation of NIR procedures is warranted as these are highly sensitive to the manufacturing process and incoming material attributes. For example, a major change might include an update of the procedure with a change in the acceptance criteria for validation, as it will have a high impact on the performance of the procedure (accuracy) and on the product quality (e.g., potency).

4.2 Example 2

Residence Time Distribution Model: RTD model used in conjunction with LIW feeder data to predict drug product assay and to orchestrate deviations.

4.2.1 Background on question of interest

A RTD model describes the in-process mixing through a probability distribution of the time material spends in the process. The RTD process model can incorporate relevant incoming material properties and process conditions, as well as equipment configuration. Real-time material feed rates then can be combined with the RTD model to predict in-process homogeneity of the powder blend and the API concentration in the final blend and/or tablets. Additionally, an in-line NIR in the feed frame monitors API concentration. The model can also be used to track material impacted by disturbances throughout the process and orchestrate diversions. A RTD model is generally a hybrid model, based on first principles from chemical engineering.

4.2.2 Context of use

RTD models are generally proposed in continuous manufacturing applications. These models are used to gain an understanding of material flows within the system, determine sampling frequency, and identify an approach for diverting non-conforming material. The manufacturing train is either modeled using the convection diffusion equation or a tank in series approach to describe the mixing in a continuous direct compression process. Model inputs are the loss in weight (LIW) from the material feeders, and potentially other process parameters such as throughput and blending speed. Model parameters are experimentally determined during development and can be dependent upon process parameters, equipment configuration, and material attributes of the blend.

In this context, we consider an RTD model used in conjunction with LIW feeder data to predict drug product assay and to orchestrate deviations. NIR measurements of blend uniformity in the tablet feed frame are present to confirm the quality of the in-process material.

4.2.3 Determination of model risk

The RTD model enables diversion of non-conforming product and thus impacts product quality. The model predicts active ingredient concentration in the tablet based on the LIW feeder data and a flowsheet model of the process. Given, the presence of in-line NIR, the model is not the sole indicator of product quality.• Model Influence: Medium

The model plays an important role orchestrating deviation, but the NIR measurement of uniformity confirms the quality of the in-process material prior to the compression unit operation.

• Decision Consequences: Medium

In-line NIR for monitoring API concentration and downstream controls in the form of release testing are in place, hence, the model is not the sole predictor of drug product quality.

The final overall model risk might be medium.

4.2.4 Model maintenance

The RTD model would appropriately be updated and verified if there are any changes to input parameters to the model during the lifecycle of the product, for example, a change in geometry of the continuous mixer. Verification can be done by comparing model predictions to measured API concentration (off-line HPLC analysis) using an intentionally introduced negative step change in API concentration.

The impact of changes to the model can be determined based on the model risk (medium), and on the impact of the change on the model's performance and on product quality. For example, minor changes might include changes to a fitted model parameter if there is a change in one of the input parameters (e.g., flowability of in-coming materials due to a change in supplier) since this has low impact both on model's performance and on product quality. Moderate changes might include changes in model structure corresponding to a change in equipment configuration or equipment design since this has high impact on the model's performance but low impact on product quality.

4.3 Example 3

AI Model for Process Control: Machine learning is used to predict product concentration with a day advance for a fed-batch bioreactor process.

4.3.1 Background on question of interest

Machine learning algorithms in biopharmaceuticals could be used for correlating real time process data to predict a product attribute. In fed-batch processes to produce therapeutic proteins, the bioreactor is filled with media and inoculated with cells. As the cells grow and consume the initial media in the system, fresh feeds of nutrients are introduced as required throughout the run. The addition of nutrients throughout the process prevents the depletion of nutrients and allows additional cell growth. The dynamics of the distinct phases of production (e.g., lag phase, exponential growth phase, and stationary phase) make it challenging to use a single modeling approach for the entire process. Predictive modeling using machine learning algorithms can be designed to learn from data over time.

4.3.2 Context of use

Machine learning is used to predict product concentration with a day advance for a fed-batch bioreactor process (Bayrak et al., 2018). The dynamics of a fed batch cell culture bioreactor in the protein production phase are based on discrete offline measured variables including viable cell density, viability, glucose, salts, and amino acids, continuous inputs including pH, temperature, dissolved oxygen, and daily protein concentration. An adaptive model selection algorithm is developed to select between five machine learning models (Support Vector Machines, Gaussian Process Regression, Partial Least Squares Regression), regression trees (RT) and ensemble trees (ET). The predicted product concentration enables operators to take preventive actions to avoid an undesired trajectory for the process. Model predictions are confirmed with offline measurement of product concentration.

4.3.3 Determination of model risk

• Model Influence: Medium

The model plays an important role in operating the bioreactor and can trigger process interventions to prevent a failed batch.

• Decision Consequences: Low

Model predictions are confirmed with offline product concentration testing. Downstream controls are in place to detect non-conforming product. Product-specific risk factors may shift the decision consequences depending on the sensitivity of critical product quality attributes to the process trajectory.

The final overall model risk might be low-medium.

4.3.4 Model maintenance

Model predictions are confirmed with periodic offline product concentration testing which can enable a performance-based model maintenance approach. The root mean square prediction error and model prediction bias acceptance criteria can be specified. The impact of changes to models can be determined based on the model risk (low-medium), and on the impact of the change on model's performance and product quality. For this example, minor changes might be retraining the model with data from additional bioreactor production runs that result in changed model parameters. Moderate changes might be loosening acceptable model performance criteria (e.g., the root mean square prediction error) or changing model structure (e.g., additional model inputs to maintain model performance).

4.4 Example 4

Multi-Omics Analyses and Predictive Modeling: Identifying CQAs of cell therapy products by leveraging ML to correlate cell product characterization data to patient outcomes.

4.4.1 Background on question of interest

Unlike small molecule drugs, API for cell therapies are live cells that respond to surrounding conditions and possess a qualitative functional fitness corresponding to the donor. Researchers and developers recognize the need for CQAs for cell-based therapies that correspond to functional performance and relate to clinical efficacy (Van Grouw et al., 2023c).

Using comprehensive characterization and targeted/personalized performance assays, multiple product attributes can be correlated to results from performance assays using computational predictive modeling (Van Grouw et al., 2023d; Srinivasan et al., 2022b). This might increase the probability of identifying novel CQAs that are indicative of clinically relevant function. Indeed, deep characterization of heterogeneous cell-based therapy products through multi-omics approaches (i.e., combining data from genomics, transcriptomics, epigenetics, and proteomics) may enhance the understanding of product variability and identify CQAs that are predictive of clinical efficacy.

4.4.2 Context of use

The goal of this modeling approach is to identify CQAs of several cell therapy products based on input data derived from characterization studies that are predictive of their clinical performance for a specific indication. In this example, novel CQAs specific to each cell therapy product will be identified using complex, high-dimensional correlation-based predictive modeling strategies. CQAs are identified from correlating multi-omics, high-content analyses to measured performance outputs from in vitro and/or in vivo assays. The modeling approach leverages ML to correlate extensive cell product characterization data to patient outcomes (for the specified clinical indication) obtained from clinical investigations.

4.4.3 Determination of model risk

• Model Influence: Low

The model plays a role in product design or process development.

• Decision Consequences: Low-Medium

Model prediction is used for identifying CQAs of cell-based therapy products that may be used to inform the development of in-process monitoring and/or lot release specifications.

The final overall model risk might be low.

4.4.4 Model maintenance

For many cell therapies (including gene-edited cells), the identification of product attributes that are predictive of clinical efficacy remains a significant challenge (Levy et al., 2020). Though this example is more focused on product design and process development and not a process model per se, there are still potential maintenance considerations over the lifecycle of product. For example, a high impact change might be a revision of the model resulting in identifying additional CQAs or changing existing CQAs based on including additional training data into the model when more batch data and/or clinical data are available after product approval.

The following figure illustrates the evaluation of the overall model risk as a function of the model influence and decision consequence as discussed above for each of the four examples.

5 Conclusion

Process models play an increasing role in pharmaceutical product development and manufacturing. Published examples illustrate that process models have been applied for product design, process design and scale-up, process monitoring, and process control across multiple types of pharmaceutical manufacturing processes. Stakeholders have noted challenges related to model validation, model lifecycle maintenance, and international harmonization which has the potential to slow the adoption of process models (Cogoni et al., 2021c). Continuing advances in AI may raise additional model validation and maintenance challenges (U.S. Food and Drug, Administration Center for Drug Evaluation and Research, 2023).

The ICH guidance for industry Q8, Q9, & Q10 Questions and Answers; Appendix Q&As from Training Sessions (July 2012) established the principle that the level of oversight and documentation for a model should be commensurate with the level of risk associated with the use of the specific model in assuring the quality of the product. The ASME V&V 40 standard is aligned with this principle and describes how a manufacturer should use the model's context of use to assess model risk on product quality considering both model influence and decision consequences. The ASME V&V 40 standard provides a framework for evaluating model risk by considering the influence of the model and its decision consequence. A model verification and validation plan might then be developed based on this risk determination. To illustrate, consider two NIR chemometric models used for in-process monitoring of blend uniformity for two different drug products. In one case, the model is used for a high-dose drug product at low risk for content uniformity issues. In the other case, the model is used for a low-dose drug product with high content uniformity risk. Both models are used as in-process controls, and as such the models have the same influence on the assessment of blend uniformity. However, factoring in the inherent risks to product quality, and decision consequence might lead to the determination that these models have different levels of risk (low risk for the high-dose drug product; medium risk for the low-dose drug product).

An approach to model lifecycle management might also be based upon risk, considering the impact of change on the model's performance and the potential impact of the change on product quality. Scientific knowledge of the model and understanding of the product and process might be used to anticipate events that impact the model's performance and thus be used to develop change control management plans including performance-based approaches. Case studies illustrate how one might determine model risk and how risk could inform model validation and lifecycle maintenance. In general, detailed plans about model maintenance are maintained onsite as a component of the manufacturing site's Pharmaceutical Quality System (PQS). The manufacturer documents the assessment and the final determination on if and how the model will be updated, in accordance with the model maintenance plan.

Continued engagement among academics, industry and regulators will help refine model risk frameworks for pharmaceutical manufacturing. Published cases studies that describe model risk assessment, the specific verification and validation studies conducted, and the subsequent evaluation of model credibility will continue to provide value to the field. The adoption of a reliable model risk framework may help to increase the implementation of process models in pharmaceutical manufacturing and contribute to the consistent supply of high-quality medicines.

Disclaimer

This publication reflects the views of the authors and should not be construed to represent FDA's views or policies.

The views expressed in this article are the personal views of the author(s) and may not be understood or quoted as being made on behalf of or reflecting the position of the European Medicines Agency or one of its committees or working parties.

CRediT authorship contribution statement

Thomas F. O'Connor: Writing – original draft, Visualization, Conceptualization. Sharmista Chatterjee: Writing – original draft, Visualization, Conceptualization. Johnny Lam: Writing – review & editing. Dolores Hernán Pérez de la Ossa: Writing – review & editing. Leticia Martinez-Peyrat: Writing – review & editing. Marcel H.N. Hoefnagel: Writing – review & editing. Adam C. Fisher: Writing – review & editing, Supervision.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability

No data was used for the research described in the article.

Acknowledgements

We would like to acknowledge the following for their contributions in technical editing of the manuscript and support: Wen Seeto (CBER), Steven Oh (CBER), Manuel Osorio (CBER), Cinque Soto (CBER), John Wan (CDER), Lisa Hughey (CDER), and Russie Tran (CDER).
==== Refs
References

Arden N.S. Fisher A.C. Tyner K. Yu L.X. Lee S.L. Kopcha M. Industry 4.0 for pharmaceutical manufacturing: Preparing for the Smart Factories of the Future Int. J. Pharm. 602 2021 120554 March 29 https://www.sciencedirect.com/science/article/pii/S0378517321003598 33794326
Assessing the Credibility of Computational Modeling and Simulation in Medical Device Submissions: Guidance for Industry and Food and Drug Administration Staff U.S. Department of Health and Human Services - Food and Drug Administration - Center for Devices and Radiological Health https://www.fda.gov/media/154985/download 2023
Assessing the credibility of computational modeling through verification and validation: Application to medical devices V&V 40 American Society of Mechanical Engineers https://www.asme.org/codes-standards/find-codes-standards/v-v-40-assessing-credibility-computational-modeling-verification-validation-application-medical-devices 2018
Assessing the credibility of computational modeling through verification and validation: Application to medical devices V&V 40 American Society of Mechanical Engineers https://www.asme.org/codes-standards/find-codes-standards/v-v-40-assessing-credibility-computational-modeling-verification-validation-application-medical-devices 2018
Assessing the credibility of computational modeling through verification and validation: Application to medical devices V&V 40 American Society of Mechanical Engineers https://www.asme.org/codes-standards/find-codes-standards/v-v-40-assessing-credibility-computational-modeling-verification-validation-application-medical-devices 2018
Bayrak E.S. Wang T. Tulsyan A. Coufal M. Undey C. Product Attribute Forecast: Adaptive Model Selection Using Real-Time Machine Learning 2018 IFAC-PapersOnLine October 8 https://www.sciencedirect.com/science/article/pii/S240589631831963
Bideault G. Scaccia A. Zahel T. Landeringer R.W. Daluwatte C. Verification and validation of computational models used in biopharmaceutical manufacturing: potential Application of the ASME verification and validation 40 standard and FDA proposed AI/ML Model Life Cycle Management Framework J. Pharm. Sci. 110 2021 1540 January 23 https://www.sciencedirect.com/science/article/abs/pii/S0022354921000319 33493480
BioPhorum Industry Proposal: Regulatory Submission and Lifecycle Management Strategy of Models used in the Manufacture of Pharmaceutical and Biological Products 2021 10.46220/2020REG002 January
Chatterjee S. Moore C.M.V. Nasr M.M. An Overview of the Role of Mathematical Models in Implementation of Quality by Design Paradigm for Drug Development and Manufacture 2017 DigitalCommons@University of Nebraska - Lincoln https://digitalcommons.unl.edu/usfda/23/
Cogoni G. Liu Y.A. Husain A. Alam M.A. Kamyar R. A hybrid NIR-Soft Sensor Method for real time in-process control during continuous direct compression manufacturing operations Int. J. Pharm. 602 2021 120620 10.1016/j.ijpharm.2021.120620
Cogoni G. Liu Y.A. Husain A. Alam M.A. Kamyar R. A hybrid NIR-Soft Sensor Method for real time in-process control during continuous direct compression manufacturing operations Int. J. Pharm. 602 2021 120620 10.1016/j.ijpharm.2021.120620
Cogoni G. Liu Y.A. Husain A. Alam M.A. Kamyar R. A hybrid NIR-Soft Sensor Method for real time in-process control during continuous direct compression manufacturing operations Int. J. Pharm. 602 2021 120620 10.1016/j.ijpharm.2021.120620
Craven S. Whelan J. Glennon B. Glucose concentration control of a fed-batch mammalian cell bioprocess using a nonlinear model Predictive Controller J. Process Control 24 2014 344 March 21 https://www.sciencedirect.com/science/article/abs/pii/S0959152414000584
Ding C. Gerberich C. Ierapetritou M. Hybrid model development for parameter estimation and process optimization of hydrophobic interaction chromatography J. Chromatogr. A 1703 2023 464113 ISSN 0021-9673 10.1016/j.chroma.2023.464113
Doron G. Klontzas M.E. Mantalaris A. Guldberg R.E. Temenoff J.S. Multiomics characterization of mesenchymal stromal cells cultured in monolayer and as aggregates Biotechnol. Bioeng. 117 6 2020 1761 1778 Jun 32100872
EMA CHMP guideline on the use of near infrared spectroscopy by the pharmaceutical industry and the data requirements for new submissions and variations (EMEA/CHMP/CVMP/QWP/17760/2009 Rev2) June https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-use-near-infrared-spectroscopy-pharmaceutical-industry-and-data-requirements-new-submissions-and-variations_en.pdf 2014
EMA CHMP guideline on the use of near infrared spectroscopy by the pharmaceutical industry and the data requirements for new submissions and variations (EMEA/CHMP/CVMP/QWP/17760/2009 Rev2) June https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-use-near-infrared-spectroscopy-pharmaceutical-industry-and-data-requirements-new-submissions-and-variations_en.pdf 2014
EMA Preliminary QIG Considerations regarding Pharmaceutical 5 Process Models Quality Innovation Group | European Medicines Agency (europa.eu) 2024 February
Erdemir A. Mulugeta L. Ku J.P. Credible practice of modeling and simulation in healthcare: ten rules from a multidisciplinary perspective J. Transl. Med. 18 2020 369 10.1186/s12967-020-02540-4 32993675
EU CHMP guideline on manufacture of the finished dosage form (EMA/CHMP/QWP/245074/2015) 2015
Executive Order 14110 of October 30 Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence https://www.federalregister.gov/d/2023-24283 2023
Garcia-Munoz S. Butterbaugh A. Leavesley I. Manley L.F. Slade D. Bermingham S. A Flowsheet Model for the Development of a Continuous Process for Pharmaceutical Tablets: An Industrial Perspective 2017 AIChE Online Library 10.1002/aic.15967
Gibbons L. Maslanka F. Le N. Magill A. Singh P. Mclaughlin J. Madden F. Hayes R. McCarthy B. Rode C. O’Mahony J. Rea R. O’Mahony-Hartnett C. An assessment of the impact of Raman based glucose feedback control on CHO cell bioreactor process development Biotechnol. Prog. 2023 10.1002/btpr.3371 June 27
Huang J. O’Connor T. Ahmed K. Chatterjee S. Garvin C. Ghosh K. Ierapetritou M. Jeffers M. Pla D.L. Lee S.L. Lovett D. Lynberg O. Mack J. McManus E. Romero-Torres S. Undey C. Venkatasubramanian V. Warman M. AiChe PD2M Advanced Process Control (APC) Workshop-Moving APC Forward in the Pharmaceutical Industry 2020, November 3 AIChE Online Library 10.1002/amp2.10071
Huang J. O’Connor T. Ahmed K. Chatterjee S. Garvin C. Ghosh K. Ierapetritou M. Jeffers M. Pla D.L. Lee S.L. Lovett D. Lynberg O. Mack J. McManus E. Romero-Torres S. Undey C. Venkatasubramanian V. Warman M. AiChe PD2M Advanced Process Control (APC) Workshop-Moving APC Forward in the Pharmaceutical Industry 2020, November 3 AIChE Online Library 10.1002/amp2.10071
Hurley S. Tantuccio A. Escotet-Espinoza M.S. Flamm M. Metzger M. Development and use of a residence time distribution (RTD) model control strategy for a continuous manufacturing drug product pharmaceutical process Pharmaceutics 14 2 2022 355 10.3390/pharmaceutics14020355 35214087
Hurley S. Tantuccio A. Escotet-Espinoza M.S. Flamm M. Metzger M. Development and use of a residence time distribution (RTD) model control strategy for a continuous manufacturing drug product pharmaceutical process Pharmaceutics 14 2 2022 355 10.3390/pharmaceutics14020355 35214087
ICH guideline on Q13 on continuous manufacturing of drug substances and drug products 2023 October 7
ICH Quality Implementation Working Group ICH quality implementation working group - points to consider (R2) - ICH-endorsed guide for ICH Q8/Q9/Q10 implementation 2011 Points to consider (R2) - ICH-Endorsed Guide for ICH Q8/Q9/Q10 Implementation - ECA Academy December 6 https://www.gmp-compliance.org/guidelines/gmp-guideline/ich-quality-implementation-working-group-points-to-consider-r2-ich-endorsed-guide-for-ich-q8-q9-q10-implementation
ICH Quality Implementation Working Group ICH quality implementation working group - points to consider (R2) - ICH-endorsed guide for ICH Q8/Q9/Q10 implementation 2011 Points to consider (R2) - ICH-Endorsed Guide for ICH Q8/Q9/Q10 Implementation - ECA Academy December 6 https://www.gmp-compliance.org/guidelines/gmp-guideline/ich-quality-implementation-working-group-points-to-consider-r2-ich-endorsed-guide-for-ich-q8-q9-q10-implementation
Kamyar R. Lauri Pla D. Husain A. Cogoni G. Wang Z. Soft sensor for real-time estimation of tablet potency in continuous direct compression manufacturing operation Int. J. Pharm. 602 2021 120624 10.1016/j.ijpharm.2021.120624
Kamyar R. Lauri Pla D. Husain A. Cogoni G. Wang Z. Soft sensor for real-time estimation of tablet potency in continuous direct compression manufacturing operation Int. J. Pharm. 602 2021 120624 10.1016/j.ijpharm.2021.120624
Kourti T. Lepore J. Liesum L. Nasr M. Chatterjee S. Moore C.M.V. Korakianiti E. Scientific and Regulatory Considerations for Implementing Mathematical Models in the Quality by Design (QbD) Framework Pharm. Eng. 34 6 2014 https://ispe.org/sites/default/files/pe/online-exclusives/14ND-online_Kourti.pdf
Kuemmel C. Yang Y. Zhang X. Florian J. Zhu H. Tegenge M. Huang S.-M. Wang Y. Morrison T. Zineh I. Consideration of a credibility assessment framework in model-informed drug development: potential application to physiologically-based pharmacokinetic modeling and simulation CPT Pharmacometrics Syst. Pharmacol. 9 1 2019, November 10 21 28 https://pubmed.ncbi.nlm.nih.gov/31652029/ 31652029
Kuemmel C. Yang Y. Zhang X. Florian J. Zhu H. Tegenge M. Huang S.-M. Wang Y. Morrison T. Zineh I. Consideration of a credibility assessment framework in model-informed drug development: potential application to physiologically-based pharmacokinetic modeling and simulation CPT Pharmacometrics Syst. Pharmacol. 9 1 2019, November 10 21 28 https://pubmed.ncbi.nlm.nih.gov/31652029/ 31652029
Kumar V. Leweke S. Heymann W. von Lieres E. Schlegel F. Westerberg K. Lenhoff A.M. Robust mechanistic modeling of protein ion-exchange chromatography J. Chromatogr. A 1660 2021 462669 10.1016/j.chroma.2021.462669
Lauri Pla D. Kamyar R. Hashemian N. Mehdizadeh H. Moshgbar M. Moisture soft sensor for batch fluid bed dryers: A practical approach Powder Technol. 326 2018 69 77 10.1016/j.powtec.2017.11.056
Lauri Pla D. Kamyar R. Hashemian N. Mehdizadeh H. Moshgbar M. Moisture soft sensor for batch fluid bed dryers: A practical approach Powder Technol. 326 2018 69 77 10.1016/j.powtec.2017.11.056
Levy O. Kuai R. Siren E.M. Bhere D. Milton Y. Nissar N. De Biasio M. Heinelt M. Reeve B. Abdi R. Alturki M. Shattering barriers toward clinically meaningful MSC therapies Sci. Adv. 6 30 2020 eaba6884
Galbraith S.C. Cha B. Huang Z. Park S. Liu H. Meyer R.F. Flamm M.H. Hurley S. Zhang-Plasket F. Yoon S. Integrated modeling of a continuous direct compression tablet manufacturing process: A production scale case study Powder Technol 354 2019 199 June 5 https://www.sciencedirect.com/science/article/abs/pii/S0032591019304255
Lipsitz Y. Timmins N. Zandstra P. Quality cell therapy manufacturing by design Nat. Biotechnol. 34 2016 393 400 April 7 https://www.nature.com/articles/nbt.3525 27054995
Liu L. Antibody glycosylation and its impact on the pharmacokinetics and pharmacodynamics of monoclonal antibodies and FC-fusion proteins J. Pharm. Sci. 104 6 2015 1866 1884 April 15 https://pubmed.ncbi.nlm.nih.gov/25872915/ 25872915
Liu Y.C. Acevedo D. Yang X. Naimi S. Wu W.-L. Pavurala N. Nagy Z.K. O’Connor T.F. Population Balance Model Development Verification and Validation of Cooling Crystallization of Carbamazepine 2020 ACS Publications 10.1021/acs.cgd.0c00434 June 20
Mautner K. Gottschalk M. Boden S.D. Akard A. Bae W.C. Black L. Boggess B. Chatterjee P. Chung C.B. Easley K.A. Gibson G. Cell-based versus corticosteroid injections for knee pain in osteoarthritis: a randomized phase 3 trial Nat. Med. 29 12 2023 3120 3126 Dec 37919438
Moreno-Benito M. Lee K.T. Kaydanov D. Verrier H.M. Blackwood D.O. Doshi P. Digital twin of a continuous direct compression line for drug product and process design using a hybrid flowsheet modelling approach Int. J. Pharm. 628 2022 122336 10.1016/j.ijpharm.2022.122336
Moreno-Benito M. Lee K.T. Kaydanov D. Verrier H.M. Blackwood D.O. Doshi P. Digital twin of a continuous direct compression line for drug product and process design using a hybrid flowsheet modelling approach Int. J. Pharm. 628 2022 122336 10.1016/j.ijpharm.2022.122336
Q12 Technical and Regulatory Considerations for Pharmaceutical Product Lifecycle Management Guidance for Industry U.S. Department of Health and Human Services - Food and Drug Administration - Center for Drug Evaluation and Research (CDER) - Center for Biologics Evaluation and Research (CBER) May https://www.fda.gov/media/148476/download 2021
Rashedi M. Khodabandehlou H. Demers M. Wang T. Garvin C. Model Predictive Controller Design for bioprocesses based on machine learning algorithms 2022 IFAC-PapersOnLine August 5 https://www.sciencedirect.com/science/article/pii/S2405896322008217
Rashedi M. Khodabandehlou H. Demers M. Wang T. Garvin C. Model Predictive Controller Design for bioprocesses based on machine learning algorithms 2022 IFAC-PapersOnLine August 5 https://www.sciencedirect.com/science/article/pii/S2405896322008217
Reddy J.V. Raudenbush K. Papoutsakis E.T. Ierapetritou M. Cell-culture process optimization via model-based predictions of metabolism and protein glycosylation Biotechnol. Adv. 67 2023 108179
Rischawy F. Saleh D. Hahn T. Oelmeier S. Spitz J. Kluters S. Good modeling practice for industrial chromatography: Mechanistic modeling of ion exchange chromatography of a bispecific antibody Comput. Chem. Eng. 130 2019 106532 10.1016/j.compchemeng.2019.106532
Rischawy F. Saleh D. Hahn T. Oelmeier S. Spitz J. Kluters S. Good modeling practice for industrial chromatography: Mechanistic modeling of ion exchange chromatography of a bispecific antibody Comput. Chem. Eng. 130 2019 106532 10.1016/j.compchemeng.2019.106532
Rischawy F. Saleh D. Hahn T. Oelmeier S. Spitz J. Kluters S. Good modeling practice for industrial chromatography: Mechanistic modeling of ion exchange chromatography of a bispecific antibody Comput. Chem. Eng. 130 2019 106532 10.1016/j.compchemeng.2019.106532
Rischawy F. Saleh D. Hahn T. Oelmeier S. Spitz J. Kluters S. Good modeling practice for industrial chromatography: Mechanistic modeling of ion exchange chromatography of a bispecific antibody Comput. Chem. Eng. 130 2019 106532 10.1016/j.compchemeng.2019.106532
Rogers A.J. Inamdar C. Ierapetritou M.G. An Integrated Approach to Simulation of Pharmaceutical Processes for Solid Drug Manufacture 2014 The State University of New Jersey Rutgers April 2 https://www.researchwithrutgers.com/en/publications/an-integrated-approach-to-simulation-of-pharmaceutical-processes
Roy K. Wang B. Chen R.Q. Li J. Interfacing data science with cell therapy manufacturing: where we are and where we need to be Cytotherapy S1465-3249(24)00099-9Apr 4 2024
Saleh D. Wang G. Müller B. Rischawy F. Kluters S. Studts J. Hubbuch J. Straightforward method for calibration of mechanistic cation exchange chromatography models for industrial applications Biotechnol. Prog. 36 4 2020 10.1002/btpr.2984
Saleh D. Wang G. Mueller B. Rischawy F. Kluters S. Studts J. Hubbuch J. Cross-scale quality assessment of a mechanistic cation exchange chromatography model Biotechnol. Prog. 37 1 2020 10.1002/btpr.3081
Saleh D. Wang G. Mueller B. Rischawy F. Kluters S. Studts J. Hubbuch J. Cross-scale quality assessment of a mechanistic cation exchange chromatography model Biotechnol. Prog. 37 1 2020 10.1002/btpr.3081
Shepard T. Scott G. Cole S. Nordmark A. Bouzom F. Physiologically based models in regulatory submissions: output from the ABPI/MHRA forum on physiologically based modeling and simulation CPT Pharmacometrics Syst. Pharmacol. 4 4 2015 221 225 10.1002/psp4.30 Epub 2015 Apr 2. PMID: 26225245; PMCID: PMC4429575 26225245
Srinivasan A. Sathiyanathan P. Yin L. Liu T.M. Lam A. Ravikumar M. Smith R.A. Loh H.P. Zhang Y. Ling L. Ng S.K. Strategies to enhance immunomodulatory properties and reduce heterogeneity in mesenchymal stromal cells during ex vivo expansion Cytotherapy 24 5 2022 456 472 May 1 35227601
Srinivasan A. Sathiyanathan P. Yin L. Liu T.M. Lam A. Ravikumar M. Smith R.A. Loh H.P. Zhang Y. Ling L. Ng S.K. Strategies to enhance immunomodulatory properties and reduce heterogeneity in mesenchymal stromal cells during ex vivo expansion Cytotherapy 24 5 2022 456 472 May 1 35227601
Tchessalov S. Latshaw D. Nulu S. Bentley M. Tharp T. Ewan S. Chen X. Application of first principles primary drying model to lyophilization process design and transfer: Case studies from the industry J. Pharm. Sci. 110 2021 2021 968 981 10.1016/j.xphs.2020.11.013 33248058
Tchessalov S. Latshaw D. Nulu S. Bentley M. Tharp T. Ewan S. Chen X. Application of first principles primary drying model to lyophilization process design and transfer: Case studies from the industry J. Pharm. Sci. 110 2021 2021 968 981 10.1016/j.xphs.2020.11.013 33248058
Tchessalov S. Latshaw D. Nulu S. Bentley M. Tharp T. Ewan S. Chen X. Application of first principles primary drying model to lyophilization process design and transfer: Case studies from the industry J. Pharm. Sci. 110 2021 2021 968 981 10.1016/j.xphs.2020.11.013 33248058
Tchessalov S. Latshaw D. Nulu S. Bentley M. Tharp T. Ewan S. Chen X. Application of first principles primary drying model to lyophilization process design and transfer: Case studies from the industry J. Pharm. Sci. 110 2021 2021 968 981 10.1016/j.xphs.2020.11.013 33248058
The American Society of Mechanical Engineers Verification, Validation and Uncertainty Quantification (VVUQ) https://www.asme.org/codes-standards/publications-information/verification-validation-uncertainty 2024
Tian G. Koolivand A. Arden N.S. Lee S. O’Connor T.F. Quality Risk Assessment and mitigation of pharmaceutical continuous manufacturing using flowsheet modeling approach Comput. Chem. Eng. 129 2019 106508 July 5 https://www.sciencedirect.com/science/article/abs/pii/S0098135419301139?via%3Dihub
Torres-Garcia W. Roy K. Kotanchek T. Edison A. Levine B. Colonna M. Dwarshuis N. Odeh-Couvertier V. Predicting T-cell quality during manufacturing through an artificial intelligence-based integrative multiomics analytical platform Bioengi. Transl. Med. 2021 10.1002/btm2.10282 , December 16
Torres-Garcia W. Roy K. Kotanchek T. Edison A. Levine B. Colonna M. Dwarshuis N. Odeh-Couvertier V. Predicting T-cell quality during manufacturing through an artificial intelligence-based integrative multiomics analytical platform Bioeng. Transl. Med. 2021 10.1002/btm2.10282 , December 16
Toye D. Sion C. Olmos E. Maillot C. De Isla Natalia Quality by design to define critical process parameters for mesenchymal stem cell expansion Biotechnol. Adv. 50 2021 107765 May 21 https://www.sciencedirect.com/science/article/abs/pii/S0734975021000719 33961977
U.S. Food & Drug Administration Pharmaceutical CGMPs for the 21st century - A Risk-Based Approach Final Report Retrieved August 25, 2023, from https://www.fda.gov/media/77391/download 2004
U.S. Food & Drug Administration Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) – Discussion Paper and Request for Feedback. FDA-2019-0156 submitted April 2 Regulations.gov 2019 https://www.regulations.gov/document/FDA-2019-N-1185-0001
U.S. Food & Drug Administration Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan. FDA-2019-0156 January https://www.fda.gov/media/145022/download 2021
U.S. Food & Drug Administration, Health Canada Medical Healthcare products Regulatory Agency, Good Machine Learning Practice for Medical Device Development Guiding Principles FDA-2019-0156. (submitted December 2 Regulations.gov 2021 https://www.regulations.gov/document/FDA-2019-N-1185-0156
U.S. Food and Drug, Administration Center for Drug Evaluation and Research Development and submission of Near Infrared Analytical Procedures 2021 U.S. Food and Drug Administration August https://www.fda.gov/regulatory-information/search-fda-guidance-documents/development-and-submission-near-infrared-analytical-procedures
U.S. Food and Drug, Administration Center for Drug Evaluation and Research Development and submission of Near Infrared Analytical Procedures 2021 U.S. Food and Drug Administration August https://www.fda.gov/regulatory-information/search-fda-guidance-documents/development-and-submission-near-infrared-analytical-procedures
U.S. Food and Drug, Administration Center for Drug Evaluation and Research Development and submission of Near Infrared Analytical Procedures 2021 U.S. Food and Drug Administration August https://www.fda.gov/regulatory-information/search-fda-guidance-documents/development-and-submission-near-infrared-analytical-procedures
U.S. Food and Drug, Administration Center for Drug Evaluation and Research Artificial Intelligence in Drug Manufacturing March https://www.fda.gov/media/165743/download?attachment 2023
Van Grouw A. Colonna M.B. Maughon T.S. Shen X. Larey A.M. Moore S.G. Yeago C. Fernández F.M. Edison A.S. Stice S.L. Bowles-Welch A.C. Development of a robust consensus modeling approach for identifying cellular and media metabolites predictive of mesenchymal stromal cell potency Stem Cells 41 8 2023 792 808 Aug 1 37279550
Van Grouw A. Colonna M.B. Maughon T.S. Shen X. Larey A.M. Moore S.G. Yeago C. Fernández F.M. Edison A.S. Stice S.L. Bowles-Welch A.C. Development of a robust consensus modeling approach for identifying cellular and media metabolites predictive of mesenchymal stromal cell potency Stem Cells 41 8 2023 792 808 Aug 1 37279550
Van Grouw A. Colonna M.B. Maughon T.S. Shen X. Larey A.M. Moore S.G. Yeago C. Fernández F.M. Edison A.S. Stice S.L. Bowles-Welch A.C. Development of a robust consensus modeling approach for identifying cellular and media metabolites predictive of mesenchymal stromal cell potency Stem Cells 41 8 2023 792 808 Aug 1 37279550
Van Grouw A. Colonna M.B. Maughon T.S. Shen X. Larey A.M. Moore S.G. Yeago C. Fernández F.M. Edison A.S. Stice S.L. Bowles-Welch A.C. Development of a robust consensus modeling approach for identifying cellular and media metabolites predictive of mesenchymal stromal cell potency Stem Cells 41 8 2023 792 808 Aug 1 37279550
Wang Z. Escotet-Espinoza M.S. Ierapetritou M. Process analysis and optimization of continuous pharmaceutical manufacturing using flowsheet models Comput. Chem. Eng. 107 2017 77 91 February 21 https://www.sciencedirect.com/science/article/abs/pii/S0098135417300947
Xu H. Xiong W. Tsao W.-C. Ramos C. Naghizadeh A. Mohamed M. Metaxas D. Liu D. Li D. Hyun Cho J. In vitro machine learning-based CAR T immunological synapse quality measurements correlate with patient clinical outcomes 2022 PLOS Computational Biology 10.1371/journal.pcbi.1009883 March 18
Xu H. Xiong W. Tsao W.-C. Ramos C. Naghizadeh A. Mohamed M. Metaxas D. Liu D. Li D. Hyun Cho J. In vitro machine learning-based CAR T immunological synapse quality measurements correlate with patient clinical outcomes 2022 PLOS Computational Biology 10.1371/journal.pcbi.1009883 March 18
Yeago C. Bowles-Welch A. Van Grouw A. Stice S. Shen X. Moore S. Maughon T. Marklein R. Larey A. Fernández F. Edison A. Colonna M. Development of a robust consensus modeling approach for identifying cellular and media metabolites predictive of mesenchymal stromal cell potency Stem Cells 41 8 2023 792 808 June 3 https://academic.oup.com/stmcls/article/41/8/792/7190158 37279550
Yon J.-M. Young Lee S. Won Kim S. Park K. Lee S. Kim H. Jeong I. Jang J. Cho D.-W. Predicting multipotency of human adult stem cells derived from various donors through deep learning Sci. Rep. 12 2022 21614 December 14 https://www.nature.com/articles/s41598-022-25423-8 36517519
Zhu T. Moussa E.M. Witting M. Zhou D. Sinha K. Hirth M. Gastens M. Shang S. Nere N. Somashekar S.C. Alexeenko A. Jameel F. Predictive models of lyophilization process for development, scale-up/tech transfer and manufacturing Eur. J. Pharm. Biopharm. 128 2018 363 378 10.1016/j.ejpb.2018.05.005 29733948
Zomer S. Zhang J. Talwar S. Chattoraj S. Hewitt C. Multivariate monitoring for the industrialisation of a continuous wet granulation tableting process Int. J. Pharm. 547 1-2 2018 June 12 506 519 https://www.sciencedirect.com/science/article/abs/pii/S0378517318304320 29906561
Zomer S. Zhang J. Talwar S. Chattoraj S. Hewitt C. Multivariate monitoring for the industrialisation of a continuous wet granulation tableting process Int. J. Pharm. 547 1-2 2018 June 12 506 519 https://www.sciencedirect.com/science/article/abs/pii/S0378517318304320 29906561
Zylberberg C. Zhang J. Saha K. Roy K. McFarland R. Lumelsky N. Lin-Gibson S. Haddock R. Manufacturing Cell Therapies: The Paradigm Shift in Health Care of This Century 2017 National Academy of Medicine June 23 https://nam.edu/manufacturing-cell-therapies-the-paradigm-shift-in-health-care-of-this-century/
