
==== Front
Ther Drug Monit
Ther Drug Monit
tdm
Therapeutic Drug Monitoring
0163-4356
1536-3694
Therapeutic Drug Monitoring

38758633
TDM23-221
10.1097/FTD.0000000000001214
00004
3
Original Article
Developing Parametric and Nonparametric Models for Model-Informed Precision Dosing: A Quality Improvement Effort in Vancomycin for Patients With Obesity
Hughes Maria-Stephanie A. PharmD *
Hughes Jasmine H. PhD *
Endicott Jeffrey PharmD †
Langton Meagan PharmD †
Ahern John W. PharmD †
Keizer Ron J. PharmD, PhD *
* InsightRX, San Francisco, California; and
† University of Vermont Medical Center, Burlington, Vermont.
Correspondence: Maria-Stephanie A. Hughes, PharmD, 548 Market St #88083, San Francisco, CA 94104 (e-mail: stephanie@insight-rx.com).
10 2024
10 5 2024
46 5 575583
22 11 2023
02 3 2024
Copyright © 2024 The Author(s). Published by Wolters Kluwer Health, Inc.
2024
https://creativecommons.org/licenses/by-nc-nd/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution-Non Commercial-No Derivatives License 4.0 (CCBY-NC-ND), where it is permissible to download and share the work provided it is properly cited. The work cannot be changed in any way or used commercially without permission from the journal.

Supplemental Digital Content is Available in the Text.

Background:

Both parametric and nonparametric methods have been proposed to support model-informed precision dosing (MIPD). However, which approach leads to better models remains uncertain. Using open-source software, these 2 statistical approaches for model development were compared using the pharmacokinetics of vancomycin in a challenging subpopulation of class 3 obesity.

Methods:

Patients on vancomycin at the University of Vermont Medical Center from November 1, 2021, to February 14, 2023, were entered into the MIPD software. The inclusion criteria were body mass index (BMI) of at least 40 kg/m2 and 1 or more vancomycin levels. A parametric model was created using nlmixr2/NONMEM, and a nonparametric model was created using Pmetrics. Then, a priori and a posteriori predictions were evaluated using the normalized root mean squared error (nRMSE) for precision and the mean percentage error (MPE) for bias. The parametric model was evaluated in a simulated MIPD context using an external validation dataset.

Results:

In total, 83 patients were included in the model development, with a median age of 56.6 years (range: 24–89 years), and a median BMI of 46.3 kg/m2 (range: 40–70.3 kg/m2). Both parametric and nonparametric models were 2-compartmental, with creatinine clearance and fat-free mass as covariates to clearance and volume parameters, respectively. The a priori MPE and nRMSE for the parametric versus nonparametric models were −6.3% versus 2.69% and 27.2% versus 30.7%, respectively. The a posteriori MPE and RMSE were 0.16% and 0.84%, and 13.8% and 13.1%. The parametric model matched or outperformed previously published models on an external validation dataset (n = 576 patients).

Conclusions:

Minimal differences were found in the model structure and predictive error between the parametric and nonparametric approaches for modeling vancomycin class 3 obesity. However, the parametric model outperformed several other models, suggesting that institution-specific models may improve pharmacokinetics management.

Key Words:

model-informed precision dosing
obese
parametric/nonparametric
pharmacokinetic models
vancomycin
OPEN-ACCESSTRUE
SDCT
==== Body
pmcBACKGROUND

Therapeutic monitoring of vancomycin has been a standard practice for over a decade because of its narrow therapeutic index and variable pharmacokinetics (PK).1,2 Initial dose selection may be guided by either dosing nomograms or PK models made available in model-informed precision dosing (MIPD); however, serum drug concentrations are still required to check if the patient is at the therapeutic target exposure to adjust doses accordingly. With professional association consensus guidelines recommending area under the concentration–time curve (AUC) exposure targets, vancomycin dosing is increasingly performed using MIPD software because AUC calculation is computationally challenging. Model-based approaches have the advantage of considering patient covariates (eg, age, weight, sex, height, and serum creatinine) and allow Bayesian forecasting for more individualized dosing.3,4 However, model-based approaches require models that adequately describe patient PK, and certain subpopulations have been identified as having different and more variable PK compared with the general population. These subpopulations are at an increased risk of needing more dose adjustments and, therefore, have delayed or reduced therapeutic effects.5,6 One particularly relevant subpopulation is obesity, with adults with obesity comprising 12.5% of the population globally and 23% of the American population.7,8

Patients with obesity, particularly those in the class 3 obesity category (body mass index [BMI] ≥40 kg/m2), showed different relationships between vancomycin PK and predictors compared with their nonobese counterparts. Fortunately, studies dedicated to describing these differences have resulted in vancomycin PK models specific to obese population.9–12 These PK models can guide obesity-specific vancomycin dosing when embedded in the MIPD software, and dose selection can be further individualized in clinical practice while allowing for the calculation of the vancomycin AUC.3 Selecting a suitable model for a patient population can be challenging and crucial given its significant impact on dosing decisions. The challenge often lies in the fact that even though the population on which the model was built matches the intended target population, underlying differences in patient populations, modeling approaches, or other factors may affect model suitability.13–15

Models may vary in many respects, including their statistical assumptions and handling of between-patient variability. Both parametric and nonparametric models have been developed and applied for therapeutic drug monitoring (TDM) and are available in MIPD software programs for clinical use, and each model has its own strengths and limitations.16,17 Parametric models use continuous parametric probability distributions to describe variability. Nonparametric models differ in that they use the entire discrete distribution of the Bayesian posterior PK parameters rather than relying on 1 set of PK parameter values. Each approach has various advantages and disadvantages, which have been reviewed elsewhere.16–19 It has, for example, been suggested that nonparametric may be better suited for use in individualized dosing because of their ability to better capture subpopulations and outliers. However, it still remains unclear whether the theoretical advantages affect predictive performance at the point-of-care in a meaningful way.20,21

In addition to the difference in statistical assumptions and estimation methods, models differ in their PK parameter estimates and model structure owing to differences in the population data informing the models, even when developed on the same specific patient subpopulation. Differences may arise from variations in the exact inclusion criteria and final distribution of age, weight, and renal function in the model development population. Intuitively, clinical pharmacists tasked with implementing MIPD at their institutions may therefore favor selecting models that most closely represent the patient population they care for. However, Colin et al22 found that a model developed on a combination of obese and nonobese populations performed better than or similar to those that were specific to obese populations, suggesting that models developed on larger datasets with adequate representation of subgroups may also result in performance models for those subgroups. Other studies have refitted or trained models based on institution-specific populations23; in theory, this approach reduces or eliminates mismatches in model development populations and the target MIPD population.

As part of the quality improvement process, we searched for refinements in vancomycin PK models for patients with class 3 obesity at the University of Vermont Medical Center (UVMMC). The objectives of the quality improvement effort were to first compare the predictiveness of parametric and nonparametric models developed on a UVMMC-specific patient dataset and then to compare the newly developed models to other currently published models for the obese population to determine if there was an advantage to this institution's customization approach.

MATERIALS AND METHODS

Ethics

Data for this analysis were collected as part of routine health care operations at contracted health care entities and were de-identified before analysis. According to the policy-defining activities that constitute research at the University of Vermont/University of Vermont Health Network, this study met the criteria for operational improvement activities and was exempted from IRB review.

Data Collection

Data were entered by clinical pharmacists into the MIPD software (InsightRX Nova), both during the trial period (before the pharmacy-wide rollout of the MIPD software) and during routine clinical use. Dosing histories, vancomycin TDM levels, and patient characteristics were exported from the software database from November 1, 2021, to February 14, 2023. Patients were included in the analysis if they had a BMI of at least 40 kg/m2 at any point during their treatment and at least 1 vancomycin TDM sample was collected during this time.

The clinical protocol at UVMMC stated that patients with a BMI of ≥40 kg/m2 receiving vancomycin were to be administered a loading dose of 20 mg/kg (with a maximum dose of 4000 mg) and then started on an intermittent infusion maintenance dosing regimen. After the loading dose, two levels were obtained at 2 hours post dose and trough, respectively. Maintenance dosing regimens were selected if predicted to get to a therapeutic AUC target of 400–600 mg × h/L while applying the obese-specific model by Carreno et al9 in the MIPD software. A follow-up trough level was obtained once the patient was in a steady-state and weekly thereafter, and the dosing regimens were adjusted accordingly.

Available covariates included age, sex, total body weight (TBW), height, and serum creatinine, as well as the derived covariates BMI, lean body weight (LBW, Green equation24), ideal body weight (IBW, Devine equation25), adjusted body weight (AjBW, Winter equation26) fat-free mass (FFM, Janmahasatian equation24), and creatinine clearance (CrCl). The FFM equations were as follows: FFM for males = (9270.0 × weight)/(6680.0 + 216.0 × BMI) and FFM for females = (9270.0 × weight)/(8780.0 + 244.0 × BMI). The Modification of Diet in Renal Disease (MDRD) Study equation and the Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) were used to calculate the GFR. CrCl was calculated using the Cockcroft–Gault equation with TBW, LBW, IBW, AjBW, or FFM. Covariates were available for all individuals, and all TDM levels were above the lower limit of quantification. An external validation dataset was created using the same inclusion criteria covering a period between January 1 and March 20, 2023, for patients treated with vancomycin using InsightRX Nova across other institutions within the United States. Details of the clinical protocols from these organizations are not available; however, the TDM and dosing regimens of the included treatment courses have been summarized.

Data Cleaning

An initial (parametric)-based PK model was fitted to the exported dataset, and the individual weighted residuals were calculated and exported. TDM levels with a weighted residual greater than 3 were flagged as potential outliers. Manual visual inspection of the PK profiles associated with these potential outliers was then performed to evaluate whether the levels were likely to be data errors and should be removed or just a reflection of aberrant PK and should be retained.

Model Development

During model development, only 2-compartment models were considered because vancomycin is known to follow multicompartmental PK.27 Some authors have reported 3-compartment models for vancomycin11; however, we deemed such a model not structurally identifiable because our dataset was not densely sampled. For both the parametric and the nonparametric approaches, a base model without any covariates was defined, on which all available covariates were tested. For both approaches, improvements in the Akaike Information Criterion (AIC), model stability, and commonly used goodness-of-fit plots were used to guide model development. Between-subject variability (BSV) was evaluated according to standard model development approaches for each method: for the parametric model, BSV was tested on all PK parameters and retained, given the improvement in AIC and other diagnostics; for the nonparametric model, BSV was included on all basic PK parameters already in the base model and not evaluated using statistical criteria. For the parametric models, proportional and combined proportional/additive residual error models were tested, and their magnitudes were estimated. For the nonparametric model, a combined proportional (10%) and additive (0.5 mg/L) residual error model was implemented using the polynomial assay error, with values fixed to 0.1 (C1) and 0.5 (C0), respectively, and a lambda of 0.1, following common practice for nonparametric model development. The parameter uncertainty for all final models was calculated using bootstraps based on 200 resampled datasets. Model fit was assessed for both models according to population (a priori) and individualized (a posteriori) predictions quantified by r2, mean percentage error (MPE), and normalized root mean squared error (nRMSE) using the following equations:MPE=1N∑i=1Nŷ^i−yiyi

nRMSE=∑iNyiN 1N∑i=1N(ŷ^i−yi)2

where N is the number of TDMs, yi is the measured vancomycin concentration, and ŷ^i is the predicted concentration.

External Validation of Predictive Performance

The predictive performance of the final parametric model was evaluated by testing it against a robust external database. The predictive performance of the developed model was evaluated with an iterative approach using the PsN proseval28 and compared against the predictive performance of other vancomycin models developed for patients with obesity. Models were selected for inclusion based on a Google scholar search with the keywords “vancomycin,” “pharmacokinetic model,” and “obese adults” and were included if they were published from 2012 to March 2023, were a parametric model, and were not specific to any other special PK population (eg, septic shock, critically ill). These parametric models included those of Adane et al, Carreno et al, Colin et al, and Smit et al,9,11,12,22 and therefore, were included in this comparison. One notable difference between these models is that the model by Adane et al was 1-compartmental, whereas the others were 2-compartmental. In addition, the models by Carreno et al, Colin et al, and Smit et al were developed on patients who had several samples collected after vancomycin doses, whereas the model by Adane et al used peak and trough samples collected from patients. A priori and a posteriori MPE, nRMSE, and accuracy were measured and compared. Accuracy was defined as the percentage of predictions within 15% or 2.5 mg/L of the observed data. These relative and absolute thresholds were considered reasonable limits for both peaks and troughs, as the magnitudes of differences were considered unlikely to result in a discordance in the patients falling within the therapeutic AUC target of 400–600 mg/L × hr in most cases. MPE and nRMSE <25% were considered acceptable bias and precision for the same rationale, respectively, and were consistent with previously published analyses.29

Software

Throughout the analysis, R (version 4.2.0+) was used for data extraction and cleaning and for the analysis of model output and generation of model diagnostics. Visual predictive checks (VPCs) were performed using the R package vpc (v1.2.2).30 NONMEM v7.4 and nlmixr2 (version 2.0.9) were used as parametric modeling software.31 In both software programs, a stochastic approximation of expectation maximization (SAEM) estimation method was used. Pmetrics (version 2.0.2) was used as the nonparametric modeling software, using the nonparametric adaptive grid (NPAG) estimation method.20 Final models developed using each software package are included as Supplemental Digital Content 1, (see http://links.lww.com/TDM/A753).

RESULTS

Data Collection and Cleaning

Three levels were flagged as suspected outliers: 1 TDM level was removed after suspicion that the dosing history was missing for several doses before the TDM level, 1 TDM level was removed because it was recorded as sampled during the infusion of vancomycin, and 1 TDM level was removed because it was an unexplainable outlier data point. After cleaning, data from 83 patients were available, providing a total of 272 TDM levels (median, 2 per patient; range, 1–29 per patient). The patient demographic statistics are summarized in Table 1.

TABLE 1. Dataset Statistics of Population Demographics

Demographic	Development Dataset (n = 83)	External Dataset (n = 576)	Unit	
Median/Count	Range	Median/Count	Range	
Weight	134	91.6–218	132	79.6–276	kg	
Height	170	122–190	168	122–198	cm	
BMI	46.3	40–70.3	45.4	40.0–91.8	kg/m2	
Age	56.3	24.2–89.3	59.0	15.6–92.4	years	
Serum creatinine	0.84	0.40–2.51	0.964	0.3–10.2	mg/dL	
Creatinine clearance (based on TBW)	174	30.2–579	140	10.8–672	mL/min	
Creatinine clearance (based on FFM)	84.1	13.0–255	70.0	4.7–265	mL/min	
Sex (M/F)	41/42	—	382/256	—	—	
Number of levels	272		1191			
Number of levels per patient	2	1–29	1	1–16		
Peak/random/trough levels*	31/126/115	82/726/383	Count	
* Peak level was defined as within 2 h after the end of infusion, trough was defined as within 1 h of the next administration, and random as any other level.

Model Development

Below, we present model development highlights for both the parametric and the nonparametric approaches. Table 2 summarizes the parameter estimates for all the software tools. Table 3 provides an overview of the main differences and similarities between the 2 modeling approaches. The VPC is shown in Figure 1, whereas standard goodness-of-fit plots are included in Supplemental Digital Content 1, (see http://links.lww.com/TDM/A753).

TABLE 2. Parameter Estimates for Population PK Models

Parameter	Description	Pmetrics	nlmixr2 (SAEM)	NONMEM (SAEM)	Unit	
Estimate	RSE, %	BSV, %	Range	Estimate	RSE, %	BSV, %	RSEBSV, %	Estimate	RSE, %	BSV, %	RSEBSV, %	
CL	Drug clearance	6.74	4.5	22.2	3.57–10.2	5.09	4.1	24.9	13	5.06	5.3	21.8	11	L/h	
θ3	Effect of CrCl on CL	1.05	10	—	—	0.887	6.1	—	—	0.851	8.1	—	—	—	
Vc	Central volume of distribution	104	5.3	25.2	61.5–161.4	64.9	8.4	17.8	21	64.34	10	18.0	54	L	
θ2	Effect of FFM on Vc and Vp	1.0	FIX	—	—	1.0	FIX	—	—	1.0	FIX	—	—	—	
Q	Intercompartmental clearance	7.5	42	54	2.12–19.2	6.36	18	—	—	6.24	28	—	—	L/h	
Vp	Peripheral volume of distribution	142	34	35.0	47.9–277	66.4	13	80.3	32	59.0	16	79.7	29	L	
RUV	Proportional error	0.1	FIX	—	—	16.8	—	—	—	13.3	—	—		—	
RUVadd	Additive error	0.5	FIX	—	—	—	—	—		—	—	—		—	
Model structure: CLi = CL·(FFM/70)θ1·(CrCL/100)*θ3, Vi = V·(FFM/70)θ2, Qi = Q·(FFM/70)θ1, Vp,i = Vp·(FFM/70)θ2. The RSE for the population parameter estimates for both the parametric and the nonparametric methods was obtained by bootstrapping. The BSV is shown as %CV for parametric methods and %CV and range for nonparametric methods. FIX indicates the parameters that are not estimated but are fixed. The RSEBSV reported for the parametric methods refers to the uncertainty in the BSV (on a SD scale).

TABLE 3. Comparison of Final Models

Comparison	Parametric (nlmixr2)	Nonparametric (Pmetrics)	
Model structure			
 Structural model	2-compartment	2-compartment	
 Covariates	CL ∼ CrCl
Vc ∼ FFM
Vp ∼ FFM	CL ∼ CrCl
Vc ∼ FFM
Vp ∼ FFM	
 Between-subject variability	CL, Vc, Vp	CL, Vc, Q, Vp	
 Residual error	15.6% + 1.2 mg/L	10% + 0.5 mg/L (fixed)
L = 0.1	
	Population	Individual	Population	Individual	
Model fit					
 r2	0.597	0.896	0.531	0.917	
 MPE, %	−6.3	0.16	2.69	0.84	
 nRMSE, %	27.2	13.8	30.7	13.1	

FIGURE 1. Visual predictive checks for the nonparametric model (Pmetrics) and the parametric model (nlmixr2).

Parametric (NLMIXR2/NONMEM)

Results between nlmixr2 and NONMEM were highly similar, and during development, the same modeling decisions were made with both software packages. Therefore, we present only the results for nlmixr2, although the final parameter estimates from both software approaches are listed in Table 2. The FFM was a better predictor of PK parameters than the other weight types when implemented allometrically. However, the inclusion of the allometric exponent on clearance (CL) and intercompartmental clearance (Q) considerably worsened the model fit when fixed and was estimated to be close to zero when estimated; therefore, this exponent was excluded. Estimation of the exponent of Vc/Vp also did not provide a better fit than fixing to 1.0. The most predictive covariate was CrCl, calculated using the Cockcroft–Gault equation, which was a better predictor than serum creatinine (sCr) alone. The GFR estimated by the CKD-EPI and MDRD equations performed worse than the CrCl calculated using the Cockcroft–Gault equation by 20 and 39 OFV points, respectively. Furthermore, using FFM as input to the Cockcroft–Gault equation instead of TBW improved the fit considerably and was also slightly better than using LBW, IBW, or AjBW. Capping with CrCl at 150 or 200 mL/min worsened the fit. BSV could be estimated on the CL, volume of distribution of the central compartment (Vc), and volume of distribution of the peripheral compartment (Vp) but not on Q. The inclusion of interoccasion variability did not improve the fit. Goodness-of-fit plots did not reveal any structural bias, and the VPC in Figure 1 shows that the model adequately described the observed data. A slight underprediction of the median was observed only after a dose of approximately 24 hours.

Nonparametric (Pmetrics)

As in the parametric model, FFM implemented as an allometric power model was a better predictor than TBW. The allometric exponents could not be estimated reliably, but when fixed to 0.75 and 1.0 for Q/CL and Vc/Vp, respectively, led to significant improvement in fit. However, once CrCl was included as a predictor of CL, the inclusion of the allometric exponent on CL and Q did not improve the model and was, therefore, removed, resulting in the same final structure as the parametric models. Using FFM as an input to the Cockcroft–Gault equation improved the fit significantly compared with using TBW and was slightly better than LBW, IBW, or AjBW. Capping with CrCl did not improve the fit. Alternative equations for estimating GFR were not tested with pmetrics because of the significantly worse predictive findings when tested with nlmixr2. The discrete distribution of individual parameters was estimated for all 4 basic PK parameters, even though fixing Q and Vp to a single population value did not result in a worse AIC. The goodness-of-fit plots, especially the normalized prediction distribution errors (NPDEs) plots (see Figure, Supplemental Digital Content 1, http://links.lww.com/TDM/A753), showed a slightly higher structural bias than the parametric model, but the VPC for the nonparametric model was highly similar to that for the parametric model and showed the same minor bias for the median prediction 24 hours after dosing.

Model Fit

Table 3 shows that the correlations between the predictions and observations were highly similar for the final versions of the parametric and nonparametric models. The differences in the bias and precision of individual predictions were also minimal between the 2 approaches.

Parameter Distributions

Figures 2A, B show the distributions of the individual parameters CL and V for the parametric and nonparametric approaches, respectively. Figure 2A shows the raw individual base parameters estimated during model fitting relative to the median value, whereas Figure 2B shows the individual parameters corrected for covariates, that is, the “effective” parameters used when fitting and predicting concentrations. Figure 2A shows that the distributions for the parametric software (nlmixr2) were narrower than the estimated parametric lognormal distribution for the same software. This is fully expected because of parameter-shrinkage (22% for CL, 32% for V). The parametric distributions for CL and V were also somewhat narrower than the nonparametric distributions. The plot of the effective parameters (Fig. 2B) also showed a somewhat larger spread between the 2 approaches, although it was somewhat obscured by the nonparametric estimates having higher values of CL and V overall.

FIGURE 2. Distribution of individual estimates for CL and V for parametric (nlmixr2) and nonparametric (Pmetrics) approaches. A, (Left) Individual parameters corrected for covariates, that is, the “effective” parameters used when fitting and predicting concentrations. B, (Right) Raw individual base parameters estimated during model fitting relative to the median value across all individual estimates.

External Validation of Predictive Performance

In Figure 3, the accuracy, MPE, and nRMSE are shown for the final parametric model in both the dataset on which the model was trained and a separate test dataset of 576 external patients treated at 74 organizations. The accuracy of the developed model improved considerably compared with existing obesity models, both a priori (before the first TDM level) and a posteriori (iteratively updating Bayesian PK parameter estimates after each TDM). The MPE improved for a priori predictions and was similar for a posteriori predictions, although in the opposite direction of the existing models. The nRMSE was similar to that of the existing models, both a priori and a posteriori.

FIGURE 3. Predictive performance in nRMSE, MPE, and accuracy (% within acceptable error) of the parametric model (nlmixr2) for a priori and a posteriori performance in the external dataset. Error bars indicate 95% CIs obtained using bootstrap.

DISCUSSION

In this quality improvement effort, we compared the parametric and nonparametric approaches for developing PK models for MIPD in adult patients with class 3 obesity who were treated with vancomycin. The 2 approaches produced similarly structured models with performances similar to those of typical model diagnostics. The parametric model was found to predict PK with a significantly higher accuracy and lower error than models taken from the literature, both when evaluated on the dataset and when evaluated on an external dataset of patients treated at other institutions. Institutions have their own data nuances, such as specific laboratory assays, dosing and TDM collection protocols, patient population mixes, and other cultural aspects. Therefore, its good performance on the dataset is expected because it is an institution-specific model that was trained on its own local data. However, it was surprising how well it performed in the external validation compared with the other existing models. Although the other patients were treated at other institutions, one reason for this finding could be that the model development and external validation data were collected from the same Bayesian software.

This quality improvement effort provides further support for the concept of continuous learning, in which models are tailored to an institution using real-world data collected at that institution23,32 as this model performs similar to or better than previously published models. Its considerably better performance in accurately predicting vancomycin exposure and PK in patients at other institutions suggests that it can also be generalized. The external validation dataset included many organizations and, therefore, had significantly more patients than the model development dataset. Although having a larger test dataset than the training data is not a typical design, it strengthened the generalizability of the results. Therefore, this quality improvement effort provides a new model option for MIPD of this challenging patient subpopulation for integration into either parametric or nonparametric MIPD software.

In addition, this quality improvement effort is the first to directly compare the parametric and nonparametric approaches for vancomycin MIPD and Bayesian approaches in an adult population. Consistent with the findings of de Velde et al for imipenem,21 we observed only minor differences between the models developed using the parametric (nlmixr2) and nonparametric (Pmetrics) approaches. The final version of the parametric model included BSV in only 3 parameters because including it in the peripheral clearance (Q) did not produce a statistically significantly better fit. By contrast, because nonparametric model development workflows do not conventionally assess the statistical relevance of BSV to basic PK parameters, BSV was incorporated for all 4 basic PK parameters in the base model. Overall, the model goodness-of-fit diagnostics showed a good description of the data and were similar between modeling approaches, although a plot of NPDEs revealed a somewhat larger bias for the Pmetrics model (see Figure, Supplemental Digital Content 1, http://links.lww.com/TDM/A753), indicating underprediction at higher predictions. This bias was also visible in a plot of population prediction versus observations (see Figure, Supplemental Digital Content 1, http://links.lww.com/TDM/A753), and in the VPC (Fig. 1), where peak levels were somewhat underpredicted. No model alterations eliminated this bias. NPDEs are simulation-based diagnostics, and nlmixr2 and Pmetrics use different distributional assumptions for these simulations, which could explain some discrepancies.

For both modeling approaches, a minor bias was observed in the VPC for predictions at 24 hours or more after the dose. However, we hypothesized that this bias was a feature of the diagnostic plot rather than a true bias in the model. Vancomycin levels collected 24 hours after the last dose will be obtained only from patients who are on 24-h or longer dosing intervals, which will mostly be patients with slower clearance of vancomycin. Therefore, unless the simulation algorithm used to generate the data for the VPC incorporates the same interval-switching logic based on patient clearance, the VPC is expected to show such a bias.

Unfortunately, we were unable to identify any subgroups in the nonparametric analysis, which is often noted as a potential advantage of this approach over other parametric approaches. It may be that the data from routine clinical practice did not provide enough information for the nonparametric method to show a benefit and that densely sampled pharmacokinetic data are needed in future analyses to better assess this. The distributional assumptions in the parametric approaches may also be valid for this subpopulation and for this drug in general. A limitation of the analysis was that patient status, level of care, and indications for vancomycin were not collected to further describe the special subpopulations.

A comparison of the distribution of individual parameter estimates showed a somewhat larger spread of parameter estimates for the nonparametric approach, potentially indicating a better ability to describe more extreme patients. However, this can be partially explained by the choice of a lower residual error magnitude for the nonparametric approach, which could result in data overfitting, that is, if the parametric model's residual error magnitude would be fixed to a lower residual error estimate (similar to an approach of “flattened priors”33), a similar larger spread in individual parameter estimates would be expected for the parametric approach. On an average, Pmetrics estimated higher values of CL and V than nlmixr2. Given that the predictive performances of both approaches, as judged by MPE and nRMSE, were very similar, the clinical significance of these distributional differences seemed minimal across the parameter distribution of these patient populations.

We found that FFM was the best body size descriptor for modeling the contribution of CrCL to vancomycin CL and the volume of distribution. FFM has been previously proposed as a good method to scale drug clearance in patients with obesity because fat mass does not contribute to drug clearance.34 We found that the vancomycin CL scaled linearly according to the FFM (incorporated in the numerator of the Cockcroft–Gault equation). A previous meta-analysis reported that an exponent of 2/3 best fits the relationship between CL and FFM35; it is possible that this relationship was not identifiable in our dataset because of its narrower variation in weight and larger median weight.

A limitation of this quality improvement effort is that we did not compare the two approaches in their ability to iteratively forecast future drug exposure in clinical scenarios, that is, in which the individual model was updated after each vancomycin level. Such a “fit-for-purpose” analysis13 was performed for the final parametric model only, both on the modeling and external datasets. In addition, dose predictions of the models and their ability to result in a concentration or AUC would be clinically relevant but were not included in this study. Here, we focused only on the comparison of nonparametric and parametric methods for model development. Future research should investigate the performance of nonparametric and parametric methods of Bayesian estimation in the MIPD context.

The availability of a stable, easy-to-use, and free software for the development of MIPD models is an opportunity for the clinical community, particularly for research groups with limited budgets. A detailed comparison between software tools is beyond the scope of this study, but both nlmixr2 and Pmetrics are accessible from R and were found to be relatively easy to use; nlmixr2 is a fully open-source software and includes various parametric estimation methods. Parameter estimates from nlmixr2 using SAEM corresponded well to those obtained with NONMEM, a commercial program considered a “gold standard” in clinical pharmacology, as has been reported before by its authors.36 Pmetrics is “shareware” and not fully open source; the source code is only made available for the R wrapper package. The final PK model, parameter estimates, and diagnostic plots from the metrics corresponded well with the final parametric results obtained in nlmixr2, which increased confidence in the use of both packages as valid alternatives to commercial solutions. Importantly, the software developers for both packages were quick to answer questions and responded with helpful feedback regarding undocumented aspects of the software.

CONCLUSIONS

The precision and bias of both the parametric and the nonparametric vancomycin PK models constructed using the patient population with class 3 obesity were comparable; thus, no preference over either approach was found in this quality improvement effort. The parametric model demonstrated an equivalent or better ability to predict vancomycin PK than previously published models, demonstrating the potential benefit of creating institution-specific models to improve clinical practice in this setting.

Supplementary Material

SUPPLEMENTARY MATERIAL

ACKNOWLEDGMENTS

The authors thank Matt Fiddler and Michael Neely for their helpful advice and feedback regarding the use of the nlmixr2 and Pmetrics packages. The authors are also grateful to the University of Vermont Medical Center pharmacists for their role in data entry as a part of patient care.

M.-S. A. Hughes, J. H. Hughes, and R. J. Keizer are employees and stockholders of Insight Rx, Inc. J. Endicott, M. Langton, J. W. Ahern declare no conflict of interest.

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal's Web site (www.drug-monitoring.com).
==== Refs
REFERENCES

1. Rybak MJ Le J Lodise TP . Therapeutic monitoring of vancomycin for serious methicillin-resistant Staphylococcus aureus infections: a revised consensus guideline and review by the American Society of Health-System Pharmacists, the Infectious Diseases Society of America, the Pediatric Infectious Diseases Society, and the Society of Infectious Diseases pharmacists. Am J Health Syst Pharm. 2020;77 :835–864.32191793
2. Rybak M Lomaestro B Rotschafer JC . Therapeutic monitoring of vancomycin in adult patients: a consensus review of the American Society of Health-System pharmacists, the Infectious Diseases Society of America, and the Society of Infectious Diseases pharmacists. Am J Health-System Pharm. 2009;66 :82–98.
3. Chen A Gupta A Do DH . Bayesian method application: integrating mathematical modeling into clinical pharmacy through vancomycin therapeutic monitoring. Pharmacol Res Perspect. 2022;10 :e01026.36398492
4. Cerenzio J Truong J . Efficacy and safety of vancomycin Bayesian-estimated area under the curve versus trough-based dosing. Ann Pharmacother. 2023;57 :931–939.36476049
5. Monteiro JF Hahn SR Gonçalves J . Vancomycin therapeutic drug monitoring and population pharmacokinetic models in special patient subpopulations. Pharmacol Res Perspect. 2018;6 :e00420.30156005
6. Kosmisky DE Griffiths CL Templin MA . Evaluation of a new vancomycin dosing protocol in morbidly obese patients. Hosp Pharm. 2015;50 :789–797.26912920
7. Chooi YC Ding C Magkos F . The epidemiology of obesity. Metabolism. 2019;92 :6–10.30253139
8. Grace E . Altered vancomycin pharmacokinetics in obese and morbidly obese patients: what we have learned over the past 30 years. J Antimicrob Chemother. 2012;67 :1305–1310.22382471
9. Carreno JJ Lomaestro B Tietjan J . Pilot study of a Bayesian approach to estimate vancomycin exposure in obese patients with limited pharmacokinetic sampling. Antimicrob Agents Chemother. 2017;61 :e02478-16.28289024
10. Crass RL Dunn R Hong J . Dosing vancomycin in the super obese: less is more. J Antimicrob Chemother. 2018;73 :3081–3086.30203073
11. Smit C Wasmann RE Goulooze SC . Population pharmacokinetics of vancomycin in obesity: finding the optimal dose for (morbidly) obese individuals. Br J Clin Pharmacol. 2020;86 :303–317.31661553
12. Adane ED Herald M Koura F . Pharmacokinetics of vancomycin in extremely obese patients with suspected or confirmed Staphylococcus aureus infections. Pharmacotherapy. 2015;35 :127–139.25644478
13. Keizer RJ Ter Heine R Frymoyer A . Model-informed precision dosing at the bedside: scientific challenges and opportunities. CPT Pharmacometrics Syst Pharmacol. 2018;7 :785–787.30255663
14. Hughes JH Tong DMH Faldasz JD . Evaluation of neonatal and paediatric vancomycin pharmacokinetic models and the impact of maturation and serum creatinine covariates in a large multicentre data set. Clin Pharmacokinet. 2023;62 :67–76.36404388
15. Colin PJ Eleveld DJ Hart A . Do vancomycin pharmacokinetics differ between obese and non-obese patients? Comparison of a general-purpose and four obesity-specific pharmacokinetic models. Ther Drug Monit. 2021;43 :126–130.33278242
16. Goutelle S Woillard JB Buclin T . Parametric and nonparametric methods in population pharmacokinetics: experts' discussion on use, strengths, and limitations. J Clin Pharmacol. 2022;62 :158–170.34713491
17. Jelliffe R Schumitzky A Van Guilder M . Population pharmacokinetics/pharmacodynamics modeling: parametric and nonparametric methods. Ther Drug Monit. 2000;22 :354–365.10850405
18. Guidi M Csajka C Buclin T . Parametric approaches in population pharmacokinetics. J Clin Pharmacol. 2022;62 :125–141.33103774
19. Rainier S Thomas D Tokarz D . Myofibrillogenesis Regulator 1 gene mutations cause paroxysmal dystonic choreoathetosis. Arch Neurol. 2004;61 :1025–1029.15262732
20. Neely MN Van Guilder MG Yamada WM . Accurate detection of outliers and subpopulations with pmetrics, a nonparametric and parametric pharmacometric modeling and simulation package for R. Ther Drug Monit. 2012;34 :467–476.22722776
21. De Velde F De Winter BCM Neely MN . Population pharmacokinetics of imipenem in critically Ill patients: a parametric and nonparametric model Converge on CKD-EPI estimated glomerular filtration rate as an impactful covariate. Clin Pharmacokinet. 2020;59 :885–898.31956969
22. Colin PJ Allegaert K Thomson AH . Vancomycin pharmacokinetics throughout life: results from a pooled population analysis and evaluation of current dosing recommendations. Clin Pharmacokinet. 2019;58 :767–780.30656565
23. Hughes JH Tong DMH Lucas SS . Continuous learning in model‐informed precision dosing: a case study in pediatric dosing of vancomycin. Clin Pharmacol Ther. 2021;109 :233–242.33068298
24. Janmahasatian S Duffull SB Ash S . Quantification of lean bodyweight. Clin Pharmacokinet. 2005;44 :1051–1065.16176118
25. Devine B . Gentamicin therapy. Drug Intell Clin Pharm. 1974;8 :650–655.
26. Winter ME . Basic Pharmacokinetics. Philadelphia, PA: Lippincott Williams and Williams; 2004.
27. Marsot A Boulamery A Bruguerolle B . Vancomycin: a review of population pharmacokinetic analyses. Clin Pharmacokinet. 2012;51 :1–13.22149255
28. Karlsson M Nordgren R . PsN: an open source toolkit for non-linear mixed effects modelling. Version 5.2.6; 2022. Available at: https://uupharmacometrics.github.io/PsN/. Accessed October 1, 2023.
29. Ter Heine R Keizer RJ van Steeg K . Prospective validation of a model-informed precision dosing tool for vancomycin in intensive care patients. Br J Clin Pharmacol. 2020;86 :2497–2506.32415710
30. Keizer RJ Denney B . Vpc: create visual predictive checks. Version 1.2.2.9000; 2021. Available at: https://github.com/ronkeizer/vpc. Accessed October 1, 2023.
31. Fidler M Wilkins JJ Hooijmaijers R . Nonlinear mixed‐effects model development and simulation using nlmixr and related R open‐source packages. CPT: Pharmacometrics Syst Pharmacol. 2019;8 :621–633.31207186
32. Maier C de Wiljes J Hartung N . A continued learning approach for model‐informed precision dosing: updating models in clinical practice. CPT: Pharmacometrics Syst Pharmacol. 2022;11 :185–198.34779144
33. Hughes JH Keizer RJ . A hybrid machine learning/pharmacokinetic approach outperforms maximum a posteriori Bayesian estimation by selectively flattening model priors. CPT: Pharmacometrics Syst Pharmacol. 2021;10 :1150–1160.34270885
34. Sinha J Duffull SB Al-Sallami HS . A review of the methods and associated mathematical models used in the measurement of fat-free mass. Clin Pharmacokinet. 2018;57 :781–795.29330781
35. McLeay SC Morrish GA Kirkpatrick CMJ . The relationship between drug clearance and body size: systematic review and meta-analysis of the literature published from 2000 to 2007. Clin Pharmacokinet. 2012;51 :319–330.22439649
36. Schoemaker R Fidler M Laveille C . Performance of the SAEM and FOCEI algorithms in the open‐source, nonlinear mixed effect modeling tool nlmixr. CPT: Pharmacometrics Syst Pharmacol. 2019;8 :923–930.31654482
