Digital Twin Implementation for Manufacturing of Adjuvants

Abstract

Pharmaceutical manufacturing processes are moving towards automation and real-time process monitoring with the help of process analytical technologies (PATs) and predictive process models representing the real system. In this paper, we present a digital twin developed for an adjuvant manufacturing process involving a microfluidic formation of lipid particles. The twin uses a hybrid model for estimating the current state of the process and predicting system behavior in real time. The twin is used to control the adjuvant particle size, a critical quality attribute, by varying process parameters such as the temperature and inlet flow rates. We describe steps in the design and implementation of the twin, starting from the conception of the mechanistic model, up to the generation of its surrogate model used as state estimator, PATs and the setup of the information technology—Operational technology architecture. We demonstrate the performance of the twin by introducing different disturbances in the process and comparing the effect on the product critical quality attributes with and without the control of the digital twin. Finally, we showcase the digital twin implementation for the process in good manufacturing practice, through an engineering run, which demonstrated the robustness of the process when controlled by the digital twin.

1. Introduction

With the emergence of industrial revolution 4.0, manufacturing processes are moving towards automation and real-time process monitoring where interconnected computers and intelligent machines communicate with each other and make decisions with minimal human involvement [1,2,3]. Digitizing manufacturing processes allows us to have a better understanding of the system, leverage available data to increase productivity, enhance product launch timelines and optimize processes to drive growth [4,5,6]. In this context, one of the most relevant tools and concepts that has arisen in the era of the digitalization of the manufacturing processes is the digital twin.

The digital twin is an in silico replica of the physical process [7,8]. In the digital twin of a manufacturing process, the real plant is connected to the virtual plant and there is a two-way connectivity between them [9,10]. The connection between the real and virtual plant allows the real plant to constantly send process analytical technology (PAT) data and process information to the virtual plant. During the perturbations, the virtual plant runs the underlying models to predict the quality attributes and transfer the control actions to the real plant (Figure 1). The underlying process models in the virtual plant are frequently updated with the knowledge from either the first principles model or data-driven model. The computation time in first principles models is limiting, and we need to build surrogate models which combine data from first principles models to build hybrid models. These hybrid models are solved [8] using new data.

There are various advantages of implementing a digital twin in a manufacturing process. The digital twin enables automation and advanced control, leading to an increase in robustness in manufacturing [11,12]. The digital twin reduces the development costs by optimizing the development cycles, reducing the actual number of experiments needed to develop a process [8,13,14,15,16,17,18,19,20,21]. It also helps to achieve real-time release with the help of the effective and precise control of desired quality attributes [22].

The present work describes the application of a digital twin to the continuous formulation of a particle-based adjuvant. Adjuvants enhance the immune response of vaccines whose active ingredient (or antigen) has an insufficient effect on the innate immune response [23]. The addition of adjuvant particles in vaccines also helps to optimize the amount of antigen needed per dosage and thus provide more doses of the vaccine during high demands. The adjuvant particles considered here are a new-generation adjuvant containing two immunostimulatory molecules. It is formulated by continuously mixing an organic and an aqueous phase in a chamber engraved in a microfluidic chip which efferently brings both phases to a complete mixing stage.

2. Materials and Methods

2.1. Process Description

The adjuvant particle manufacturing process involves the mixing of organic and aqueous solutions in a microfluidic chip (Figure 2). A solvent-based solution (organic phase) and a water-based solution (aqueous phase) are brought together. A precipitation process then begins with the formation of disc-like particles that are suspended in the surrounding liquid. The size of the discs increases as the process evolves. Eventually, the discs reach critical size and close, forming spherical particles. The size of these particles and the concentration of materials in the final product are the critical quality attributes of this process. The temperature and the flow rates, the main process parameters, are measured using temperature sensors and flow rate transmitters. The concentrations are measured using near-infrared (NIR) probes as described in Section 2.2.1.

2.2. Digital Twin Components

Building the digital twin involves six components (Figure 3): PAT tools, process models, control models, experimental validation, building information technology (IT)–operational technology (OT) architecture, and the user interface. We have described the role of each component and key challenges in building the digital twin for adjuvant manufacturing.

2.2.1. PAT Tools

The PAT is one of the key pillars of the digital twin. It can be considered as the eyes of a digital twin as it is used to measure the evolution of the process in real time. The sensors allow us to measure process parameters such as the temperature, pressure, and flow rate while probes such as spectrometers generate spectra that can be linked to the composition of the raw materials and the critical quality attributes.

The first step of the PAT evaluation was the screening of the right technology. NIR and Raman technologies were tested on different concentrations of the raw materials and attributes in order to compare the acquisition time, sensitivity and accuracy of models. In this exercise, NIR and Raman gave similar results on the prediction error and the clustering of the same conditions. From there, a risk assessment was evaluated and the NIR technology was selected due to its lower environmental health and safety (EHS) risk, its ease of use and its high acquisition speed.

An NIR spatially resolved spectroscopy (NIR-SRS) device with three channels was implemented: a reflexion probe (P1) on the first channel was installed in the organic phase to check its composition before starting the process, a flow cell (P2) on the second channel was installed at the exit of the microfluidic chip to check the quality attributes of the product in real time and a third reflexion probe (P3) on the third channel was installed in the harvest vessel for a last check of the final product. From the process monitoring perspective, the flow cell (Figure 4) is the most important attribute, which is a focus of this article. The flow cell measures along two angles. The first one at 90° (diffusion) is the average of the spectra of five optic fibers. The second one at 180° (transmission) is the average of the spectra of three optic fibers. That means, for each timepoint of measurement, two different spectra are acquired in order to observe geometric shapes in addition to the chemical information that this technology offers. The spectra were measured in absorbance with a mix of ethanol and isopropanol as a reference. The integration time was 150 ms and a spectrum from each optic fiber was the average of 30 scans to increase the signal/noise ratio.

The quality attributes measured by the NIR-SRS flow cell were the concentrations of the three lipids initially present in the organic phase and a fourth component initially present in the aqueous phase. The spectrometer was connected during the execution of the design of experiments (DoE) by varying several process parameters to extend the ranges of the quality attributes for model-building perspectives. In-line spectra were collected every five seconds, aligned with other sensors’ data and saved in the SIPAT (Siemens, Munich, Germany) platform. From there, chemometrics models were built offline on each quality attribute by using multivariate data analysis techniques to correlate these attributes to the spectra. The models were built using Matlab (Mathworks, Natick, MA, USA) and PLS Toolbox (Eigenvector, Manson, WA, USA) software and then transferred in SIPAT for their use in real time to estimate the concentrations. To build the models, NIR-SRS spectra were preprocessed using two filters: first, the standard normal variate algorithm was applied to normalize the spectra and then a first derivative was obtained using Savitzky–Golay smoothing and the differentiation algorithm (number of points in filter = 15, the order of the polynomial = 2). A principal component analysis was performed on the samples generated in the design space to discriminate those that were different from the consistent samples. Finally, the partial least square models were built using the spectra and their concentrations for each quality attribute. These models are used to predict the concentrations of each of them in routine and in real time from the new spectra generated in-line.

2.2.2. Process Models

The computational fluid dynamics (CFD) model was developed to study the effect of the temperature and flow rates of the aqueous phase and organic phase on the size of the adjuvant particles formed at the outlet of the microfluidic chip. The model comprised the equations representing the process of formation of the adjuvant particles coupled with mixing in the microfluidic chamber. The model was solved using STAR-CCM+ (Siemens) software. The model results were validated using experimental data (see Section 2.2.4). The CFD simulations are computationally expensive, and it is not practically possible to use them in real time for controlling the process.

CFD model results were used to gather the average particle size data and then these data were combined with the experimental ones to build a state estimator model. The state estimator model is a machine learning (ML) model mapping input process parameters (flow rate of an aqueous phase, flow rate of an organic phase and temperature) to the particle size quality attribute as an output. The state estimator model’s main goal is to cope with the CFD model being impractical for real-time control. To deal with CFD and experimental data distributions being slightly different, we leveraged a two-stage transfer learning approach [24]. About 1000 simulations from mechanistic models combined with around 100 experimental data points were used to train the state estimator model. This in turn would allow us to correct any model mismatch from the CFD model predictions. The predictions from the state estimator model are compared to the specifications to anticipate deviations and send to the control model to calculate corrective actions.

2.2.3. Control Models

Control models were built to suggest the control action based on predictions from the state estimator model and desired/optimal critical process parameters and critical quality attributes. According to the implemented control logic, the state estimator model predicts, at defined time steps, the size of adjuvant particles for the given flow rates and temperature conditions. The control model checks if the adjuvant particle sizes are within the desired/normal range of operation, and in case of any deviations, the model back-calculates the control action, i.e., optimized flow rate set points for aqueous and organic phases for controlling the deviated behavior of the system. To take the control action, it is required that data flow between models, sensors, systems, and cloud services.

Various tools such as SIPAT, Matlab, AzureML, etc., work in unison to generate data that in turn are used to optimize/control the process. It is essential that data from each of these systems are stored, monitored and tracked for posterity.

2.2.4. Experimental Data Generation and Model Validation

Experimental data consist of two data sets, a D-optimal DoE at a small scale and a long run at a large scale. During the DoE run at a small scale, the following process parameters were varied: temperature (three levels tested), total flow rate (five levels tested), aqueous:organic ratio (three ratios tested), concentration of organic phase (three flow rates tested) and concentration of aqueous phase (three flow rates tested). A total of 90 conditions were tested. The transitions between the setpoints were changed dynamically (no complete stop between experiments), then the process was allowed to stabilize for 4 to 6 min, after which samples were taken for offline analysis. The following responses were recorded offline: adjuvant size by dynamic light scattering (DLS), adjuvant polydispersity index by DLS, concentration of aqueous and organic phases. The product was also monitored in-line through NIR.

A second experimental data set consisted of a long run carried out at a large scale. In contrast to the first data set, the second one took place at the typical production flow rate and temperature in a larger scale system, aiming to mimic production circumstances. Several conditions were tested during the run, namely: one long section at steady state (1.9 h), two short sections with a variation in temperature and flow rate (5 min each) and two short sections where the flow was stopped for up to 1 hr and restarted (5 min each). Similarly to the first data set, NIR spectra were collected in-line whereas samples were taken for DLS analysis offline.

A Zetasizer (Malvern Panalytical Ltd., Malvern, UK) was used for the collection of size distribution through DLS. Figure 5 displays a typical size distribution:

2.2.5. IT-OT Architecture

The digital twin is the combination of many different components that, once connected, allow the data to flow from the physical lab (point a in Figure 6), through the various automation and IT systems (point b1 and b2), to the models or digital replica of the process (point c) and in the opposite direction to reach the physical lab with control actions on the process (point d1 and d2).

Lab Floor

• Lab equipment
• Programmable logic controllers (PLCs): ruggedized industrial computer used to perform control actions on the lab process
• I/O sensors: devices connected to the equipment that detect events or changes and send the information to the data historian
• PAT sensors: PAT capable of measuring in real-time raw materials, intermediates, and products, providing insights on how process variables affect chemistry, bioprocess, or particle-based systems

IT and Automation Systems

• Open platform communication (OPC) server: it allows communication to automation controllers, I/O sensors, and field devices
• Data historian: automation software that records process data in a time-series fashion. In our context, it is used to record equipment and I/O sensors data, as well as ML models predictions and control actions
• PAT software: controls PAT sensors, calibrating them and managing alarms, collects raw data measured by PATs, runs/integrates with statistical and/or physical models to calculate a multivariate quantitative or qualitative representation of the process operation or the products quality attributes, interprets statistical results and exchanges data with the OT data streaming platform
• OT data streaming platform: provides near-real-time data transmission, ingestion and processing within the OT network. It serves as a single point of connection for the different data producers (i.e., data historian, PAT software, etc.) and data consumers (i.e., ML models) and transforms data to match a defined logical data model, minimizing the impact of changes in one system to others. As a result, the digital twin becomes agnostic of the underlying technological components and horizontal, simplifying its application to other labs
• Data pipelines: from the lab to the models to the data lake, through the data streaming platform, both in an online fashion for process monitoring and control and in an offline fashion, for user training and process simulation

Digital Equivalent of the Physical Process

• ML models: State estimator model combined with control models establish the behavior of the physical process which can be solved in real-time
• Docker container: where the models are deployed for process monitoring and control

Closed Loop Systems

• OPC server: see point IT and automation systems above
• Control system: acts on the physical process and implements the identified control actions
• Monitoring and control user interface: shows, in near-real time, the process performances and the control actions

On Edge Cloud Platform

• Data lake: where the process data, predictions and control actions are stored and made available to the end users for further analysis, process simulation and end-user training
• Model versioning and retraining [24]
• Simulation user interface: allows the end users to execute the digital twin models offline and to perform in silico experimentation

2.2.6. User Interface

Online user interface corresponds to the interface that is connected to the physical process in the lab through OsiPi. This interface displays real-time values that are captured (for example: size of the adjuvant particles).

3. Results

3.1. Digital Twin Proof of Concept Run

The digital twin was used as a proof of concept where an external perturbation was introduced in the process. The real-time measurements of critical process parameters, i.e., the flow rate of aqueous phase (panels D and H), flow rate of organic phase (panels C and G) and temperature (panels B and F) and critical quality attribute, i.e., predicted size of adjuvant particles (panels A and E) are plotted in Figure 7. To test the system behavior, the temperature was perturbed (~10%), and we captured the changes in process parameters in control-enabled and control-disabled conditions.

The control-enabled condition considered that the digital twin was estimating the state of the system, predicting the size of the adjuvant particles regardless of the adjuvant particle composition, comparing it with the target size in normal operating ranges, calculating new operating process parameters (flow rates of aqueous and organic phases) if required and transmitting the new calculated set points to the system to maintain the adjuvant particle size within target size. The control-disabled condition considered that the digital twin estimated the state of the system and predicted/monitored the size of the adjuvant, but no control action was taken to curb the perturbation.

The temperature perturbation was introduced at 150 s; the state estimator model estimated the new size of the adjuvant particles based on the inputs and compared with the normal range. As the particle size started to deviate from desired values, the digital twin calculated new flow rates for aqueous and organic phases to bring the deviated adjuvant particle size within the acceptable range. The control actions helped to maintain the critical quality attribute within specifications.

The control disabled model showed that since there were no controls in place, during temperature perturbation, the flow rates of aqueous and organic phases were kept constant. This led to an excessive increase/decrease in the size of the adjuvant particles. In the control-disabled case, the important critical quality attribute was not maintained in the desired range. These observations were validated by taking samples during this test and the actual sizes of adjuvant particles were measured offline.

3.2. Digital Twin Engineering Run

The digital twin was tested during an engineering run (small scale) in a good manufacturing practice (GMP) facility, and Figure 8 represents the data. Temperatures of aqueous and organic phases were measured along with organic and aqueous flow ratio. The plots represent the normalized values of temperatures and total flow rate of aqueous and organic phases over the period of 6 min (we normalized parameters for better representation; normalization was performed based on the lowest and highest value of the same parameter) at the beginning of the operation. Therefore, it is demonstrated how the digital twin would have acted to take the process at its defined steady state. The ratio of aqueous and organic phase was maintained constant and the twin predicted the size of the adjuvant particles. The size of the particles was kept at the target. At each time point, the state estimator model calculated the size of the particles based on the temperature and the flow rates of the two phases. The predicted size was compared with the target size and the twin further suggested an action for the total flow rate. The twin proposed an action for the total flow rate since the model was maintaining a constant flow ratio of aqueous and organic phases. The twin was not connected with the control system in this case; hence, it was not directly implementing the control action but suggesting an operator to take one.

At the beginning of the run, there were slight variations in the temperatures, and the pumps took a minute to stabilize the flow rates. As the pumps were acting on the set points of the flow rates for the aqueous and organic phases, the flow ratio increased momentarily (between 0.15 and 0.3 normalized time) and the predicted size increased. The state estimator model predicted an increase in the suggested flow rate to curb this deviation. As the flow rates attained their desired set points, the predicted size decreased. There was a slight variation in the temperatures, which further impacted the predicted size. The system reached a steady state at around 0.4 normalized time, and there were rarely any deviations in the process parameters and critical quality attribute. Hence, the suggested flow rate remained constant throughout the rest of the run.

4. Discussion

We developed and implemented a digital twin for the manufacturing of adjuvant particles. The twin was tested in a lab set up with control-enabled and -disabled conditions. We later implemented the twin in a GMP production environment and performed an engineering run. To our knowledge, the implemented case study represents the first published application showing how to build a real digital twin for a biopharmaceutical process step and its transfer to a GMP environment. The process of development and implementation was helpful to understand the key challenges of developing such a twin in a development and GMP environment.

Experimental data were not widely available due to the limited raw materials. Additionally, the number of process variables made the description of the experimental space especially difficult. The lack of a reliable particle size and content measurement in-line forced the measurements to take place offline. Therefore, we have relied on offline DLS and HPLC measurements to construct our models. We work with the assumption that the process is scalable. Additional experiments carried out during development showed that working at small or large scale does not significantly impact the product attributes (particle size, polydispersity and content).

As demonstrated in the present work, there are considerable advantages for control strategies relying on digital twins and ML. The regulatory framework and guidance related to such process modeling and control schemes are limited, including ICH Q8/Q9/Q10 Q&As Points to Consider and ICH Q13. While the scientific literature around model-based and predictive controls is becoming quite abundant in relation to pharmaceutical applications [25,26], there is no specific reference to artificial intelligence (AI)/ML in pharmaceutical manufacturing regulation and guidance, and while integration into direct process control clearly meets the criteria of a high-impact model, there are unique aspects related to GMP (e.g., model validation, model change management) and regulatory submission (e.g., requisite filing information, post-marketing change reporting) that are currently unaddressed. While some regulators have begun to address aspects of AI/ML, these were only tangentially tied to pharmaceutical manufacturing or were only applicable to specific regions. Several industry positions directly address AI model validation and the maintenance of models in the post-marketing phase; however, it is unclear whether regulators will fully embrace these considerations or whether such approaches will be readily harmonized with existing regional frameworks [27,28]. Regulators are clearly focused on facilitating the development and deployment of advanced pharmaceutical manufacturing technology, especially given the challenges encountered during the COVID-19 pandemic. The recently published PDUFA VII bill in the US directly addresses advanced manufacturing technology and tasks the FDA with the organization of public workshops and the development of guidance [29]. The EMA has recently solicited feedback on the development of the Quality Innovation Group (QIG); advanced manufacturing technology, such as digital twins and AI/ML-based process control models, appear to be clearly within its remit [30]. An ongoing dialogue with regulators will be critical to realizing the potential of this advanced manufacturing technology while assuring a suitable level of regulatory oversight and assurance of quality [31,32].

5. Conclusions

Adjuvants are added to vaccine drug products to increase the efficacy of the vaccines and optimize the amount of antigen needed per dosage. Here, we developed a digital twin for the adjuvant manufacturing process. The digital twin is based on a hybrid model that estimates the current state of the process and predicts any control action needed to maintain product critical quality attributes. The PAT models acted as the eyes of the process and helped us to continuously monitor the process performance. The surrounding IT/OT structure helped us to deliver automated control responses to the feedback loop. We demonstrated that the digital twin is capable of taking control actions in case of any perturbations. We also tested the twin performance without a control mode in a GMP environment where the twin suggested the control actions to maintain critical quality attributes in desired ranges.