
==== Front
Heliyon
Heliyon
Heliyon
2405-8440
Elsevier

S2405-8440(24)13700-0
10.1016/j.heliyon.2024.e37669
e37669
Research Article
Learning complex nonlinear dynamics of a 3D translational parallel manipulator using neural network
Elgammal Abdullah T. abdalla.elgammal@bue.edu.eg
abdalla.elgammal@bhit.bu.edu.eg
ab
Magdy Mahmoud Mahmoud.Mohamed@bue.edu.eg
mahmoud.elsharkawy@bhit.bu.edu.eg
ac
Lashin Manar manar.lashin@bhit.bu.edu.eg
b⁎
a The British University in Egypt, El-Sherouk City, Egypt
b Electrical Eng. Dept., Benha Faculty of Engineering, Benha University, Benha, Egypt
c Mechanical Eng. Dept., Benha Faculty of Engineering, Benha University, Benha, Egypt
⁎ Corresponding author. manar.lashin@bhit.bu.edu.eg
11 9 2024
30 9 2024
11 9 2024
10 18 e376691 4 2024
24 7 2024
8 9 2024
© 2024 The Authors. Published by Elsevier Ltd.
2024

https://creativecommons.org/licenses/by-nc-nd/4.0/ This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
This study investigates modeling the dynamics of a 3D translational parallel manipulator with closed chains using feedforward neural networks (FFNNs). The dataset exceeds 50,000 samples, incorporating experimental data collected from a robot prototype using MATLAB® real-time workshop and the National InstrumentsTM DAQ toolbox, as well as CAD simulation data from MSC ADAMS software. While achieving satisfactory mean squared error (MSE), some predictions did not fully capture the manipulator's dynamics, with small overfitting observed. A Deep Neural Network (DNN) was tested but faced overfitting and high computational costs, despite being trained on a subset of the dataset. This highlighted the limitations of DNNs for modeling such complicated parallel robots with closed chains and parallelograms. FFNNs were preferred for their simplicity and lower overfitting risk. L2 regularization and k-fold validation were applied to improve performance. Transfer learning (TL) was also employed, fine-tuning a new network with weights from pre-trained FFNNs using a smaller, unseen dataset. This approach significantly reduced MSE and completely eliminated overfitting, demonstrating the effectiveness of TL in refining model performance for forward and inverse dynamics. These findings suggest that FFNNs, combined with TL, L2 regularization, and k-fold validation, offer a robust method for accurately modeling complex robotic dynamics, enhancing control and optimization strategies for complicated robotic systems. Training for all networks was conducted within the MATLAB® environment.

Keywords

Parallel robot
Feedforward neural networks (FFNNs)
Transfer learning
Machine learning
Forward/inverse dynamics
L2 regularization
K-fold validation
==== Body
pmc1 INTRODUCTION

Recent advancements in parallel manipulators have yielded significant advantages over their serial counterparts, including superior stiffness, accuracy, and lower inertia. However, these benefits come at the cost of a limited workspace, complex kinematic and dynamic modeling, and high manufacturing expenses [1]. Accurately modeling the dynamics of parallel manipulators remains challenging due to their inherent kinematic complexity and the intricate interactions within their closed-loop structures [2]. Additionally, reducing a manipulator's degrees of freedom can lead to coupled motions of the end-effector, complicating kinematic and dynamic analysis and control techniques [3]. Efficient control strategies rely heavily on accurate dynamic models, but the complexity of most parallel manipulator models renders them unsuitable for use with traditional model-based controllers. This is compounded by the non-conventional structure of the 3D translational parallel manipulator employed in this study, which introduces additional complexity. Traditional methods like the Euler-Lagrange and Newton-Euler approaches offer valuable insights but are computationally demanding and limited in capturing real-time robot behavior [4], [5].

In recent years, artificial neural networks (ANNs) have emerged as a promising alternative for modeling robot dynamics. Their ability to learn complex nonlinearities and adapt to varying operating conditions makes them well-suited for tasks like forward and inverse dynamics modeling of parallel robots [6], [7], [8]. Unlike analytical methods that require in-depth knowledge of a robot's physical characteristics and the effects of forces and torques, ANNs leverage data-driven learning to construct dynamic models, simplifying the modeling process [9], [10].

Traditional methods like the Euler-Lagrange or Newton-Euler approaches rely on precise mathematical formulations and often involve solving complex differential equations, which can be computationally intensive and time-consuming, especially for parallel robots with limited degrees of freedom and intricate kinematic structures. In contrast, neural network models, once trained, offer significant computational efficiency during inference, as the forward pass of a neural network is essentially a series of matrix multiplications, highly optimized on modern hardware. However, the initial training phase for neural networks is resource-intensive. The feasibility of deploying these models in real-time systems is highly favorable due to the rapid inference capabilities of NNs. During task execution, such as trajectory planning, the neural network utilizes pre-trained models, eliminating the need for retraining and thus saving time. This contrasts with traditional dynamic models that require solving complex equations repeatedly, making them time-consuming for applications like model-based controllers. Extensive research has been conducted on using NNs or learning techniques to replace/enhance traditional methods [11], [12], [13], [14]. Notably, FFNN models boast faster execution speeds than numerical counterparts, making them ideal for real-time control scenarios [15].

This work was conducted in three stages. The first stage involved training two separate FFNNs: one for forward dynamics and another for inverse dynamics. The modeling efficiency was then assessed using mean squared error (MSE) [16], [17]. The second stage focused on a custom DNN designed to capture forward dynamics only, due to the computational cost required.

While the FFNNs achieved fair performance in capturing the dynamics of the complex translational manipulator during the first phase, the DNN struggled with the system's inherent nonlinearities and high-dimensional interactions, resulting in a high MSE. To enhance the performance and robustness of the neural network, several key steps were implemented. Initially, L2 regularization was applied to prevent overfitting by penalizing large weights. A 5-fold cross-validation approach was adopted to evaluate the model's robustness and generalization capability. Transfer learning (TL) was then employed to improve modeling efficiency on a new, limited-size dataset by using the FFNNs obtained in the first phase.

Recent studies highlight the effectiveness of NNs in modeling robotic dynamics. Bazzi et al. demonstrated that FFNNs achieve faster execution speeds in identifying robot forward dynamics, making them ideal for real-time control [6]. Yilmaz et al. showed that NNs effectively handle inverse dynamics, accommodating non-ideal dynamic effects that traditional models often fail to capture [8]. Prado et al. applied artificial NNs to solve forward kinematics for wearable parallel robots, managing complex dynamics with semi-rigid links [10]. Unlike DNNs, which face high computational costs and overfitting issues in large-scale applications, these findings are consistent with Salman and Liu's observations of DNN challenges [18].

The outputs of this study highlight the robustness and accuracy needed for industrial applications like trajectory planning and control in parallel manipulators. Similar implementations have been successful in real-time industrial applications utilizing robotic systems [19], [20], [21]. Data-driven models and optimization techniques reduce the computational burden of traditional methods, making our approach suitable for real-time industrial robotic environments. Previous research shows that AI models combined with optimization techniques are effective in various fields, such as optimizing shear capacity in fiber-reinforced concrete beams [22]. One approach used ANNs and genetic algorithms (GA), proving feasible for real-time industrial applications. An ANN with 30 neurons achieved superior performance, balancing computational efficiency and accuracy. Another study demonstrated the effectiveness of ANNs combined with Monte Carlo simulations (MCS) in analyzing and optimizing the reliability of SFRC suspended slabs, significantly improving safety and cost efficiency by considering parameters like material properties and loading as random variables [23]. These studies collectively demonstrate that data-driven models and optimization techniques reduce the computational burden of traditional methods, making our approach suitable for real-time industrial robotic environments.

The remaining sections of the paper are structured as follows: Section 2 provides a detailed description of the 3D translational manipulator, outlining its design and operational features. Section 3 explains the manipulator's dynamic modeling using the traditional Euler-Lagrange method and the controller used to generate the required experimental dataset. Section 4 elaborates on the experimental setup and methodology for data acquisition, including details on the CAD model dataset used. Section 5 discusses the utilization of FFNNs for the dynamic representation of the manipulator, covering network architecture, training process intricacies, and applications for both forward and inverse dynamics modeling, as well as the use of transfer learning and other refinement strategies to enhance FFNN performance. Section 6 presents the results and discussion, detailing training outcomes and assessment metrics, and comparing FFNN performance with a custom DNN trained for forward dynamics using the same dataset. Section 7 summarizes key findings and concludes the research.

2 Kinematic analysis of the 3D translational manipulator

The CAD model of the 3D translational manipulator is shown in Fig. 1. This manipulator comprises a pantograph mechanism featuring two types of parallelograms, as depicted in Fig. 2. One type fixes the end-effector orientation within the pantograph plane, while the other ensures orthogonal orientation. The end-effector achieves a consistent 3D orientation. The mechanism consists of two parallelograms with horizontal axis joints and four with vertical axis joints, enhancing torsional stiffness. Actuators are positioned at the base, enabling low-inertia links with high speed, stiffness, and accuracy.Figure 1 3D Translational Manipulator CAD Model.

Figure 1

Figure 2 3D Translational Manipulator Schematic Diagrams.

Figure 2

A fixed global reference system o-xyz is used, as shown in Fig. 2. The end-effector's positional coordinates are denoted by the generalized coordinates E=(qx,qy,qz). The analysis begins with forward kinematics, followed by inverse kinematics. Notably, including parallelograms mitigates the mathematical complexities associated with forward kinematics in parallel manipulators [24]. This reduction in complexity stands as a significant advantage of this 3D translational manipulator over conventional parallel manipulators, which are known for their complex forward kinematics. The position of the moving platform is described in terms of the actuated variables xa, ya, and za. As depicted in Fig. 2, these variables correspond to xa=OB, ya=A′A, and za=OA′. Two vector-loop closure equations are formulated as follows:(1) OE→=OA→+AD→+DE→,

(2) OE→=OB→+BF→+FE→.

Here, O represents the origin of the fixed coordinate system O-xyz, where the axes x, y, and z align with the actuation axes of the three translational actuators. The position vector of the end-effector, denoted as E=[qx,qy,qz]T, and the vector of linear actuated joint variables, represented by p=[xa,ya,za]T, are established accordingly. Given that the end-effector of the proposed manipulator solely undergoes translational motions, the rotation matrix is identified. Furthermore, let AC‾=BC‾=DF‾=a and CD‾=BF‾=FE‾=b denote the design constraints. Utilizing loop equation (1), three scalar equations are derived:(3) qx=(cos⁡θ1−cos⁡θ2)(a+b)cos⁡ϕ,

(4) qy−ya=(sin⁡θ1−sin⁡θ2)(a+b),

(5) qz−za=−(cos⁡θ1−cos⁡θ2)(a+b)sin⁡ϕ.

Likewise, loop equation (2) yields three corresponding scalar equations:(6) qx−xa=(cos⁡θ1−cos⁡θ2)b,cos⁡ϕ,

(7) qy=(sin⁡θ1−sin⁡θ2)b,

(8) qz=−(cos⁡θ1−cos⁡θ2)b,sin⁡ϕ.

The division of equation (3) by equation (6) yields the relationship between the input actuator displacement xa and the resulting output displacement qx along the x-axis:(9) qx=Mxxa.

Here, Mx=1+ba represents the magnification factor associated with the input variable xa. Similarly, equations (4) and (7), and (5) and (8) establish the connections between ya and qy, and between za and qz, respectively:(10) qy=Myya,qz=Mzza.

In these expressions, My=Mz=−ba symbolize the magnification factors corresponding to the input variables ya and za. Equations (9) and (10) illustrate the linear relationship between the input and output displacements and the independent translational motions. The angles ϕ, θ1, and θ2 are determined by the following equations:(11) ϕ=atan2(za,xa)

while,(12) θ1,2=2atan2(∓2aya±4a2ya2−(xa2cos⁡ϕ2+ya2)2+4a2xa2cos⁡ϕ2,xa2cos⁡ϕ2+ya2±2axacos⁡ϕ).

Fig. 3 demonstrates the independent motions of the 3D pantograph manipulator along the X-, Y-, and Z-directions in response to the movements of the linear actuators influencing the three sliders of the manipulator.Figure 3 The movement of the 3D translational manipulator along the X, Y, and Z axes.

Figure 3

3 Dynamic modeling and control system implementation

3.1 Dynamic modeling of the manipulator

The dynamic model and model-based controller employed in this study are derived from the work of Lashin (2018) [9], included specifically due to the relevance of their experimental setup in generating the dataset utilized herein. Although a comprehensive derivation of the dynamic model using conventional methodologies such as the Euler-Lagrange approach is beyond the scope of this paper, a concise overview will be provided. The derived constraint functions are:(13) Γ1=(a+ba−1)xa−bcos⁡ϕ(cos⁡θ1−cos⁡θ2),

(14) Γ2=−(ba+1)ya−(a+b)(sin⁡θ1−sin⁡θ2),

(15) Γ3=−(ba+1)za+(a+b)(cos⁡θ1−cos⁡θ2)sin⁡ϕ.

These constraints, along with their derivatives, are incorporated into the equations of motion using Lagrange multipliers λi. The resulting Euler-Lagrange equations of the first type are:(16) ddt(∂L∂q`˙j)−∂L∂q`j=Q`j+∑i=13λi∂Γi∂q`j,j=1 to 6.

For inverse dynamics, Lagrange multipliers λi are computed from the first three equations in (16):(17) ∑i=13λi∂Γi∂q`j=ddt(∂L∂q`˙j)−∂L∂q`j,j=1 to 3.

Actuator forces fx, fy, and fz are derived from the last three equations in (16):(18) fx=ddt(∂L∂x˙a)−∂L∂xa−∑i=13λi∂Γi∂xa,

(19) fy=ddt(∂L∂y˙a)−∂L∂ya−∑i=13λi∂Γi∂ya,

(20) fz=ddt(∂L∂z˙a)−∂L∂za−∑i=13λi∂Γi∂za.

The complete dynamic equations are given by:(21) M(q)q˙+C(q,q˙)q˙+g(q)=Q,

where M(q) is the inertia matrix, C(q,q˙) is the Coriolis and centrifugal matrix, and g(q) is the gravity vector. Q and q are elements of R3. The actuator forces in the x- and y-directions cause coupled motions, while the z-direction actuator force results in independent motion. Explicitly deriving the C-matrix is complex due to the intricate kinematic terms. The dynamic model requires significant memory and computation time.

3.2 Controller implementation

The controller applied consists of a Feed-Forward controller for the nominal plant and an inverse PID (IPID) controller, utilizing the mathematical dynamic model. The design procedures are detailed in [9]. The closed-loop dynamics of the 3D translational manipulator are described by:(22) τc+τa=MR(qR)q¨R+CR(qR,q˙R)q˙R+gR(qR)+d(t)

where MR, CR, and gR represent the inertia matrix, Coriolis and centrifugal matrix, and gravitational vector for the real plant, and d(t) denotes external disturbances. The controller torque τc is derived from the nominal plant's dynamic model with desired configurations qD, q˙D, and q¨D as inputs:(23) τc=MN(qD)q¨D+CN(qD,q˙D)q˙D+gN(qD)

where MN, CN, and gN are the inertia matrix, Coriolis and centrifugal matrix, and gravitational vector for the nominal plant. The torque τa is given by:(24) τa=M˜(q¨R−q¨D)+C˜(q˙R−q˙D)+g˜+d(t)

with M˜=MR−MN, C˜=CR−CN, and g˜=gR−gN. Extended disturbances, including parameter uncertainties, nonlinearities, and external disturbances d(t), are expressed as:(25) ω(t,e˙,e,∫e)=M˜q¨D+MR(KPe˙+KIe)+C˜q˙D+CR(KPe+KI∫edt)+g˜+d(t)

where KP and KI are diagonal constant matrices, and the tracking error is e=qD−qR. The extended disturbance ω and the configuration derivatives q˙D and q˙R are assumed to be bounded. Stability analysis and criteria for selecting gain matrices are detailed in [9].

4 Experimental setup and dataset generation

4.1 Experimental setup

The experimental setup features a 3D translational manipulator prototype with three linear guides to facilitate movement in the x-, y-, and z-directions. The prototype is constructed using aluminum links, with an I-section for the pantograph mechanism and rectangular cross-sections for the guiding mechanisms. It is equipped with actuators selected based on the maximum forces (Fx=250 N, Fy=130 N, Fz=5 N) required to support a maximum payload of five kilograms, determined through MSC ADAMS software. The manipulator's actuators generate a trajectory comprising three sinusoidal waveforms with frequencies up to 5 rad/s. The manipulator follows a predetermined 3D trajectory guided by the controller [9]. The experimental trajectories consist of sinusoidal waveforms with amplitudes (for actuators) of 0.0056 m in the x-direction, 0.0112 m in the y-direction, and 0.0112 m in the z-direction, all at a frequency of 1 rad/sec. Experiments span 50 seconds and include a variety of trajectories that have been tried with different frequencies (e.g., 2, 3, 5 rad/sec) and actuator amplitudes, ranging from very small displacements to near workspace boundaries. Data is collected and stored via MATLAB Real-Time Workshop, utilizing six 32-bit counters/timers (three for PWM of the linear actuators and three for the encoders). The setup includes Maxon motors with gearheads and encoders, SN75175 chips from Texas InstrumentsTM as differential line receivers, and Pololu Dual VNH5019 motor drivers.

Data are collected post-trajectory execution and stored in an Excel file comprising actuator, forces, displacements and velocities along the x-, y-, and z-axes. These measurements, initially related to the actuators, are adjusted using magnification factors to correspond to the end-effector, assuming a rigid robot body. The collected data are subsequently combined, shuffled, and normalized for training. The training time for neural networks depends on the network structure and machine specifications. For instance, simple FFNNs trained on over 50,000 samples took less than 15 minutes on a machine equipped with an 11th Gen Intel Core i7-1165G7, 16 GB RAM, and a 64-bit OS. However, training a DNN took up to 67 hours (100 epoch) without K-fold validation and approximately 67hour mulitolied by 5 with K-fold validation (k=5). To tackle the issue of high computational times, we developed a MATLAB script for efficient downsampling while ensuring data integrity. This script handles tasks such as loading the dataset, separating predictors and targets, determining the downsampling factor, and preserving data distribution. The 3D translational parallel robot's dynamic model, derived using the Euler-Lagrange method, was implemented for real-time experiments. The initialization process took 50 seconds, and the motion was limited to 1 rad/sec due to the complex dynamic calculations involved. Attempting higher frequencies (5 and 7 rad/sec) led to undesirable motion and session shutdowns. Comparison of the dynamic model against a CAD model took over 10 minutes to substitute one point (x, y, z), which motivated us to explore the development of an AI-based dynamic model to eliminate the need for complex calculations once trained. Even though the nominal plant model significantly reduced computation time, it required improvements before being suitable for trajectory planning tasks. See Fig. 5.

4.2 Dataset generation

The neural network was trained using datasets from two different sources. The first source was the data collected from the experimental setup of the 3D translational manipulator, as shown in Fig. 4. This dataset includes the forces experienced by the actuators (Fx, Fy, and Fz) and the corresponding end-effector positions, velocities, and accelerations (x, y, z, x˙, y˙, z˙, ẍ, ÿ, z¨).Figure 4 Prototype of the 3D Translational Manipulator.

Figure 4

Figure 5 3D Trajectory of the Robot End-Effector.

Figure 5

The second dataset originated from the CAD model, exported from MSC ADAMS software without any controller application, enabling random 3D motion across the entire workspace. This phase necessitated careful force magnitude and frequency selection to prevent singularity or aggressive undesirable motions. The forces applied included fixed components with sinusoidal elements of different frequencies. This configuration facilitated movement across the entire workspace while minimizing the risk of encountering boundary singularities. Additionally, linear actuators adhered to displacement constraints. The systematic collection and storage of end-effector positions, velocities, and accelerations in the x, y, and z directions were compiled to form the training dataset. Visualization of the CAD dataset, depicted in Figure 6, Figure 7, Figure 8, Figure 9, illustrated the application of the three forces to stimulate the robot's movement within the entire workspace.Figure 6 The actuator forces [N] Fx,   Fy and Fz.

Figure 6

Figure 7 The endeffector position [m] along the X, Y, and Z axes.

Figure 7

Figure 8 The endeffector velocity [m/sec] along the X, Y, and Z axes.

Figure 8

Figure 9 The endeffector acceleration [m/sec2] along the X, Y, and Z axes.

Figure 9

Combining both datasets offers the best prospects for training a versatile and resilient model. The dataset collected from the experimental model captures the dynamics and responses of the robot to control inputs, enabling the neural network to learn the relationship between controller inputs and resulting robot behavior. Conversely, the dataset collected without controller application captures the inherent dynamics and physics of the robot system, fostering a more general understanding of its behavior across varied situations. Prior to training, dataset preparation, including augmentation and scaling, is essential. Additionally, shuffling the dataset before training ensures that the training process proceeds smoothly, preventing the model from learning biased patterns in the data.

5 Artificial neural network for dynamic representation

Robot dynamics is a fundamental aspect of robotics that examines the relationship between forces and a robot's motion. Within this field, forward dynamics and inverse dynamics are critical areas, each serving distinct functions. Forward dynamics, also known as direct dynamics, predicts a robot's motion given known forces and torques on its joints by solving equations of motion. This involves using differential equations to detail how applied forces, mass, inertia, and accelerations interact over time to determine position and velocity. Applications of forward dynamics include simulation, control algorithm development, and performance analysis.

Inverse dynamics calculates the forces and torques necessary at each joint to achieve a desired motion trajectory. This process involves solving for joint torques based on desired positions, velocities, and accelerations. Inverse dynamics is crucial for trajectory planning, human-robot interaction, and robot learning through reinforcement learning. The first step of this research involves using a FFNN with two hidden layers, each configured with specific parameters. This network is trained on data pertaining to both forward and inverse dynamics, after which it undergoes testing. Subsequently, the same dataset is utilized to train a deep neural network (DNN), which focuses primarily on forward dynamics, in the second step. This emphasis on forward dynamics is due to its significance in control and trajectory planning applications, which are our primary concerns. The performance of all models is then evaluated aims to determine the most effective neural network architecture for modeling robot dynamics. And finally, refinements are applied in the third step to the FFFNN models. See Fig. 10.Figure 10 Feedforward neural network for (a) Forward dynamics, and (b) Inverse dynamics.

Figure 10

5.1 Forward/inverse dynamics using FeedForward neural network

Feedforward Neural Networks effectively address the complexities of analytical solutions by leveraging patterns from experimental and simulation data. Once trained, these neural network models execute significantly faster than numerical methods while maintaining accuracy across the robot's operational range [6], [25]. This makes them ideal for real-time control applications in various payload and motion scenarios. Additionally, neural models adeptly accommodate non-ideal dynamic effects, a challenge for traditional models based on rigid body and smooth joint assumptions. However, traditional techniques offer deeper insights into physical principles and trained neural networks function as efficient “black-box” dynamic mappers optimized for control tasks. This approach demonstrates how machine learning enhances conventional robot dynamics modeling by providing rapid, data-driven approximations without extensive prior system knowledge. The inputs to the FFNN network consist of the three linear actuator forces (Fx, Fy, and Fz). At the same time, outputs encompass the end-effector pose (x, y, z) and its derivatives (x˙, y˙, z˙, ẍ, ÿ, z¨) for the forward dynamics model. In the case of the inverse dynamics model, end-effector pose and derivatives serve as inputs, with the required actuator forces as outputs. The network architecture, depicted in Fig. 11, consists of an input layer, two hidden layers, and an output layer. Each hidden layer is configured with 128 neurons for forward dynamics and inverse dynamics, utilizing hyperbolic tangent sigmoid (tanh) transfer functions, defined by the equation:(26) f(x)=21+e−2x−1

The output layer is configured with nine nodes, the network trained for 100 epochs using backpropagation to minimize prediction errors between the model and training data obtained from dynamic simulations. Fig. 12 presents a graphical representation of the neural network and the system block diagram for forward/inverse dynamics.Figure 11 Artificial Neural Network (ANN) Structure for (a) Forward dynamics and (b) Inverse dynamics.

Figure 11

Figure 12 The block diagram of the feedforward neural networks for (a) Forward dynamics and (b) Inverse dynamics.

Figure 12

5.2 Transfer learning approach

Transfer learning, a fundamental concept in machine learning, serves as a mechanism to enhance the performance of predictive models for a target task by utilizing knowledge obtained from related source tasks or domains. A domain D=(χ,P(X)) includes a feature space χ and an associated edge probability distribution P(X), where X={x1,…,xn}∈χ. Correspondingly, a task T=(y,f(x)) involves a label space y and a target prediction function f(x), which can be seen as a conditional probability function P(y|x). With this foundational understanding, transfer learning can be carefully explained:

Definition 1.[26](Transfer Learning). Given a learning task,Ttbased onDt, and we can get the help fromDsfor the learning taskTs, Transfer learning aims to improve the performance of predictive functionfT(⋅)for learning taskTtby discover and transfer latent knowledge fromDsandTs, whereDs≠Dtand/orTs≠Tt. In addition, in the most case, the size ofDsis much larger than the size ofDt,Ns>>Nt. In this study, transfer learning is employed to utilize pre-existing knowledge obtained from a larger dataset, aiming to enhance the performance of a neural network model. The process starts with preparing a new, limited-size dataset, comprising normalized forces and end-effector data. Subsequently, two new neural networks, for both forward and inverse dynamics, are constructed with a designated structure featuring a hidden layer size identical to the previous FFNNs. To ensure compatibility with the pre-trained network, the transfer functions and output size of net_new are aligned with those of the existing network. The training parameters are carefully configured, specifying 100 epochs for training. It is of utmost importance that the new network's weights are initialized with the pre-trained weights from the original network, facilitating knowledge transfer for both forward and inverse dynamics. Finally, the new FFNNs are trained using the updated dataset, thereby integrating insights from the pre-existing data into the model's learning process, ultimately enhancing its predictive capabilities.

6 Model refinement techniques

Several refinement procedures were implemented to enhance the performance and robustness of the designed neural network. First, L2 Regularization was applied. This technique adds a penalty to the loss function based on the squared values of the weights. By discouraging large weights, L2 Regularization helps prevent overfitting, ensuring the model remains more straightforward and less sensitive to small variations in the training data [27].

During training, issues were encountered with the Levenberg-Marquardt (LM) algorithm. The LM algorithm adjusts its parameter μ to control the step size of weight updates. When μ reaches its maximum value, it indicates difficulty reducing the error, often leading to early termination [28]. This can be due to complex or sharply curved loss landscapes or unsuitable initial weights. To address this, the resilient backpropagation (Rprop) algorithm was employed. Unlike LM, Rprop updates weights based on the gradient direction rather than its magnitude, providing more stable and robust training. Additionally, Bayesian regularization (trainbr) was utilized, incorporating regularization terms into the error function to stabilize the training process. These techniques mitigate the convergence issues of the LM algorithm by ensuring consistent training progress and robustness to initial conditions.

A 5-fold cross-validation approach was also adopted to evaluate the neural network model's robustness and generalization capability. This method splits the dataset into five subsets, training the model on four subsets and validating it on the remaining one, iterating this process five times. This approach helps ensure that the model generalizes well to unseen data and is not overly dependent on specific training subsets. As demonstrated in the results discussion section, these refinement procedures contribute to a more stable, robust, and generalizable neural network model.

7 Results discussion

The FFNN utilized for forward dynamics training was trained for 100 epochs. Model performance was evaluated using mean squared error (MSE), detailed in Table 1. Fig. 13 shows the training, validation, and testing errors over the epochs, and Fig. 15 illustrates the errors in end-effector positions x, y, and z (forward dynamics network). For clarity, a subset of 200 samples is presented. The dataset was randomly partitioned with 70% for training. The results show satisfactory performance without significant overfitting, despite occasional spikes in gradient values, likely due to high learning rates or data anomalies from robot motion. For the inverse dynamics FFNN model, the inputs are the end-effector pose and its derivatives, while the outputs are the actuator forces (Fx, Fy, Fz). Fig. 14 depicts the training, validation, and testing errors over the epochs, while Fig. 16 shows the error in actuator force predictions. Although the results are acceptable for inverse dynamics modeling, further refinement of the FFNN structure is recommended to improve performance. See Table 2.Table 1 MSE for Robot Forward Dynamics Prediction (1st network).

Table 1Error Metric	MSE [m]	
x	7.2964e-05	
y	3.5939e-05	
z	0.0040484	
x˙	0.0158	
y˙	0.00012406	
z˙	0.0073386	
x¨	0.022667	
y¨	0.012496	
z¨	0.061738	

Figure 13 The 1st NN (Forward dynamics) error curve vs epochs (training, testing and validation errors).

Figure 13

Figure 14 The 2nd NN (Inverse dynamics) error curve vs epochs (training, testing and validation errors).

Figure 14

Figure 15 Position error [m] along the x−, y−, and z− axes for the forward dynamics FFNN model without L2 regularization and k-fold cross-validation.

Figure 15

Figure 16 Forces error [N] along the x−, y−, and z− axes for the inverse dynamics FFNN model without L2 regularization and k-fold cross-validation.

Figure 16

Table 2 MSE for Robot Forward Dynamics Prediction (2nd network).

Table 2Error Metric	MSE [N]	
fx	1.4583	
fy	0.78644	
fz	0.74009	

Another dataset validates the resultant FFNNs when obtaining forward dynamics generated by the trajectory of Fig. 17. This dataset represents another 3D trajectory of the end-effector of the translational parallel manipulator. This dataset is not included in the primary dataset used in training the neural networks and is entirely unseen; the performance of the trained FFNN is satisfied as illustrated in Table 3 that shows the MSE for the end-effector pose (x, y, z) and its derivatives (x˙, y˙, z˙, ẍ, ÿ, z¨). As our primary focus is on forward dynamics due to its practical importance in applications such as trajectory planning, collision avoidance, and obstacle navigation. To enhance forward dynamics modeling, a deep neural network (DNN) with four hidden layers was employed, as detailed in Table 4 and shown in Fig. 18. Despite the potential of DNNs to model complexity, they face challenges such as overfitting when applied to 3D translational parallel manipulators with closed chains as shown in Fig. 19. FFNNs, while simpler, showed promising results in capturing the manipulator's dynamics with the provided dataset.Figure 17 3D Trajectory of the Robot End-Effector.

Figure 17

Table 3 MSE for Robot Forward Dynamics Using 2nd 3D trajectory (Fig. 17).

Table 3Error Metric	MSE [m]	
x	8.192e-05	
y	2.8837e-05	
z	0.0042678	
x˙	0.006411	
y˙	0.0013015	
z˙	0.0070164	
x¨	0.017463	
y¨	0.01192	
z¨	0.052718	

Table 4 Specifications of the Deep Neural Network.

Table 4Parameter	Value	
Hidden Layer Sizes (neurons)	[128, 64, 64, 32]	
Transfer Function (Hidden Layers)	tansig	
Transfer Function (Output Layer)	tansig	
Number of Epochs	100	
Learning Rate	0.001	
Momentum	0.9	
Training-Validation Split Ratio	0.7	

Figure 18 The structure of the DNN for Forward dynamics.

Figure 18

Figure 19 The DNN error curve vs epochs (training, testing and validation errors).

Figure 19

Upon implementing refinement techniques including regularization, k-fold cross-validation, and changing the LM algorithm, the minor overfitting observed in the FFNN for forward and inverse dynamics has been eliminated, with a notable reduction in MSE. Figure 20, Figure 21 demonstrate how regularization and TL initially reduce the MSE. As training progresses, the MSE stabilizes, indicating that the model has converged and can effectively generalize to new data. This stabilization shows that overfitting has been successfully addressed, resulting in an accurate and reliable model for unseen data. Additionally, Figure 22, Figure 23 display the x, y, z motions in the forward dynamics model and the Fx, Fy, Fz motions in the inverse dynamics model, respectively. Figure 24, Figure 25 compare the real and predicted actuator velocities (x˙, y˙, z˙) and accelerations (x¨, ÿ, z¨). The predicted velocities and accelerations closely follow the real values. Minor discrepancies in certain regions highlight the inherent noise in the data and/or the complexity of capturing rapid changes, suggesting potential areas for further enhancement. Overall, the improved FFNNs with TL model accurately captures the dynamic behavior of this parallel robot.Figure 20 The 1st NN (Forward dynamics) error curve vs epochs (Improved network).

Figure 20

Figure 21 The 2nd NN (Inverse dynamics) error curve vs epochs (Improved network).

Figure 21

Figure 22 The 1st NN (Forward dynamics) x, y, z motion after model improvements.

Figure 22

Figure 23 The 2nd NN (Inverse dynamics) Fx, Fy, Fz forces after model improvements.

Figure 23

Figure 24 Comparison of real vs. predicted actuator velocities (xd,yd,zd).

Figure 24

Figure 25 Comparison of real vs. predicted actuator accelerations (xdd,ydd,zdd).

Figure 25

On the other hand, when regularization is applied to a structured DNN network, there are no improvements noticed in terms of MSE or in reducing the risk of overfitting. This suggests that the model has difficulty learning effectively from the data. It emphasizes that the designed DNN is not well-suited for the complex manipulation dynamics of closed-chain systems- at our case, as it fails to adequately capture their nonlinear dynamics. The approach presented in this work demonstrates the effective application of transfer learning to improve the forward and inverse dynamics models using neural networks. The results, as shown in Table 5, highlight a significant reduction in the Mean Squared Error (MSE) after applying transfer learning. This reduction in MSE indicates enhanced model accuracy and robustness, showcasing the strength and efficacy of transfer learning in refining neural network performance for dynamic modeling tasks.Table 5 MSE Before and After TL for Forward and Inverse Dynamics using FFNN.

Table 5Dynamics	Stage	Axis/Force	Average MSE	
Forward	Before TL	x	0.0098824	
y	0.012969	
z	0.13234	
After TL	x	0.0069708	
y	0.0050968	
z	0.0046209	
	
Inverse	Before TL	Fx	0.064861	
Fy	0.066918	
Fz	0.06603	
After TL	Fx	0.0027687	
Fy	0.0021503	
Fz	0.0021621	

8 Conclusions

This study explored the use of neural networks to model the forward and inverse dynamics of a 3D translational parallel manipulator with closed chains. We focused on Feedforward Neural Networks (FFNNs) for their simplicity, reduced overfitting risk, and interpretability. FFNNs, with two hidden layers of 128 neurons each for the forward and inverse dynamics, effectively captured the dynamics, achieving satisfactory performance with minimal mean squared error (MSE). To improve our model's accuracy, we used transfer learning (TL) by fine-tuning a new network with weights from pre-trained FFNNs on a smaller, unseen dataset. This combined approach, along with L2 regularization and k-fold validation with k=5, significantly reduced the MSE, completely eliminating overfitting and demonstrating the effectiveness of transfer learning in refining model performance. These results indicate promising advancements in control and optimization strategies for robotic systems, leading to more accurate and responsive manipulator behavior in real-time applications. A Deep Neural Network (DNN) with four hidden layers was also tested, but it faced challenges with overfitting and computational demands. These issues underscore the practical limitations of the DNNs specifically for our problem, which involves a highly nonlinear robot with closed chains. The presented approach offers a reliable and accurate plan. Additionally, the method of obtaining the dataset for training is sufficient to derive accurate dynamic models for complex and new manipulators, especially those with cross links and closed chains such as our 3D translational parallel manipulator, which have few published studies. This lack of studies makes it challenging to perform trajectory planning or integrate advanced control systems that rely on accurate models.

9 Declaration

During the preparation of this work the authors used [Grammarly.AI TOOL / SERVICE] in order to [enhance language and readability]. After using this tool/service, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication.

CRediT authorship contribution statement

Abdullah T. Elgammal: Writing – review & editing, Writing – original draft, Software, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. Mahmoud Magdy: Writing – review & editing, Writing – original draft, Investigation, Conceptualization. Manar Lashin: Writing – review & editing, Writing – original draft, Validation, Methodology, Investigation, Formal analysis, Data curation, Conceptualization.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A Supplementary material

The following is the Supplementary material related to this article.MMC

The experimental dataset used for training the Feedforward Neural Networks (FFNNs).

MMC

Appendix A Supplementary material related to this article can be found online at https://doi.org/10.1016/j.heliyon.2024.e37669.
==== Refs
References

1 Noshadi A. Mailah M. Zolfagharian A. Appl. Math. Model. 36 2012 2370 2383
2 Gosselin C. Angeles J. IEEE Trans. Robot. Autom. 6 1990 281 290
3 Kanaan D. Wenger P. Chablat D. Mech. Mach. Theory 44 2009 487 498
4 Sciavicco L. Siciliano B. Modelling and Control of Robot Manipulators 2012 Springer Science & Business Media
5 Featherstone R. Rigid Body Dynamics Algorithms 2014 Springer
6 Bazzi D. Messeri C. Zanchettin A.M. Rocco P. Identification of Robot Forward Dynamics via Neural Network 2020 4th International Conference on Automation, Control and Robots (ICACR) 2020 IEEE 13 21
7 Chassiakos A. Kosmatopoulos E. Christodoulou M. Neural Netw. Robot. 1993 165 176
8 Yilmaz N. Wu J.Y. Kazanzides P. Tumerdem U. Neural network based inverse dynamics identification and external force estimation on the da vinci Research Kit 2020 IEEE International Conference on Robotics and Automation (ICRA) 2020 IEEE 1387 1393
9 Lashin M. Fanni M. Mohamed A.M. Miyashita T. Int. J. Control. Autom. Syst. 16 2018 39 54
10 Prado A. Zhang H. Agrawal S.K. Artificial neural networks to solve forward kinematics of a wearable parallel robot with semi-rigid links 2021 IEEE International Conference on Robotics and Automation (ICRA) 2021 IEEE 14524 14530
11 Zhu Q. Zhang Z. IEEE Access 7 2019 128758 128766
12 Yang S.X. Meng M. Neural Netw. 13 2000 143 148 10935758
13 Jorge D. Pizzuto G. Mistry M. Efficient learning of inverse dynamics models for adaptive computed torque control 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2022 IEEE 11203 11208
14 Dalla Libera A. Giacomuzzo G. Carli R. Nikovski D. Romeres D. IFAC-PapersOnLine 2023 519 524
15 Abdulridha H.M. Hassoun Z.A. J. Dyn. Syst. Meas. Control 140 2018 061002
16 Liang B. Li T. Chen Z. Wang Y. Liao Y. Robot arm dynamics control based on deep learning and physical Simulation 2018 37th Chinese Control Conference (CCC) (IEEE) 2018 2921 2925
17 Zhang B. Highlights in Sci. Eng. Technol. 81 2024 109 120
18 Salman S. Liu X. arXiv preprint arXiv:1901.06566 2019
19 Soori M. Arezoo B. Dastres R. Cogn. Robot. 3 2023 54 70
20 Li S. Nguyen H.T. Cheah C.C. IEEE Access 11 2023 21992 22006
21 Liu Z. Peng K. Han L. Guan S. Iran. J. Sci. Technol. Trans. Mech. Eng. 47 2023 1307 1347
22 Nassif N. Al-Sadoon Z.A. Hamad K. Altoubat S. Struct. Eng. Mech. 83 2022 671 680
23 Alotaibi E. Nassif N. Barakat S. Struct. Concr. 24 2023 1856 1867
24 Magdy M. Fanni M. Mohamed A.M. Miyashita T. Mech. Mach. Theory 117 2017 253 275
25 Sharkawy A.N. Mech. Energy Eng. 6 2022
26 Tan C. Sun F. Kong T. Zhang W. Yang C. Liu C. A Survey on Deep Transfer Learning Artificial Neural Networks and Machine Learning–ICANN 2018: 27th International Conference on Artificial Neural Networks Rhodes, Greece, October 4-7, 2018, Proceedings, Part III 27 2018 Springer 270 279
27 Van Laarhoven T. arXiv preprint arXiv:1706.05350 2017
28 Yan Z. Zhong S. Lin L. Cui Z. Mathematics 9 2021 2176
