The two-stage Stretch-Blow Molding (SBM) process is the most popular technique used for the manufacturing of Polyethylene Terephthalate (PET) bottles  . This process involves feeding a structurally amorphous semi-product, called preforms, made by injection molding of PET resin granules into the blow-molding machine. A pre-heating step using a set of infrared (IR) emitting lamps conditions the preforms to the appropriate temperature distribution along the walls. In the second stage, the preforms are stretched axially by a cylindrical rod and blown using two levels of air pressure. Then, the bottles are cooled down by a mold whose temperature is regulated using cooling channels. Most bottles blown in this matter require a minimum blow air pressure of 35 bar with 40 bar as a maximum. Compressed air takes up approximately 50% of bottle production energy cost and higher air pressure increases overall bottle cost. Recycling high-pressure air for pre-blow pressure is now widely used to reduce energy and cut overall costs.
To achieve the performance speciﬁcations deﬁned by water, dairy and juice manufacturers, the bottles must satisfy a large number of performance criteria such as top load, thickness distribution, transparency and barrier properties. Several factors weigh in on the compromise between optimal bottle quality and economic considerations and they include blowing temperature, heat profile along the preform wall, stretch ratios, wall thickness distribution, and air blow timing. The parameters affecting the ﬁnal properties of a bottle fall into two main categories; biaxial stretching, and process heating conditions. The preform distribution temperature controls the blowing kinematics (stretching and blowing), and consequently the wall thickness distribution of the bottle. Numerical optimization methods for SBM have received more and more attention in the last decade in order to substitute the costly, time consuming and inefﬁcient trial and error method used by operators and factories. There have seen several numerical methods developed simulating the SBM heating and blowing processes. Researchers used the numerical models developed using available Finite Element (FE) packages    or have developed their own software     in order to predict the process temperature distribution. Pham et al.  characterized two grades of PET preforms using biaxial tests to determine the model parameters of a robust nonlinear curve-fitting program. The developed model adequately represented the stretching behavior of PET. Based on this model, a finite element formulation was developed to simulate the stretch blow molding process. In  Yang et al. carried out finite element analysis SBM process of polyethylene terephthalate (PET) bottles with a view to optimizing preform designs and process conditions. Simulations with bottle thickness predictions were achieved. Bagherzadeh et al.  used finite element method model to predict an overall trend of thickness distribution. However, some differences can be seen in preform regions of 10 mm and 125 mm. The results were used for an overall prediction of bottle properties. In  Brovidal et al. presented an optimization strategy developed for the stretch-blow molding process. The method was based on a coupling between the Nelder-Mead optimization algorithm and Finite Element (FE) simulations of the forming process developed and the temperature distribution of the perform was predicted using a 3D ﬁnite-volume software. In addition, a thermodynamic model was used to predict the air pressure applied inside the preform. Chettiar et al.  presented an investigation on the optimization of the preform design to produce PET bottles with uniform thickness distribution using a blow molding simulation software. The bottle blown up using the new preform design had a better barrier property due to increased molecular orientation. Thibault et al.  developed a predictive preform geometry software and optimal operating conditions for the stretch blow molding process. The numerical approach combined a constrained gradient-based optimization algorithm together with finite difference technique for operating condition optimization. Venkateswaran et al.  investigated the inﬂuence of nonuniform temperature distributions on preform sidewalls, in relation to their effects on functional properties of PET bottles. The study demonstrated that the optical anisotropy through the bottle wall thickness is minimal, when the inside surface is at a higher temperature than the outside surface.
Temperature is considered one of the most important variables in SBM. However, its measurement and predictions remain a delicate task, especially in the wall thickness direction. Temperature affects the orientation induced by biaxial stretching, affects mechanical, optical, and barrier properties of bottles. Heating simulation of multilayer preforms, authors took into account the relationship between the scattering coefﬁcient of PET and crystallization rate. Therefore, the simulation of the preform heating cannot be carried out adequately without an understanding of the radiative heat transfer properties. Another aspect is the blowing and stretching pressures that could very well determine the full bottle strength. The internal pressure and the enclosed volume of the preform are fully coupled and applying the air pressure directly as a boundary condition could lead to unrealistic results    . These models are generally based on thermodynamic equations in order to automatically compute the air pressure inside the preform, instead of applying the pressure directly as boundary conditions. Only a few studies have been reported on this speciﬁc point, and the subject is still opened. The models proposed in literature simulate the deformation process, and generally lead to accurate predictions of bottle thickness distribution. However, even though kinematics of blowing may be the criteria to test the accuracy of results, a lack of heat transfer modeling is also apparent due to high non-linearity and the temperature distribution zones through the preform wall thickness. Moreover, heat transfer coefﬁcients are generally estimated without reference to experimental measurements. Finally, only a few studies have proposed cost reduction modeling of preform weight, a key to successful operation, and the resulting heating and the blowing parameters.
In this work, we present Neural Networks (NN) approach to achieve a predictive model of the SBM process. The model is mainly based on a Least-Mean-Square (LMS) algorithm, also known as delta rule, trained with back-propagation to simulate the Infra-Red (IR) heating and blowing stages of the forming stage. Throughout the testing and simulation of neural network parameters, Swing Neural Networks  was used on a set of real-time data. This method provides a predictive approach for the relationship between the internal temperature and air pressure and the enclosed parameters of the preform. In a second step, we propose an optimization strategy for SBM of the lowest possible weight that can be used for the preform providing a uniform thickness for the bottle. For that, we developed an iterative procedure allowing to automatically compute the best temperature and pressure distribution along the preform length. To validate our approach, numerical and experimental temperature and pressure proﬁles were compared. Results were validated by careful in situ tests and measurements performed on various weights preforms of 330 ml PET bottles.
2. Characteristics of PET
The properties of the PET polymer are largely dependent upon the average molecular weight, which is usually determined by measurement of the Intrinsic Viscosity (IV). The higher the IV is, the higher the molecular weight is. The range of IVs used in preform injection molding is 0.72 to 0.88 depending on the size of the preform. The degree of crystallinity of the preform is determined by its density measurement. Amorphous PET has a density of 1.335 g/m3, the calculated density of perfect PET crystal is 1.455 g/m3 at room temperature  . An increase in IV reduces the rate of crystallization making it more difficult for a given chain to separate from other chains and form an ordered crystal  . The Natural Stretch (NSR) or draw ratio of a polymer is the ratio of the resulting length (in the direction of applied stress) to its original length. The point at which the PET requires extra force to continue stretching is the NSR for a particular set of stretching conditions. A resin with a low IV has a higher NSR than a high IV resin since polymer chains in a low IV resin are shorter, and therefore, can be easily stretched more. Whereas, in high IV resin, chain entanglement limits the amount of stretching. For this reason, preform designs differ when considering low IV or high IV PET. The NSR is reached when strain hardening occurs on the stress-strain curve for materials. Stretching beyond the yield point results in permanent deformation of the bottle, and further stretching will result in fracture. The design of a PET preform for the optimum orientation is achieved when the stretching of the preform walls meets the mold boundaries. This point occurs beyond the NSR. Proper stretching results in longer shelf life, better top load performance and less gas permeability, for example, higher carbon dioxide retention for carbonated drinks. Overstretching results in excessive deformation of the bottle. Axial stretch ratio dividing the length of the bottle by the length of the preform as measured from underneath the neck support ring (NSR) to the end of bottle and preform. Hoop stretch ratios can be measured by dividing the average bottle diameter by the corresponding average preform diameter. Stretch ratio calculations and typical values are as follows:
HSR typical Figurers are 3 to 4 whereas ASR are 2.5 to 3  . These values give a more uniform stretch distribution and top load strength to the bottle. Figure 1 shows a combined 12-gram preform and bottle with IV of 0.738 dl/g used in stretching of a 330 ml bottle.
PET has temperature process range from 90˚C - 115˚C which can be adjusted by the oven lamps voltage range of 0 - 220 V (a percentage setting of 0% - 99%). Higher temperatures cause the PET material to crystallize whereas temperatures below 85˚C causes microcracks in the PET structure. Conduction, convection and radiation are all present heat transfer mechanisms in the SBM process. Conduction occurs when two parts touch each other and heat flows from the colder to the warmer part. However, PET is a poor conductor and it is not suitable to heat preforms this way. On the other hand, convection is done by heating up of air in the ovens of the blow machine takes place. Air heating is difficult to control and highly dependent on surrounding conditions and furthermore the outside walls of the preform heats up more than the inside as depicted in Figure 2. Radiation occurs when the output of the oven lamps reaches the preforms in the form of waves.
3. Neural Network Architecture
Most physical systems have to be modelled separately due to complexity and presence of non-linear characteristics   . Neural Networks (NN) have been used in a large number of applications and have proven to be effective in
Figure 1. Bottle 330 ml and preform IV 0.738 dimensions and stretch ratios used.
Figure 2. Typical preform temperatures measured on outside and inside walls.
performing complex functions in various fields. These include pattern recognition, classification, control systems, vision, and predictions  . NN accomplishes these tasks by learning from a series of data sets to the system, then applies what has learned to approximate or predict the corresponding output. The most successful modern network-learning model has been back-propagation. Based on Least Mean Square (LMS) algorithm, the back-propagation method has been widely used to solve a number of applications     . Typically, neural networks consist of three layers linked together by a weighted connection between neurons of different layers. Figure 3 shows a typical multi-layer neural network structure.
The input layer receives information from the external sources and passes the information to the network for processing. The next layer consists of one or more hidden layers that receive information from the input layer and processes them to other layers within the system. The output layer receives processed information and sends output signals out of the system. Bias connections are offsets that provide a threshold for the activation of neurons. During learning phase, NN attempts to optimize the connection weights to give best approximation of the parameter in question. Once trained, NN can be used on line to estimates output parameters of the non-linear system. The node’s internal threshold is the magnitude offset that affects the activation of the node output yk as follows:
where Wkp is weight factors associated with each node to determine the strength of input row vector Xp.
The transfer function is a non-linear activation function, which performs a mathematical operation on the output. Two examples of activation transfer function include sigmoidal transfer function of the form.
Figure 3. Typical back-propagation network.
And hyperbolic tangent transfer function of the form:
4. Training the Neural Network
The objective of training the network is to adjust the weights with a set of training data. The iterative process is designed to minimize the mean square error between the actual and desired output. Typically, new inputs will lead to outputs close to the desired output for input vectors used in training that are similar to the new input being presented. This property makes it possible to train a network on a representative set of input-output target pairs and get good results without training the network on all possible input/output pairs. A trained multi-layer perceptron can perform non-linear input-output mapping. In Least-Mean-Square (LMS) algorithm training method, it is based on estimates of the auto-correlation function and estimate of the cross-correlation function . These estimators are used to update the weight parameter in the steepest decent method calculations, the output function of the neuron is computed as:
A generalization of the LMS algorithm, known as backpropagation, is an error-correction learning rule, which is widely used to train multi-layer feed forward networks. The iterative training process of the backpropagation network is designed to minimize the LMS error E defined by:
where dk and yk are the desired and actual outputs respectively. Basic steps involved in this procedure are the following:
Step 1: The weights and offsets of different layers are initialized to small random values within the interval [−1, 1].
Step 2: The input is presented and output of the first layer is calculated. The input can have a new value for each iteration, or it can be sampled from a training set.
Step 3: The actual output is calculated using the non-linear activation function.
Step 4: The weights are adapted and adjusted by the following equation,
where δj is the error term for node j, η is the gain term, and α is the momentum term. The momentum coefficient provides faster convergence and is usually set at a value in the interval of [0, 1]. This allows the network to respond to recent trends in the error gradient and therefore, the network will not get stuck in local minimum. The error reached by this local minimum does not represent the best set of weights. A too high momentum coefficient can cause instability, and will result in oscillation across the global minimum    . The learning rate setting is crucial in training, too small or too high values may lead to instability.
5. Model Development and Optimization—Backpropagation
Throughout the back-propagation simulation presented, the network was initialized based on the technique described by Nguyen and Widrow   . To determine the number of layers and the corresponding number of hidden neurons M, the criterion used is smallest number of neurons that yield a minimum RMS error with the least number of iterations. The values of the learning rate η and momentum coefficient α were arbitrary set at small values. The network was simulated on 800 training data set for various numbers of hidden neurons in each layer. Table 1 and Table 2 summarize the results of the simulation for various numbers of hidden neurons.
The second run resulted in a considerable drop in the number of cycles. The third run did not significantly contribute to the MSE. Accordingly, the number of hidden neurons was held at 3 for the first layer, and 3 for the second layer.
6. Cross-Validation Analysis
When a training set is picked from the available data, there is a need to validate the model on a data set that is different from the training set. Over-fitting can
Table 1. Summary of Simulation Results for determination of number of hidden neurons.
Learning rate parameter η = 0.7 and momentum coefficient α = 0.8
Table 2. Optimal configuration of back-propagation network.
occur when too many parameters are selected, while under-fitting can occur when few parameters are used. The particular model, which gives the best performance, is then trained on the full training data set and generalization is then tested on the verification set. Cross-validation can give essential information that can help in network parameter selection such as size of training set and value of learning rate. The available data is partitioned to two sets, training and test set. The training set is further partitioned into two sets: training and validation. The network is then simulated with the selected training data set. Results of the cross-validation curves are shown in Figure 4.
The results, in general, are satisfactory since the test and validation errors have similar characteristics. No over- or under-fitting is visible and therefore, we conclude that the training set and parameters chosen are adequate.
7. Simulation Results
Based on plant experience we included 9 input variables in the training of the SBM networks. The following L2 and norms error criteria were used for comparison and error analysis.
where T is the actual target value, n is the number of input-output input vectors, and Ann is the neural net predicted output. The SMB system is a two preheat-stage COMBI SBO14 SIDEL system that is operated by New Technology Bottling Company, Kuwait Free Trade Zone, Kuwait. With 14 molds for PET
Figure 4. LMS and corresponding validation curves for selected values of learning rate.
bottle, the system has a capacity of 28,000 bottles per hour (bph). Preforms used throughout the study were 30/25 neck finish clear with IV 0.747 dl/gram and has residual Acetaldehyde (AA) content of 0.670 ppm. The beating zones were powered by six-zone IR tungsten lamps. The compressed air for stretch blow molding was powered by 40 bar Ateliers François compressor. The plant was operated and tested over a six-month period for the study, with data collection sampled every hour during the operation. In this example, 7 input variables were used in the input vector.
The input variables used for the NN were preform weight, length, width, outside diameter, inner diameter, ASR, HSR and wall thickness. Figure 5 shows that preform weight, length and thickness at the top of the list as far as relative importance to the output values. These three parameters are also the most cost-effective for risen converters and bottle manufacturers as well. Figure 6 shows the training network used and heat zoning and pressure selection. We attempt to predict the zone heating values, blowing and stretching pressures as our output variables. In addition, we investigated the lowest possible weight that can be used for a 330 ml bottle provided that the bottle maintains stability and rigidness. The pressure and temperature proﬁles, scaled from 0 - 10 for different preform types and sizes, shown in Figures 7-9 and provided a prediction of the blowing kinematics.
Figure 5. Input variables relative importance.
Figure 6. Training network used.
Figure 7. Grid heating and blowing pressure set up.
Figure 8. Actual vs. estimated predicted blowing.
Figure 9. Actual vs. estimated predicted temperature (˚C) for selected zone 3.
This work proposed a modeling of the full Stretch Blow Molding process. The IR heating, blow molding and stretch blow molding steps were simulated using Swing NN software. All the conditions required for the predictive simulations were carefully measured and calibrated. A major contribution of this work remains the modeling of the air-barrier interaction existing between the outside of the preform and the air ﬂow applied inside. A network validation has shown that the model successfully captures pressure and temperature proﬁles, scaled from 0 - 10 for different preform types and sizes, shown in Figure 8 and Figure 9, and provided a prediction of the blowing kinematics. The model was also used to predict the parameters for a lower preform weight to 11.5 gram for the bottle, thereby reducing the cost per unit, the parameter results were tested and the produced bottles met the manufacturer specifications of strength and top load characteristics. Future work will aim to further improve the model by providing more data on different preforms’ weights and thickness, which would account for the relationship between the temperature and the material orientation and crystallization. As for SBM optimization, we proposed a practical methodology to numerically optimize a lower weight of a PET preform for a particular 0.330 ml size bottle, while providing a uniform thickness alongside the bottle walls. Encouraging prediction results have shown the viability of NN approach. However, it would probably be more desirable to have the outer wall IR heat temperature, along with the inside wall, measured and used as an input for the NN model. But to do so, major blower modifications will have to be carried out. Nevertheless, this approach would implicitly account for the inﬂuence of the temperature and pressure distributions through the preform weight, thickness, length, and stretch ratios.
NN can be very effective in predicting non-linear behavior problems associated with operational variables of SBM process. NN algorithms achieved an overall good generalization for predictions of blowing kinematics, mainly blowing pressure and zone temperatures. The prediction of the last zone temperature resulted in overall lower accuracy due to the fact of the high non-linearity of hot air distribution between inner and outer preform walls and dependence on other factors such as distance, wall thickness, stretching ratios and inner- and outer-diameters.
Financial support of the Public Authority for Applied Education and Training (PAAET), Kuwait during this research is greatly appreciated.
 Luersen, M.A. and Le Riche, R. (2002) Globalized Nelder-Mead Method for Engineering Optimization. Proceedings of the Third International Conference on Engineering Computational Technology, Stirling, Scotland, 165.
 Champin, C., Bellet, M., Schmidt, F.M., Agassant, J.-F. and Le Maoult, Y. (2005) 3D Finite Element Modeling of the Blow Molding Process. 8th ESAFORM Conference on Material Forming, Cluj-Napoca, Romania, 905-908.
 Bordival, M., Schmidt, F.M., Le Maoult, Y. and Velay, V. (2009) Optimization of Preform Temperature Distribution for the Stretch-Blow Molding of PET Bottles: Infrared Heating and Blowing Modeling. Polymer Engineering and Science, 49, 783-793.
 Venkateswaran, G., Cameron, M.R. and Jabarin, C.A. (1998) Optimization of Preform Temperature Distribution for the Stretch-Blow Molding of PET Bottles: Infrared Heating and Blowing Modeling. Advances in Polymer Technology, 17, 237-249.
 Martin, L., Stracovsky, D., Laroche, D., Bardetti, A., Ben Yedder, R. and DiRaddo, R. (1999) Modeling and Experimental Validation of the Stretch Blow Molding of PET. SPE ANTEC Technical Papers, New York.
 Bordival, M., Schmidt, F.M. and Le Maoult, Y. (2006) Numerical Modeling and Optimization of Infrared Heating System for the Blow Molding Process. 9th ESAFORM Conference on Material Forming, Glasgow, April 2006, 511-514.
 Pham, X.T., Thibault, F. and Lim, L.T. (2004) Modeling and Simulation of Stretch Blow Molding of Polyethylene Terephthalate. Polymer Engineering and Science, 44, 1460-1472.
 Yang, Z.J., Harkin-Jones, E., Armstrong, C.G. and Menary, G.H. (1996) Finite Element Modelling of Stretch-Blow Molding of PET Bottles Using Buckley Model: Plant Tests and Effects of Process Conditions and Material Parameters. Proceedings of the Institution of Mechanical Engineers Part E: Journal of Process Mechanical Engineering, 218, 237-250.
 Bagherzadeh, S., Biglari, F.R. and Nikbin, K. (2010) Parameter Study of Stretch-Blow Moulding Process of Polyethylene Terephthalate Bottles Using Finite Element Simulation. Proceedings of the Institution of Mechanical Engineers Part B: Journal of Engineering Manufacture, 224, 1217-1227.
 Thibault, F., Malo, A., Lanctot, B. and Diraddo, R. (2007) Preform Shape and Operating Condition Optimization for the Stretch Blow Molding Process. Polymer Engineering and Science, 47, 289-301.
 Bonnebat, C., Roullet, G. and de Vries, A.J. (2004) Biaxially Oriented Polyethylene Terephthalate Bottles: Effects of Resin Molecular Weight on Stretching Behavior. Polymer Engineering & Science, 21, 189-195.
 Zilouchian, A. and Bawazeer, K. (1997) Prediction of Products Quality of a Crude Fractionation Section of an Oil Refinery Using Neural Networks. Proceedings of International Conference on Neural Networks, Houston, TX, 12-12 June 1997, 157-162.
 Zilouchian, A. and Hamano, F. (1997) Recent Trends and Industrial Applications of Intelligent Control Systems Using Artificial Neural Networks and Fuzzy Logic. In: Tzafestas S.G., Ed., Methods and Applications of Intelligent Controls, Kluwer Academic Publishers, Dordrecht.
 Nguyen, D. and Widrow, B. (1990) Improving the Learning Speed of 2-Layer Neural Networks by Choosing Initial Values of the Adaptive Weights. Proceedings of the International Joint Conference on Neural Networks, 3, 21-26.