Control industrial por aprendizaje reforzado

Yessica Cindy Mora Cubides; Daniel Steven Arias Otálora; José Antonio Tumialan Borja; Hugo Fernando Velasco Peña

doi:10.24054/rcta.v2i46.4141

Authors

Yessica Cindy Mora Cubides Universidad de la Salle https://orcid.org/0009-0000-0831-9452
Daniel Steven Arias Otálora Universidad de la Salle https://orcid.org/0000-0002-3985-6020
José Antonio Tumialan Borja Universidade São Paulo https://orcid.org/0000-0002-1751-2657
Hugo Fernando Velasco Peña Universidad de la Salle https://orcid.org/0000-0002-4606-3889

DOI:

https://doi.org/10.24054/rcta.v2i46.4141

Keywords:

reinforcement learning, efficiency, artificial intelligence, industrial processes

Abstract

In this article, the implementation of the flow control technique with artificial intelligence (DDPG) is presented in a fully instrumented functional prototype with industrial sensors and actuators, simulating flow recirculation through three tanks. The methodology used for process identification (first-order plus dead time model (FOPDT)) through ClientServer OPC communication with Matlab® is presented. The design of the reinforcement learning algorithm and its adaptation in the learning environment with experimental data are also presented. The simulation results were satisfactory compared to traditional control techniques, demonstrating robustness against forced disturbances. Finally, the implementation of reinforced learning control integrating TIA Portal and Matlab (trough a PLC-S7-1500 controller) was evaluated with a reference of 600 l/h, achieving 0% overshoot with a settling time of 22s. Compared to other control systems, a better response in settling time and overshoot-free control was observed. Finally, perturbations were applied to the system, observing their effect in relation to the flow.

Downloads

Download data is not yet available.

References

Kaelbling, Leslie Pack; Littman, Michael L.; MOORE, Andrew W. Reinforcement learning: A survey. Journal of artificial intelligence research, 1996, vol. 4, p. 237-285

Carlos G´omez, J., Verrastro, C. A.. Aprendizaje por refuerzo y control difuso para generar comportamiento de robots. Edu.ar. from 2022, de https: //www.f rba.utn.edu.ar/wp content/uploads/2021/02/jar8submission30.pdf

Wang, Zhe; Hong, Tianzhen. Reinforcement learning for buildingcontrols: The opportunities and challenges. Applied Energy, 2020, vol.269, p. 115036.

Damjanovi ´C, Ivana, et al. High Performance Computing Reinforcement Learning Framework for Power System Control. En 2023 IEEEPower Energy Society Innovative Smart Grid Technologies Conference (ISGT). IEEE, 2023. p. 1-5.

V. Abad-Alcaraz, M. Castilla, J.D. Álvarez (2024) A Comparison of Classical and Reinforcement Learning-based Tuning Techniques for PI controllers, IFAC-papers OnLine Volume 58, Issue 7, 2024, Pages 180-185. doi: 10.1016/j.ifacol.2024.08.031.

Gunther, J., Reichensd¨orfer, E., Pilarski, P.M., and Diepold, K. (2020). Interpretable PID parameter tuning for control engineering using general dynamic neural networks: An extensive comparison. Plos One, 15(12), e0243320.

Ali, M., Firdaus, A.A., Arof, H., Nurohmah, H., Suyono, H., Putra, D.F.U., and Muslim, M.A. (2021). The comparison of dual axis photovoltaic tracking system using artificial intelligence techniques. IAES Int. J. Artif. Intell, 10(4), 901.

Daye Yang, Jingcheng Wang, Huihuang Cai, Jun Rao, Chengtian Cui, (2025) Intelligent control strategy for electrified pressure-swing distillation processes using artificial neural networks-based composition controllers, Separation and Purification Technology, Volume 360, Part 2, 2025, 130991, ISSN 1383-5866.

Ibtihaj Khurram Faridi, Evangelos Tsotsas, Wolfram Heineken, Marcus Koegler, Abdolreza Kharaghani, Development of a neural network model predictive controller for the fluidized bed biomass gasification process, Chemical Engineering Science, Volume 293, 2024, 120000, ISSN 0009-2509.

Hong-Gui Han, Lu Zhang, Hong-Xu Liu, Jun-Fei Qiao, Multiobjective design of fuzzy neural network controller for wastewater treatment process, Applied Soft Computing, Volume 67, 2018, Pages 467-478, ISSN 1568-4946.

P. Siva Krishna, P.V. Gopi Krishna Rao, Fractional-order PID controller for blood pressure regulation using genetic algorithm, Biomedical Signal Processing and Control, Volume 88, Part B, 2024, 105564, ISSN 1746-8094.

Marco Paz Ramos, Axel Busboom, Andriy Slobodyan, Genetic PI Controller Tuning to Emulate a Pole Assignment Design This work was supported by the Bavarian Ministry of Economic Affairs, Regional Development and Energy (StMWi) under Grant DIK0397/03., IFAC-PapersOnLine, Volume 58, Issue 7, 2024, Pages 138-143, ISSN 2405-8963.

K.G Aparna, R. Swarnalatha, Dynamic optimization of a wastewater treatment process for sustainable operation using multi-objective genetic algorithm and non-dominated sorting cuckoo search algorithm, Journal of Water Process Engineering, Volume 53, 2023, 103775, ISSN 2214-7144

Himanshukumar R. Patel, Optimal intelligent fuzzy TID controller for an uncertain level process with actuator and system faults: Population-based metaheuristic approach, Franklin Open, Volume 4, 2023, 100038, ISSN 2773-1863.

Slavica Prvulovic, Predrag Mosorinski, Dragica Radosav, Jasna Tolmac, Milica Josimovic, Vladimir Sinik, Determination of the temperature in the cutting zone while processing machine plastic using fuzzy-logic controller (FLC), Ain Shams Engineering Journal, Volume 13, Issue 3,2022, 101624, ISSN 2090-4479.

Rios H. R, Apráez B. A., Rodriguez C. J and Tumialan. B. J. A., PI tuning based on Bacterial Foraging Algorithm for flow control, (2020) IX International Congress of Mechatronics Engineering and Automation (CIIMA), Cartagena, Colombia, 2020, pp. 1-6, doi: 10.1109/CIIMA50553.2020.9290289.

Ali, M., Firdaus, A.A., Arof, H., Nurohmah, H., Suyono, H., Putra, D.F.U., and Muslim, M.A. (2021). The comparison of dual axis photovoltaic tracking system using artificial intelligence techniques. IAES Int. J. Artif. Intell, 10(4), 901.

Shuprajhaa, T., Sujit, S.K., and Srinivasan, K. (2022). Reinforcement learning based adaptive pid controller design for control of linear/nonlinear unstable processes. Applied Soft Computing, 128, 109450.

Lei, Y., Zhan, S., Ono, E., Peng, Y., Zhang, Z., Hasama, T., and Chong, A. (2022). A practical deep reinforcement learning framework for multivariate occupant centric control in buildings. Applied Energy, 324, 119742.

Yu, L., Xie, W., Xie, D., Zou, Y., Zhang, D., Sun, Z., Zhang, L., Zhang, Y., and Jiang, T. (2019). Deep reinforcement learning for smart home energy management. IEEE Internet of Things Journal, 7(4), 2751–2762.

Matetic, I., Stajduhar, I., Wolf, I., and Ljubic, S. (2023). Improving the efficiency of fan coil units in hotel buildings through deep-learning-based fault detection. Sensors, 23(15), 6717.

Andina de Integración Tecnológica (ANITCO), Manual de operación y mantenimiento unidad de entrenamiento en automatización, 1ra ed. Bogotá, Colombia: ANITCO, 2016.

Mathworks, Reinforcement Learning Toolbox™ User’s Guide, Matlab and Simulink, MathWorks, Natick, MA, USA, 2020. [En Línea] Disponible en: https://www.mathworks.com/help/reinforcement-learning/

Alfaro, Víctor M. Método de identificación de modelos de orden reducido de tres puntos 123c. Escuela de Ingeniería Eléctrica Universidad de Costa Rica, 2007, p. 1-7