Agentes de software basados en técnicas de aprendizaje automático. Perspectivas desde 2010 hasta 2023

Hipatia Cazares Alegría; Pablo Pico Valencia

doi:10.24054/rcta.v1i45.3131

Autores/as

Hipatia Cazares Alegría Pontificia Universidad Católica del Ecuador https://orcid.org/0009-0006-1841-8853
Pablo Pico Valencia Universidad de Granada https://orcid.org/0000-0003-3518-3313

DOI:

https://doi.org/10.24054/rcta.v1i45.3131

Palabras clave:

aprendizaje automático, agente software, sistema multiagente, inteligencia artificial

Resumen

Este estudio tiene como objetivo analizar las principales propuestas teóricas y prácticas en las que los agentes de software se han integrado con modelos de aprendizaje automático para determinar su alcance en términos de inteligencia, proactividad, colaboración y aprendizaje. Para el desarrollo de esta investigación, se utilizó la metodología propuesta por Kofod-Peterson. Aplicando dicha metodología, se analizaron 55 estudios. Los estudios mostraron que en la interacción entre agentes de software y aprendizaje automático, los procesos cooperativos y colaborativos han sido ampliamente utilizados en la resolución de problemas de control y en la optimización de datos en escenarios distribuidos como el hogar, los juegos y las telecomunicaciones. También se encontró que, en su mayoría, se utilizaron modelos de aprendizaje por refuerzo en comparación con los modelos de aprendizaje automático, ya que contribuyen de manera más significativa a la modelización de tareas cooperativas, lo cual es ampliamente utilizado en sistemas inteligentes.

Descargas

Los datos de descargas todavía no están disponibles.

Citas

S. K. Polu, “Modeling of efficient multi-agent based mobile health care system,” Int J Innov Res Sci Technol, vol. 5, no. 8, pp. 10–14, 2019.

S. Munawar, S. K. Toor, M. Aslam, and E. Aimeur, “PACA-ITS: A Multi-agent system for intelligent virtual laboratory courses,” Applied Sciences (Switzerland), vol. 9, no. 23, 2019, doi: 10.3390/app9235084.

S. Cho and F. Zhang, “An adaptive control law for controlled lagrangian particle tracking,” WUWNet 2016 - 11th ACM International Conference on Underwater Networks and Systems, 2016, doi: 10.1145/2999504.3001077.

R. K. Jain et al., “Stability analysis of piezoelectric actuator based micro gripper for robotic micro assembly,” ACM International Conference Proceeding Series, no. c, 2013, doi: 10.1145/2506095.2506105.

M. Tanti, S. Fossey, L. Madrid-Briand, P. Carrieri, B. Spire, and P. Roux, “Une analyse de Twiter pour mieux comprendre les acteurs de la communication des nouvelles drogues et leurs discussions,” ACM International Conference Proceeding Series, pp. 36–38, 2018, doi: 10.1145/3240431.3240438.

Y. Islen and S. Juan, “Componente para la extracción y transformación de datos en el proceso de vigilancia tecnológica Component for the data mining and transformation within the technological surveillance process,” no. June 2017, 2016.

M. Kaisers, D. Bloembergen, and K. Tuyls, “A Common Gradient in Multi-agent Reinforcement Learning (Extended Abstract),” Proc. of 11th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS), pp. 1393–1394, 2012.

S. H. Chen and T. K. Fu, “Eliminating artificial-natural dichotomy a formal study on a core cognitive process in artificial intelligence,” ACM International Conference Proceeding Series, vol. Part F1285, 2017, doi: 10.1145/3080845.3080866.

S. R. Hamidi, E. N. M. Ibrahim, M. F. B. A. Rahman, and S. M. Shuhidan, “Industry 4.0 urban mobility: goNpark smart parking tracking module,” ACM International Conference Proceeding Series, pp. 503–507, 2017, doi: 10.1145/3162957.3163042.

C. Cappelli, G. V. Pereira, M. B. Bernardes, F. Bernardini, and A. Gomyde, “Building a reference model & an evaluation method for cities of the Brazilian network of smart & human cities,” ACM International Conference Proceeding Series, vol. Part F1282, pp. 580–581, 2017, doi: 10.1145/3085228.3085257.

A. A. F. Brandao, L. Vercouter, S. Casare, and J. Sichman, “Exchanging reputation values among heterogeneous agent reputation models: An experience on ART testbed,” Proceedings of the International Conference on Autonomous Agents, vol. 5, pp. 1047–1049, 2007, doi: 10.1145/1329125.1329405.

I. Menchaca, M. Guenaga, and J. Solabarrieta, “Using learning analytics to assess project management skills on engineering degree courses,” ACM International Conference Proceeding Series, vol. 02-04-Nove, pp. 369–376, 2016, doi: 10.1145/3012430.3012542.

A. Kofod-petersen, “How to do a Structured Literature Review in computer science,” 2014. [Online]. Available: https://research.idi.ntnu.no/aimasters/files/SLR_HowTo2018.pdf

E. Saadatian, T. Salafi, H. Samani, Y. De Lim, and R. Nakatsu, “An affective telepresence system using smartphone high level sensing and intelligent behavior generation,” HAI 2014 - Proceedings of the 2nd International Conference on Human-Agent Interaction, pp. 75–82, 2014, doi: 10.1145/2658861.2658878.

J. Jumadinova, P. Dasgupta, and L. K. Soh, “Strategic capability-learning for improved multi-agent collaboration in ad-hoc environments,” Proceedings - 2012 IEEE/WIC/ACM International Conference on Intelligent Agent Technology, IAT 2012, vol. 2, pp. 287–292, 2012, doi: 10.1109/WI-IAT.2012.57.

J. A. Manrique, J. S. Rueda-Rueda, and J. M. T. Portocarrero, “Contrasting Internet of Things and Wireless Sensor Network from a Conceptual Overview,” Proceedings - 2016 IEEE International Conference on Internet of Things; IEEE Green Computing and Communications; IEEE Cyber, Physical, and Social Computing; IEEE Smart Data, iThings-GreenCom-CPSCom-Smart Data 2016, pp. 252–257, 2017, doi: 10.1109/iThings-GreenCom-CPSCom-SmartData.2016.66.

T. H. Teng, A. H. Tan, J. A. Starzyk, Y. S. Tan, and L. N. Teow, “Integrating motivated learning and k-winner-take-all to coordinate multi-agent reinforcement learning,” Proceedings - 2014 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology - Workshops, WI-IAT 2014, vol. 3, pp. 190–197, 2014, doi: 10.1109/WI-IAT.2014.167.

D. P. Kingma, D. J. Rezende, S. Mohamed, and M. Welling, “Semi-supervised learning with deep generative models,” Adv Neural Inf Process Syst, vol. 4, no. January, pp. 3581–3589, 2014.

E. Levy, O. E. David, and N. S. Netanyahu, “Genetic algorithms and deep learning for automatic painter classification,” GECCO 2014 - Proceedings of the 2014 Genetic and Evolutionary Computation Conference, no. Dl, pp. 1143–1150, 2014, doi: 10.1145/2576768.2598287.

H. Kim, Y. Kim, and J. Hong, “Cluster management framework for autonomic machine learning platform,” Proceedings of the 2019 Research in Adaptive and Convergent Systems, RACS 2019, pp. 128–130, 2019, doi: 10.1145/3338840.3355691.

R. Radulescu, P. Vrancx, and A. Nowé, “Analysing congestion problems in multi-agent reinforcement learning,” Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, vol. 3, pp. 1705–1707, 2017.

S. Kim, Y. K. Row, and T. J. Nam, “Thermal interaction with a voice-based intelligent agent,” Conference on Human Factors in Computing Systems - Proceedings, vol. 2018-April, pp. 1–6, 2018, doi: 10.1145/3170427.3188656.

J. Yan, D. Hu, S. S. Liao, and H. Wang, “Mining agents’ goals in agent-oriented business processes,” ACM Trans Manag Inf Syst, vol. 5, no. 4, 2015, doi: 10.1145/2629448.

K. Hassani and W. S. Lee, “On designing migrating agents: From autonomous virtual agents to intelligent robotic systems,” SIGGRAPH Asia 2014 Autonomous Virtual Humans and Social Robot for Telepresence, SA 2014, 2014, doi: 10.1145/2668956.2668963.

D. Singh, L. Padgham, and B. Logan, “Integrating BDI Agents with Agent-Based Simulation Platforms,” Auton Agent Multi Agent Syst, vol. 30, no. 6, pp. 1050–1071, 2016, doi: 10.1007/s10458-016-9332-x.

A. Leite, R. Girardi, and P. Novais, “Using ontologies in hybrid software agent architectures,” Proceedings - 2013 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology - Workshops, WI-IATW 2013, vol. 3, pp. 155–158, 2013, doi: 10.1109/WI-IAT.2013.172.

R. Amin and S. Khalid, “Machine Learning Algorithms for Depression”.

A. Wilson and A. Fern, “Bayesian Role Discovery for ( Extended Abstract ),” Learning, pp. 1587–1588.

M. E. Taylor, B. Kulis, and F. Sha, “Metric learning for reinforcement learning agents,” 10th International Conference on Autonomous Agents and Multiagent Systems 2011, AAMAS 2011, vol. 2, pp. 729–736, 2011.

S. Hoet and N. Sabouret, “Reinforcement learning of communication in a multi-agent context,” Proceedings - 2011 IEEE/WIC/ACM International Conference on Intelligent Agent Technology, IAT 2011, vol. 2, pp. 240–243, 2011, doi: 10.1109/WI-IAT.2011.125.

C. Wu et al., “Spectrum Management of Cognitive Radio Using Multi-agent Reinforcement Learning,” in Proc. of 9th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 2010), 2010, pp. 10–14. [Online]. Available: www.ifaamas.org

S. Bromuri, “A tensor factorization approach to generalization in multi-agent reinforcement learning,” Proceedings - 2012 IEEE/WIC/ACM International Conference on Intelligent Agent Technology, IAT 2012, vol. 2, pp. 274–281, 2012, doi: 10.1109/WI-IAT.2012.21.

W. T. L. Teacy et al., “Decentralized Bayesian reinforcement learning for online agent collaboration,” 11th International Conference on Autonomous Agents and Multiagent Systems 2012, AAMAS 2012: Innovative Applications Track, vol. 1, pp. 312–319, 2012.

X. Zhu, C. Zhang, and V. Lesser, “Combining dynamic reward shaping and action shaping for coordinating multi-agent learning,” Proceedings - 2013 IEEE/WIC/ACM International Conference on Intelligent Agent Technology, IAT 2013, vol. 2, pp. 321–328, 2013, doi: 10.1109/WI-IAT.2013.127.

L. Torrey and M. E. Taylor, “Teaching on a Budget: Agents advising agents in reinforcement learning,” 12th International Conference on Autonomous Agents and Multiagent Systems 2013, AAMAS 2013, vol. 2, pp. 1053–1060, 2013.

C. Zhang and V. Lesser, “Coordinating multi-agent reinforcement learning with limited communication,” 12th International Conference on Autonomous Agents and Multiagent Systems 2013, AAMAS 2013, vol. 2, no. Aamas, pp. 1101–1108, 2013.

W. Rand, “Machine Learning Meets Agent-Based Modelling: When Not To Go: When Not To Go To a Bar,” 2006. [Online]. Available: https://ccl.northwestern.edu/papers/agent2006rand.pdf

P. Mannion, K. Mason, S. Devlin, J. Duggan, and E. Howley, “Multi-objective dynamic dispatch optimisation using Multi-Agent Reinforcement Learning,” Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, pp. 1345–1346, 2016.

H. Wang et al., “Integrating reinforcement learning with multi-agent techniques for adaptive service composition,” ACM Transactions on Autonomous and Adaptive Systems, vol. 12, no. 2, 2017, doi: 10.1145/3058592.

A. Marinescu, I. Dusparic, and S. Clarke, “Prediction-based multi-agent reinforcement learning in inherently non-stationary environments,” ACM Transactions on Autonomous and Adaptive Systems, vol. 12, no. 2, 2017, doi: 10.1145/3070861.

G. Henri and N. Lu, “A Multi-Agent Shared Machine Learning Approach for Real-time Battery Operation Mode Prediction and Control,” IEEE Power and Energy Society General Meeting, vol. 2018-Augus, pp. 1–5, 2018, doi: 10.1109/PESGM.2018.8585907.

P. Rosello and M. J. Kochenderfer, “Multi-agent reinforcement learning for multi-object tracking,” Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, vol. 2, pp. 1397–1413, 2018.

B. Khelifa and M. R. Laouar, “Multi-agent reinforcement learning for urban projects planning,” ACM International Conference Proceeding Series, 2018, doi: 10.1145/3330089.3330134.

H. Kazmi, J. Suykens, and J. Driesen, “Valuing knowledge, information and agency in multi-agent reinforcement learning: A case study in smart buildings: Industrial applications track,” Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, vol. 1, pp. 585–587, 2018.

P. Sunehag et al., “Value-decomposition networks for cooperative multi-agent learning based on team reward,” Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, vol. 3, pp. 2085–2087, 2018.

G. Palmer and K. Tuyls, “Lenient Multi-Agent Deep Reinforcement Learning,” no. Aamas, pp. 443–451, 2018.

J. Wang and L. Sun, “Dynamic holding control to avoid bus bunching: A multi-agent deep reinforcement learning framework,” Transp Res Part C Emerg Technol, vol. 116, no. April, p. 102661, 2020, doi: 10.1016/j.trc.2020.102661.

W. Amaral, G. Braz, L. Rivero, and D. Viana, “Using machine learning technique for effort estimation in software development,” ACM International Conference Proceeding Series, 2019, doi: 10.1145/3364641.3364670.

G. Palmer, R. Savani, and K. Tuyls, “Negative update intervals in deep multi-agent reinforcement learning,” Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, vol. 1, pp. 43–51, 2019.

R. B. Diddigi, K. J. Prabuchandran, D. Sai Koti Reddy, and S. Bhatnagar, “Actor-critic algorithms for constrained multi-agent reinforcement learning,” Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, vol. 4, pp. 1931–1933, 2019.

S. Bhalla, S. G. Subramanian, and M. Crowley, “Training cooperative agents for multi-agent reinforcement learning,” Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, vol. 3, pp. 1826–1828, 2019.

M. Kaushik, N. Singhania, S. Phaniteja, and K. M. Krishna, “Parameter sharing reinforcement learning architecture for multi agent driving,” ACM International Conference Proceeding Series, pp. 0–6, 2019, doi: 10.1145/3352593.3352625.

G. Bacchiani, D. Molinar, and M. Patander, “Microscopic traffic simulation by cooperative multi-agent deep reinforcement learning,” Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, vol. 3, pp. 1547–1555, 2019.

T. Molderez, B. Oeyen, C. De Roover, and W. De Meuter, “Marlon - a domain-specific language for multi-agent reinforcement learning on networks,” Proceedings of the ACM Symposium on Applied Computing, vol. Part F1477, pp. 1322–1329, 2019, doi: 10.1145/3297280.3297413.

M. Zhou et al., “Factorized Q-Learning for Large-Scale Multi-Agent Systems,” 2019.

D. S. K. Reddy, A. Saha, S. G. Tamilselvam, P. Agrawal, and P. Dayama, “Risk averse reinforcement learning for mixed multi-agent environments,” Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, vol. 4, no. 2, pp. 2171–2173, 2019.

Y. Zhao and X. Ma, “Learning efficient communication in cooperative multi-agent environment,” Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, vol. 4, pp. 2321–2323, 2019.

W. Zhou, Y. Chen, and J. Li, “Competitive Evolution Multi-Agent Deep Reinforcement Learning,” in CSAE2019, China, 2019.

H. R. Lee and T. Lee, “Improved cooperative multi-agent reinforcement learning algorithm augmented by mixing demonstrations from centralized policy,” Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, vol. 2, no. Aamas, pp. 1089–1098, 2019.

M. Ossenkopf, M. Jorgensen, and K. Geihs, “Hierarchical multi-agent deep reinforcement learning to develop long-term coordination,” Proceedings of the ACM Symposium on Applied Computing, vol. Part F1477, pp. 922–929, 2019, doi: 10.1145/3297280.3297371.

J. Castellini, R. Savani, F. A. Oliehoek, and S. Whiteson, “The representational capacity of action-value networks for multi-agent reinforcement learning,” Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, vol. 4, no. 1, pp. 1862–1864, 2019.

H. Zhang et al., “CityFlow: A multi-agent reinforcement learning environment for large scale city traffic scenario,” The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019, pp. 3620–3624, 2019, doi: 10.1145/3308558.3314139.

X. Li, J. Zhang, J. Bian, Y. Tong, and T. Y. Liu, “A cooperative multi-agent reinforcement learning framework for resource balancing in complex logistics network,” Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, vol. 2, pp. 980–988, 2019.

J. Hook, V. De Silva, and A. Kondoz, “Deep Multi-Critic Network for accelerating Policy Learning in multi-agent environments,” Neural Networks, vol. 128, pp. 97–106, 2020, doi: 10.1016/j.neunet.2020.04.023.

M. Uzair, L. Li, J. G. Zhu, and M. Eskandari, “A protection scheme for AC microgrids based on multi-agent system combined with machine learning,” 2019 29th Australasian Universities Power Engineering Conference, AUPEC 2019, pp. 17–22, 2019, doi: 10.1109/AUPEC48547.2019.211845.

N. El Ghouch, M. Kouissi, and E. M. En-Naimi, “Multi-agent adaptive learning system based on incremental hybrid case-based reasoning (IHCBR),” ACM International Conference Proceeding Series, 2019, doi: 10.1145/3368756.3369030.

J. Ma and F. Wu, “Feudal multi-agent deep reinforcement learning for traffic signal control,” Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, vol. 2020-May, pp. 816–824, 2020.

Y. Li, Y. Zheng, and Q. Yang, “Cooperative Multi-Agent Reinforcement Learning in Express System,” International Conference on Information and Knowledge Management, Proceedings, pp. 805–814, 2020, doi: 10.1145/3340531.3411871.

H. Mao, Z. Zhang, Z. Xiao, Z. Gong, and Y. Ni, Learning multi-agent communication with double attentional deep reinforcement learning, vol. 34, no. 1. Springer US, 2020. doi: 10.1007/s10458-020-09455-w.

C. Hu, “A confrontation decision-making method with deep reinforcement learning and knowledge transfer for multi-agent system,” Symmetry (Basel), vol. 12, no. 4, pp. 1–24, 2020, doi: 10.3390/SYM12040631.

O. Batata, V. Augusto, and X. Xie, “Mixed Machine learning and Agent-based Simulation for Respite Care Evaluation,” in Proceedings of the 2018 Winter Simulation Conference, 2016, pp. 1–23.

D. Dasic, M. Vucetic, M. Peric, M. Beko, and M. Stankovic, “Cooperative Multi-Agent Reinforcement Learning for Spectrum Management in IoT Cognitive Networks,” ACM International Conference Proceeding Series, vol. Part F1625, no. Cm, pp. 238–247, 2020, doi: 10.1145/3405962.3405996.

J. Yang, I. Borovikov, and H. Zha, “Hierarchical cooperative multi-agent reinforcement learning with skill discovery,” Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, vol. 2020-May, pp. 1566–1574, 2020.

S. Gupta, R. Hazra, and A. Dukkipati, “Networked multi-agent reinforcement learning with emergent communication,” Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, vol. 2020-May, no. i, pp. 1858–1860, 2020.

D. E. Hostallero, D. Kim, S. Moon, K. Son, W. J. Kang, and Y. Yi, “Inducing cooperation through reward reshaping based on peer evaluations in deep multi-agent reinforcement learning,” Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, vol. 2020-May, pp. 520–528, 2020.

Z. Zhang, J. Yang, and H. Zha, “Integrating independent and centralized multi-agent reinforcement learning for traffic signal network optimization,” Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, vol. 2020-May, pp. 2083–2085, 2020.

D. Zelasko, P. Plawiak, and J. Kolodziej, “Machine learning techniques for transmission parameters classification in multi-agent managed network,” Proceedings - 20th IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGRID 2020, pp. 699–707, 2020, doi: 10.1109/CCGrid49817.2020.00-20.