Multi-agent system for raw material order validation based on a vision- language model
DOI:
https://doi.org/10.24054/rcta.v2i48.4320Keywords:
Deepseek OCR model, multi-agent system, document verification, orders, raw materials, vision-language modelAbstract
This document presents the results of integrating the Deepseek OCR model with a vision-to-language model and a multi-agent system to develop an automated order validation system. This system operates by interpreting and comparing documents related to a raw material order, such as invoices, purchase orders, and delivery notes. It is integrated with a web interface for uploading and processing documents and viewing results. This allows for obtaining accuracy metrics for the overall system, facilitating the analysis of multiple documents, and identifying inconsistencies during order receipt and management. For validation, 35 tests were conducted using real data from a company that manufactures cleaning products. The information interpretation and comparison workflow were successfully executed in 82.353% of the cases.
Downloads
References
W. Chu, S. Yin, L. Huang, L. Lin, X. Wang, Z. Zhang and H. Li, "Verify-Agent: Large Language Model Multi-Agent for Intelligent Verification," in 2024 International Conference on Ubiquitous Computing and Communications (IUCC), Chengdu, China, 2024, doi:10.1109/IUCC65928.2024.00072.
P. H. Luzolo, Z. Elrawashdeh, I. Tchappi, S. Galland and F. Outay, "Combining multi-agent systems and artificial intelligence of things: Technical challenges and gains," Internet of Things, vol. 28, p. 101364, 2024, doi:10.1016/j.iot.2024.101364.
M. Z. R. Farazi, "Enhancing supply chain resilience with multi-agent systems and machine learning: a framework for adaptive decision-making," The American Journal of Engineering and Technology, vol. 7, no. 3, pp. 6-20, 2025, doi:10.37547/tajet/Volume07Issue03-02.
M. Mousa, D. Van de Berg, N. Kotecha, E. A. del Rio Chanona and M. Mowbray, "An analysis of multi-agent reinforcement learning for decentralized inventory control systems," Computers & Chemical Engineering, vol. 188, p. 108783, 2024, doi:10.1016/j.compchemeng.2024.108783.
G. Ziegner, M. Choi, H. M. C. Le, S. Sakhuja and A. Sarmadi, "Iterative Multi-Agent Reinforcement Learning: A Novel Approach Toward Real-World Multi-Echelon Inventory Optimization," arXiv, 2025, doi:10.48550/arXiv.2503.18201.
N. Kotecha and A. del Rio Chanona, "Leveraging graph neural networks and multi-agent reinforcement learning for inventory control in supply chains," Computers & Chemical Engineering, vol. 199, p. 109111, 2025, do:10.1016/j.compchemeng.2025.109111.
J. Hu, L. Xia, T. Huang and H. Wu, "A multi-agent deep reinforcement learning approach for multi-echelon inventory optimization and its application to the beer game," Transportation Research Part E: Logistics and Transportation Review, vol. 203, p. 104367, 2025, doi:10.1016/j.tre.2025.104367.
G. Kim, T. Hong, M. Yim, J. Nam, J. Park, J. Yim, W. Hwang, S. Yun, D. Han and S. Park, "OCR-Free Document Understanding Transformer," in Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol. 13688, S. Avidan, G. Brostow, M. Cissé, G. Farinella and T. Hassner, Eds., Springer, 2022, doi:10.1007/978-3-031-19815-1_29.
S. Chen, X. Guo, Y. Li, T. Zhang, M. Lin, D. Kuang, Y. Zhang, Z. Zhou and W. Chen, "Ocean-OCR: Towards general OCR application via a vision-language model," arXiv, 2025, doi:10.48550/arXiv.2501.15558.
L. Hamdi, A. Tamasna, P. Boisson and T. Paquet, "VISTA-OCR: Towards generative and interactive end to end OCR models," arXiv, 2025, doi:10.48550/arXiv.2504.03621.
G. Monteiro, L. Camelo, G. Aquino, R. Fernandes, R. Gomes, A. Printes, I. Torné, H. Silva, J. Oliveira and C. Figueiredo, "A Comprehensive Framework for Industrial Sticker Information Recognition Using Advanced OCR and Object Detection Techniques," Applied Sciences, vol. 13, no. 12, p. 7320, 2023, doi:10.3390/app13127320.
R. Sapkota and M. Karkee, "Object detection with multimodal large vision-language models: An in-depth review," Information Fusion, vol. 126, no. Part A, p. 103575, 2026, doi:10.1016/j.inffus.2025.103575.
K. Medini, D. Romero and T. Wuest, "Developing a multi-agent system to support multi-variant production ramp-up management," Smart and Sustainable Manufacturing Systems, vol. 5, no. 1, pp. 129-147, 2021, doi:10.1520/SSMS20200082.
Z. Zhao, D. Tang, C. Liu, L. Wang, Z. Zhang, H. Zhu, K. Chen, Q. Nie and Y. Ji, "A Large language model-based multi-agent manufacturing system for intelligent shopfloors," Advanced Engineering Informatics, vol. 69, no. Part A, p. 103888, 2026, doi: 10.1016/j.aei.2025.103888.
V. Jannelli, S. Schöpf, M. Bickel, T. Netland and A. Brintrup, "Agentic LLMs in the supply chain: towards autonomous multi-agent consensus-seeking," International Journal of Production Research, p. 1–31, 2025, doi:10.1080/00207543.2025.2604311.
Google, "Agent Development Kit," 2025. [Online]. Available: https://google.github.io/adk-docs/agents/. [Accessed 28 Enero 2026].
A. Bandi, B. Kongari, R. Naguru, S. Pasnoor and S. Vilipala, "The Rise of Agentic AI: A Review of Definitions, Frameworks, Architectures, Applications, Evaluation Metrics, and Challenges," Future Internet, vol. 17, no. 9, p. 404, 2025, doi:10.3390/fi17090404.
H. Guo, Y. Li, Y. Liu and W. Li, "Evaluation of string comparators for record linkage in Chinese environment," International Journal of Wavelets, Multiresolution and Information Processing, vol. 20, no. 6, p. 2250023, 2022, doi:10.1142/S0219691322500230.
C. Nantasenamat, A. Biswas, J. Nápoles-Duarte, M. I. Parker and R. L. Dunbrack, "Chapter 27 - Building bioinformatics web applications with Streamlit," in Cheminformatics, QSAR and Machine Learning Applications for Novel Drug Development, K. Roy, Ed., Academic Press, 2023, pp. 679-699, doi:10.1016/B978-0-443-18638-7.00001-3.
H. Wei, Y. Sun and Y. Li, "Deepseek-OCR: Contexts optical compression.," arXiv , 2025. https://doi.org/10.48550/arXiv.2510.18234.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Anny Astrid Espitia Cubillos, Robinson Jiménez Moreno, Mateo Pulido Aponte
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
