A cascading hybrid model for assigning reviewers to scientific articles
DOI:
https://doi.org/10.24054/rcta.v2i48.4492Keywords:
machine learning (ML), peer review, artificial intelligence (AI), reviewer assignment, hybrid cascade models, natural language processing (NLP)Abstract
Scientific journals face delays in publishing new issues due to a shortage of expert reviewers and an increase in article submissions. This study proposes a combination of models with a cascading desing that utilizes natural language processing (NLP) and machine learning (ML) tecniques to automate the assignment of reviewers. A quantitative approach with an experimental design was adopted, using historical records extracted from the OJS platform of a real scientific journal in the field of Social Sciences, focusing on the disciplines of Business administration, accounting, economics, pedagogy applied to business development, entrepreneurship, local development, innovation and technology (2021-2025). The model is structured in three sequential stages: abstract acceptance classification, disciplinary field classification, and reviewer speed classification. The results were as follows: 100 combinations of textual representations, classifiers, and data balancing techniques were evaluated using 5-fold stratified cross-validation; stage 1, F1-macro=0.663 (SBERT+SMOTE+Random Forest); Stage 2, F1-Macro=0.966(TF-IDF+SMOTE+XGBoost); and stage 3, F1=0.9578 (TF-IDF+NumCat+RUS+Random Forest). In conclusión, this cascade design proved to be viable, modular, and interpretable for automating reviewer assignment using real-world data (213 articles, 117 evaluators). The sequential architecture allowed each stage to operate with the optimal representation for its data, contain errors, and validate the model with real-world data.
Downloads
References
J. P. Gisbert y M. Chaparro, «Reglas y consejos para ser un buen revisor por pares de manuscritos científicos», Gastroenterol. Hepatol., vol. 46, n.o 3, pp. 215-235, mar. 2023, doi: 10.1016/j.gastrohep.2022.03.005.
P. K. Bharti, T. Ghosal, M. Agarwal, y A. Ekbal, «PEERRec: An AI-based approach to automatically generate recommendations and predict decisions in peer review», Int. J. Digit. Libr., vol. 25, n.o 1, pp. 55-72, mar. 2024, doi: 10.1007/s00799-023-00375-0.
J. A. Huete-Perez y N. Salvatierra, «Assessing biomedical research capacities in selected countries of Latin America: challenges, opportunities, and recommendations», Front. Res. Metr. Anal., vol. 10, jul. 2025, doi: 10.3389/frma.2025.1594303.
A. Severin y J. Chataway, «Overburdening of peer reviewers: A multi-stakeholder perspective on causes and effects», Learn. Publ., vol. 34, n.o 4, pp. 537-546, 2021, doi: 10.1002/leap.1392.
A. Becerril García y S. Córdoba González, Eds., Conocimiento abierto en América Latina: trayectoria y desafíos. en Colección Grupos de trabajo. Erscheinungsort nicht ermittelbar: CLACSO, 2021.
J. A. C. Osorio, «Problemáticas de las revistas científicas colombianas»:, Sci. Tech., vol. 29, n.o 4, pp. 145-147, dic. 2024, doi: 10.22517/23447214.25746.
J. C. Calderón, «Dear Editor», Colomb. Medica, vol. 42, n.o 3, pp. 406-407, 2011, doi: 10.25100/cm.v42i3.889.
N. Felisi, R. Vecchio, y A. Odone, «When peer review drags on: the harm to early career researchers», Front. Res. Metr. Anal., vol. 11, feb. 2026, doi: 10.3389/frma.2026.1740381.
B.-C. Björk y D. Solomon, «The publishing delay in scholarly peer-reviewed journals», J. Informetr., vol. 7, n.o 4, pp. 914-923, oct. 2013, doi: 10.1016/j.joi.2013.09.001.
J. Huisman y J. Smits, «Duration and quality of the peer review process: the author’s perspective», Scientometrics, vol. 113, n.o 1, pp. 633-650, oct. 2017, doi: 10.1007/s11192-017-2310-5.
Publons, «Publons’ Global State Of Peer Review 2018», Publons, London, UK, sep. 2018. doi: 10.14322/publons.GSPR2018.
B. Aczel, B. Szaszi, y A. O. Holcombe, «A billion-dollar donation: estimating the cost of researchers’ time spent on peer review», Res. Integr. Peer Rev., vol. 6, n.o 1, p. 14, nov. 2021, doi: 10.1186/s41073-021-00118-2.
T. A. Mahmud, B. M. M. Hossain, y D. Ara, «Automatic Reviewers Assignment to a Research Paper Based on Allied References and Publications Weight», en 2018 4th International Conference on Computing Communication and Automation (ICCCA), Greater Noida, India: IEEE, dic. 2018, pp. 1-5. doi: 10.1109/CCAA.2018.8777730.
X. Zheng, P. Li, y X. Wu, «An Automatic Paper-Reviewer Recommendation Algorithm Based on Depth and Breadth», IEEE Trans. Emerg. Top. Comput. Intell., vol. 9, n.o 1, pp. 607-616, feb. 2025, doi: 10.1109/TETCI.2024.3402694.
P. K. Bharti, S. Ranjan, T. Ghosal, M. Agrawal, y A. Ekbal, «PEERAssist: Leveraging on Paper-Review Interactions to Predict Peer Review Decisions», en Towards Open and Trustworthy Digital Societies, Springer, Cham, 2021, pp. 421-435. doi: 10.1007/978-3-030-91669-5_33.
L. Ramachandran, E. F. Gehringer, y R. K. Yadav, «Automated Assessment of the Quality of Peer Reviews using Natural Language Processing Techniques», Int. J. Artif. Intell. Educ., vol. 27, n.o 3, pp. 534-581, sep. 2017, doi: 10.1007/s40593-016-0132-x.
A. Perkovic Palos et al., «Linguistic and semantic characteristics of articles and peer review reports in Social Sciences and Medical and Health Sciences: analysis of articles published in Open Research Central», Scientometrics, vol. 128, n.o 8, pp. 4707-4729, ago. 2023, doi: 10.1007/s11192-023-04771-w.
J. Jovanovic y E. Bagheri, «Reviewer assignment problem: A scoping review», 13 de mayo de 2023, arXiv: arXiv:2305.07887. doi: 10.48550/arXiv.2305.07887.
X. Zhao y Y. Zhang, «Reviewer assignment algorithms for peer review automation: A survey», Inf. Process. Manag., vol. 59, n.o 5, p. 103028, sep. 2022, doi: 10.1016/j.ipm.2022.103028.
C. Cousins, J. Payan, y Y. Zick, «Into the Unknown: Assigning Reviewers to Papers with Uncertain Affinities», arXiv.org. Accedido: 4 de mayo de 2026. [En línea]. Disponible en: https://arxiv.org/abs/2301.10816v2
D. Mimno y A. McCallum, «Expertise modeling for matching papers with reviewers», en Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining, en KDD ’07. New York, NY, USA: Association for Computing Machinery, ago. 2007, pp. 500-509. doi: 10.1145/1281192.1281247.
H. Bauchner y F. P. Rivara, «Use of artificial intelligence and the future of peer review», Health Aff. Sch., vol. 2, n.o 5, p. qxae058, may 2024, doi: 10.1093/haschl/qxae058.
I. Boukhris y C. Zaâbi, «A GAN-BERT based decision making approach in peer review», Soc. Netw. Anal. Min., vol. 14, n.o 1, p. 107, may 2024, doi: 10.1007/s13278-024-01269-y.
S. M. Kadri, N. Dorri, M. Osaiweran, P. Garyali, y M. Petkovic, «Scientific Peer Review in an Era of Artificial Intelligence», en Scientific Publishing Ecosystem, Springer, Singapore, 2024, pp. 397-413. doi: 10.1007/978-981-97-4060-4_23.
T. Ghosal, S. Kumar, P. K. Bharti, y A. Ekbal, «Peer review analyze: A novel benchmark resource for computational analysis of peer reviews», PLOS ONE, vol. 17, n.o 1, p. e0259238, ene. 2022, doi: 10.1371/journal.pone.0259238.
A. Barnett, L. Allen, A. Aldcroft, T. L. Lash, y V. McCreanor, «Examining uncertainty in journal peer reviewers’ recommendations: a cross-sectional study», R. Soc. Open Sci., vol. 11, n.o 9, p. 240612, doi: 10.1098/rsos.240612.
W. Yuan, P. Liu, y G. Neubig, «Can We Automate Scientific Reviewing?», 30 de enero de 2021, arXiv: arXiv:2102.00176. doi: 10.48550/arXiv.2102.00176.
R. Verma, K. Shinde, H. Arora, y T. Ghosal, «Attend to Your Review: A Deep Neural Network to Extract Aspects from Peer Reviews», en Neural Information Processing, Springer, Cham, 2021, pp. 761-768. doi: 10.1007/978-3-030-92310-5_88.
C. Candal-Pedreira, J. Rey-Brandariz, L. Varela-Lema, M. Pérez-Ríos, y A. Ruano-Ravina, «Challenges in peer review: how to guarantee the quality and transparency of the editorial process in scientific journals», An. Pediatría Engl. Ed., vol. 99, n.o 1, pp. 54-59, jul. 2023, doi: 10.1016/j.anpede.2023.05.006.
S. P. J. M. (. S. Horbach y W. (. W. Halffman, «The changing forms and expectations of peer review», Res. Integr. Peer Rev., vol. 3, n.o 1, p. 8, sep. 2018, doi: 10.1186/s41073-018-0051-5.
J. P. Tennant et al., «A multi-disciplinary perspective on emergent and future innovations in peer review», 29 de noviembre de 2017, F1000Research. doi: 10.12688/f1000research.12037.3.
M. Salatino y G. Banzato, «Confines históricos del Acceso Abierto latinoamericano», Conoc. Abierto En América Lat. Trayectoria Desafíos Ciudad Autónoma B. Aires CLACSO Univ. Autónoma Estado México 2021 P 79-115, 2021, Accedido: 19 de septiembre de 2025. [En línea]. Disponible en: https://www.memoria.fahce.unlp.edu.ar/library?a=d&c=libros&d=Jpm4982
M. Majadly y M. Last, «Leveraging peer-review aspects for extractive and abstractive summarization of scientific articles», Int. J. Data Sci. Anal., vol. 19, n.o 3, pp. 537-550, abr. 2025, doi: 10.1007/s41060-024-00665-z.
A. Checco, L. Bracciale, P. Loreti, S. Pinfield, y G. Bianchi, «AI-assisted peer review», Humanit. Soc. Sci. Commun., vol. 8, n.o 1, p. 25, ene. 2021, doi: 10.1057/s41599-020-00703-8.
S. Azad et al., «A Reviewer Recommender System for Scientific Articles Using a New Similarity Threshold Discovery Technique», en Proceedings of the Fourth International Conference on Trends in Computational and Cognitive Engineering, Springer, Singapore, 2023, pp. 503-518. doi: 10.1007/978-981-19-9483-8_42.
J. Devlin, M.-W. Chang, K. Lee, y K. Toutanova, «BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding», presentado en Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), jun. 2019, pp. 4171-4186. doi: 10.18653/v1/N19-1423.
M. Kubát y S. Matwin, «Addressing the Curse of Imbalanced Training Sets: One-Sided Selection», presentado en International Conference on Machine Learning, 1997. Accedido: 21 de noviembre de 2025. [En línea]. Disponible en: https://www.semanticscholar.org/paper/Addressing-the-Curse-of-Imbalanced-Training-Sets%3A-Kub%C3%A1t-Matwin/ebc3914181d76c817f0e35f788b7c4c0f80abb07
N. V. Chawla, K. W. Bowyer, L. O. Hall, y W. P. Kegelmeyer, «SMOTE: Synthetic Minority Over-sampling Technique», J. Artif. Intell. Res., vol. 16, pp. 321-357, jun. 2002, doi: 10.1613/jair.953.
T. Wongvorachan, S. He, y O. Bulut, «A Comparison of Undersampling, Oversampling, and SMOTE Methods for Dealing with Imbalanced Classification in Educational Data Mining», Information, vol. 14, n.o 1, p. 54, ene. 2023, doi: 10.3390/info14010054.
L. Xu, M. Skoularidou, A. Cuesta-Infante, y K. Veeramachaneni, «Modeling Tabular data using Conditional GAN», 28 de octubre de 2019, arXiv: arXiv:1907.00503. doi: 10.48550/arXiv.1907.00503.
M. E. S.-G. González-Pérez Pedro P., «Addressing the class imbalance in tabular datasets from a generative adversarial network approach in supervised machine learning - Máximo E Sánchez-Gutiérrez, Pedro P González-Pérez, 2023», J. Algorithms Comput. Technol., nov. 2023, Accedido: 5 de mayo de 2026. [En línea]. Disponible en: https://journals.sagepub.com/doi/10.1177/17483026231215186
J. M. Johnson y T. M. Khoshgoftaar, «Survey on deep learning with class imbalance», J. Big Data, vol. 6, n.o 1, p. 27, mar. 2019, doi: 10.1186/s40537-019-0192-5.
G. Kovács, «An empirical comparison and evaluation of minority oversampling techniques on a large number of imbalanced datasets», Appl. Soft Comput., vol. 83, p. 105662, oct. 2019, doi: 10.1016/j.asoc.2019.105662.
A. C. Ribeiro, A. Sizo, H. Lopes Cardoso, y L. P. Reis, «Acceptance Decision Prediction in Peer-Review Through Sentiment Analysis», en Progress in Artificial Intelligence, Springer, Cham, 2021, pp. 766-777. doi: 10.1007/978-3-030-86230-5_60.
P. K. Bharti, T. Ghosal, M. Agrawal, y A. Ekbal, «How Confident Was Your Reviewer? Estimating Reviewer Confidence from Peer Review Texts», en Document Analysis Systems, Springer, Cham, 2022, pp. 126-139. doi: 10.1007/978-3-031-06555-2_9.
A. Kumar, T. Ghosal, y A. Ekbal, «A Deep Neural Architecture for Decision-Aware Meta-Review Generation», en 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL), sep. 2021, pp. 222-225. doi: 10.1109/JCDL52503.2021.00064.
A. Kumar, T. Ghosal, S. Bhattacharjee, y A. Ekbal, «Towards automated meta-review generation via an NLP/ML pipeline in different stages of the scholarly peer review process», Int. J. Digit. Libr., vol. 25, n.o 3, pp. 493-504, sep. 2024, doi: 10.1007/s00799-023-00359-0.
P. K. Bharti, M. Navlakha, M. Agarwal, y A. Ekbal, «PolitePEER: does peer review hurt? A dataset to gauge politeness intensity in the peer reviews», Lang. Resour. Eval., vol. 58, n.o 4, pp. 1291-1313, dic. 2024, doi: 10.1007/s10579-023-09662-3.
A. Kumar, T. Ghosal, S. Bhattacharjee, y A. Ekbal, «Investigations on Meta Review Generation from Peer Review Texts Leveraging Relevant Sub-tasks in the Peer Review Pipeline», Lect. Notes Comput. Sci. Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinforma., vol. 13541 LNCS, pp. 216-229, 2022, doi: 10.1007/978-3-031-16802-4_17.
C.-H. Lai y P.-Y. Peng, «A Hybrid Deep Learning Method to Extract Multi-features from Reviews and User–Item Relations for Rating Prediction», Int. J. Comput. Intell. Syst., vol. 16, n.o 1, p. 109, jun. 2023, doi: 10.1007/s44196-023-00288-5.
V. Lakshmanan, S. Robinson, y M. Munn, «Machine Learning Design Patterns».
«Microsoft Word - ANEXO 7. AREAS OCDE». Accedido: 21 de noviembre de 2025. [En línea]. Disponible en: https://minciencias.gov.co/sites/default/files/upload/convocatoria/anexo_7._areas_ocde_0.pdf
«Convocatoria de Clasificación y Reconocimiento de Revistas Científicas Nacionales - Publindex 2026», Minciencias. Accedido: 8 de julio de 2026. [En línea]. Disponible en: https://minciencias.gov.co/convocatorias/convocatoria-clasificacion-y-reconocimiento-revistas-cientificas-nacionales-publindex
H. He y E. A. Garcia, «Learning from Imbalanced Data», IEEE Trans. Knowl. Data Eng., vol. 21, n.o 9, pp. 1263-1284, sep. 2009, doi: 10.1109/TKDE.2008.239.
R. Kohavi, «A study of cross-validation and bootstrap for accuracy estimation and model selection», en Proceedings of the 14th international joint conference on Artificial intelligence - Volume 2, en IJCAI’95. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., ago. 1995, pp. 1137-1143.
J. Dodge, G. Ilharco, R. Schwartz, A. Farhadi, H. Hajishirzi, y N. Smith, «Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping», arXiv.org. Accedido: 7 de mayo de 2026. [En línea]. Disponible en: https://arxiv.org/abs/2002.06305v1
M. Sokolova y G. Lapalme, «A systematic analysis of performance measures for classification tasks», Inf. Process. Manag., vol. 45, n.o 4, pp. 427-437, jul. 2009, doi: 10.1016/j.ipm.2009.03.002.
G. C. Cawley y N. L. C. Talbot, «On Over-fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation», J Mach Learn Res, vol. 11, pp. 2079-2107, ago. 2010.
J. T. Hancock y T. M. Khoshgoftaar, «CatBoost for big data: an interdisciplinary review», J. Big Data, vol. 7, n.o 1, p. 94, nov. 2020, doi: 10.1186/s40537-020-00369-8.
L. Yang y A. Shami, «On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice», arXiv.org. Accedido: 7 de mayo de 2026. [En línea]. Disponible en: https://arxiv.org/abs/2007.15745v3
L. Breiman, «Random Forests», Mach. Learn., vol. 45, n.o 1, pp. 5-32, oct. 2001, doi: 10.1023/A:1010933404324.
R. Gopalan, D. Onniyil, G. Viswanathan, y G. Samdani, «Hybrid models combining explainable AI and traditional machine learning: A review of methods and applications», World J. Adv. Eng. Technol. Sci., vol. 15, pp. 1388-1402, may 2025, doi: 10.30574/wjaets.2025.15.2.0635.
R. Blagus y L. Lusa, «SMOTE for high-dimensional class-imbalanced data», BMC Bioinformatics, vol. 14, n.o 1, p. 106, mar. 2013, doi: 10.1186/1471-2105-14-106.
D. H. Wolpert, «La falta de distinciones a priori entre algoritmos de aprendizaje | Computación neuronal | MIT Press», vol. 8, n.o 7, Neural Computation, 1 de octubre de 1996. Accedido: 8 de mayo de 2026. [En línea]. Disponible en: https://direct.mit.edu/neco/article-abstract/8/7/1341/6016/The-Lack-of-A-Priori-Distinctions-Between-Learning?redirectedFrom=fulltext
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Liseth Paola Claro Ascanio, Cristian Camilo Tirado Cifuentes
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
