RETRIEVAL-AUGMENTED TEST AUTOMATION USING LARGE LANGUAGE MODELS AND SOFTWARE REQUIREMENT KNOWLEDGE

Abstract

Software testing is a critical phase of the Software Development Life Cycle (SDLC) that ensures software reliability, functionality, security, and overall quality before deployment. As software systems become increasingly complex and requirements evolve rapidly within Agile and DevOps environments, conventional manual testing and rule-based automated testing techniques often struggle to generate accurate, maintainable, and comprehensive test cases. Recent advances in Artificial Intelligence (AI), Natural Language Processing (NLP), and Large Language Models (LLMs) have introduced new opportunities for intelligent software test automation by enabling automated understanding of software requirements, generation of test scenarios, and adaptive test execution. However, standalone LLMs frequently suffer from hallucination, outdated knowledge, limited project-specific context, and inconsistent interpretation of software requirements, reducing their reliability for industrial software testing. Retrieval-Augmented Generation (RAG) has emerged as an effective solution by combining the reasoning capabilities of LLMs with external knowledge retrieval mechanisms, allowing models to access accurate and project-specific requirement information during test generation. This research proposes a RetrievalAugmented Test Automation framework that integrates Large Language Models with software requirement knowledge repositories to generate intelligent, accurate, and contextaware software test cases. The proposed framework combines Software Requirement Specifications (SRS), user stories, functional requirements, use case documents, API documentation, design specifications, historical defect reports, test repositories, and domain knowledge into a centralized knowledge base indexed using vector embeddings. User queries and testing requests are transformed into semantic embeddings and matched with the most relevant requirement documents using similarity search techniques. The retrieved contextual information is then supplied to the Large Language Model, enabling contextaware generation of functional test cases, regression tests, API tests, boundary value tests, negative test cases, security validation scenarios, and acceptance testing procedures. The proposed framework employs RetrievalAugmented Generation (RAG), transformerbased embedding models, vector databases, semantic search, prompt engineering, and explainable artificial intelligence techniques to improve the quality, consistency, and traceability of generated software tests. The system automatically establishes links between software requirements, generated test cases, implementation artifacts, and verification results, thereby improving requirement coverage, reducing redundant testing effort, and enhancing software quality assurance. Continuous integration with Agile development pipelines enables automatic updating of the knowledge repository whenever software requirements evolve, ensuring that generated test cases remain synchronized with changing project specifications. Experimental evaluation demonstrates that the proposed Retrieval-Augmented Test Automation framework significantly improves requirement coverage, test generation accuracy, context relevance, defect detection capability, and overall testing efficiency compared with conventional LLM-based test generation approaches. The integration of requirement retrieval substantially reduces hallucinations, increases consistency, improves explainability, and enhances user confidence in AI-generated testing artifacts. Furthermore, the framework supports continuous software testing, intelligent regression testing, automated documentation, and scalable quality assurance within modern DevOps and Industry 5.0 software engineering environments. The research contributes to the development of trustworthy, context-aware, and intelligent AI-assisted software testing systems capable of accelerating software delivery while maintaining high software quality and traceability.