From documentations to intelligence: Retrieval-augmented troubleshooting for telecommunications software

Abstract

Managing complex telecommunications software requires navigating technical documentations, often leading to inefficiencies in troubleshooting and deployment. This thesis presents the Retrieval-Augmented Troubleshooting Assistant (RATA) designed to automate technical support by integrating Large Language Models (LLMs) with domain-specific documentations of a modern telecommunication software product, Lightweight real-Time Monitoring and Troubleshooting (LTMT). The research implements a privacy-compliant application that transforms XML-based documentations into semantic vector indices.

Various architectural configurations are evaluated and verified, specifically contrasting single-stage versus multi-stage hybrid retrieval and fixed-length versus syntax-aware chunking strategies. Evaluation results from a curated golden dataset of LTMT documentations demonstrate that a multi-stage retrieval approach combined with syntax-based chunking yields superior performance, achieving an average Recall@15 score of nearly 98%. Furthermore, the integration with Llama3.3-70B model proved most effective for intent recognition and structured JSON generation when utilizing n-shot (n >= 2) prompting for LTMT-related questions. These findings validate that specific architectural optimizations are essential for deploying reliable AI assistants in telecommunications environment.