Quantiphi developed LLM-driven insights to monitor and manage IT operations, enhancing incident prioritization and resolution for a major banking client.

Read how Quantiphi’s solution achieved 90% accuracy in the SQL agent pipeline and led to a 5x improvement in IT incident prioritization and resolution.

About the Client

The client is a leading multinational bank in the financial sector, managing extensive IT operations and data critical to its business functions. Recognized for driving innovation in banking, the client continuously explores cutting-edge technologies like generative AI to optimize operations and enhance efficiency.

Problem Statement

The client struggled with critical incident prioritization and handling complex queries with their existing search system. Traditional machine learning approaches were faltering, leading to inefficiencies in IT operations management.

Challenges

Accuracy Issues: The risk-scoring models were insufficiently accurate to prioritize critical incidents effectively.
Complex Query Management: The IT marketplace data and search systems struggled with high complexity queries.
Outdated Models: Legacy machine learning methods were ill-equipped for the dynamic and voluminous IT data landscape.

The Solution

Quantiphi deployed a Generative AI-powered solution aimed at revolutionizing IT operations management and Role-Based Access Control (RBAC) security analysis leveraging advanced LLMs and NVIDIA technologies to address the client’s challenges.

Key Features:

LLM-Powered Agentic Chatbot:
- Extracts insights from structured IT logs using SQL agent technology.
- Responds to user inquiries with natural language precision.
Knowledge Graphs:
- Extracts insights from unstructured IT marketplace data for enhanced query resolution.
Fine-Tuned NLP Models:
- Improves severity classification and intent matching accuracy.
Real-Time Insights:
- Uses Retrieval-Augmented Generation (RAG) for actionable, protocol-compliant monitoring.
Optimized for Performance:
- Powered by NVIDIA’s NeMo framework and fine-tuned versions of LLaMA 2-7B and SQLCoder 7B, enhancing model accuracy and operational efficiency.

Technology Stack

NVIDIA NeMo Framework: Enables rapid development and deployment of custom generative AI models.
NVIDIA Inference Model Server (NIM): Supports scalable and efficient inference for real-time operations.

Results and Impact Created

Quantiphi’s solution delivered transformative results for the client’s IT operations:

5x Improvement: Streamlined IT incident prioritization and resolution.
90% Accuracy: Achieved for the SQL agent pipeline.
Enhanced Operational Efficiency: Significantly reduced computational costs and improved resource utilization.
Scalability: Supported large-scale IT operations with consistent performance, ensuring readiness for future data growth.
Improved User Satisfaction: Enhanced system performance and responsiveness, leading to better end-user experiences.