GenAI-Based On-Device Support
The customer’s vision was to ensure a low downtime for the mission critical devices they sell. The idea was to build a support capability within the device itself to cut through the support process.
Alumnus did the design thinking and implementation.
The Support Agent is a 100% local, GenAI-driven solution – an AI powerhouse residing directly on the client’s rugged devices, delivering instant, intelligent assistance. This capability will eventually transform into a solution engineered for the frontline, purpose-built for harsh and remote industrial environments.
Key
Innovations
Uninterrupted Operations
This fully autonomous, GenAI-driven support agent can work in remote locations, factory floors, and where enterprise-confidential information must remain secure.
Instant Expertise
Users get real-time troubleshooting and contextual help without waiting for IT support or searching through manuals.
The Power of Two-Stage Edge AI
We architected a sophisticated, fast, two-stage AI strategy to deliver the results:
- Stage One – Intuitive Query Handling: An on-device Small Language Model (SLM), leveraging the power of Sentence BERT or DistilBERT, instantly understands user intent, matching queries against a comprehensive local FAQ database with astonishing speed.
- Stage Two – Deep Query Resolution: For complex issues, a highly optimized, quantized LLM (like Llama 3.2 - 3B) kicks in, employing Retrieval-Augmented Generation (RAG) to pull precise, actionable insights from a secure, internal vector store – all without ever touching the cloud.
Performance Beyond Expectation
Optimized for responsive performance on constrained hardware platforms, the solution is a great step towards rediscovering the support paradigm through on-device support.
Tech Stack Highlights
DistilBERT and SentenceBERT for intelligent inferencing, Annoy for ultra-fast vector similarity search, SQLite for robust knowledge storage, and a seamless Android API layer for deep device integration.
