Self-Hosted LLM for Customer Support: Llama 3 & Mistral in Production

For hosting providers with strict data sovereignty requirements, self-hosted LLMs offer complete control over customer data. Run Llama 3, Mistral, or other open-source models on your own infrastructure with DeskNexo’s OpenAI-compatible API support.

Why self-host LLMs for support?

External LLM providers (OpenAI, Anthropic, Google) process your data on their servers. Even with DPAs and PII masking, some hosting providers - especially those serving regulated industries like healthcare, finance, or government - need complete data sovereignty. Self-hosted LLMs solve this: all data stays on your infrastructure.

Supported models

DeskNexo supports any LLM that exposes an OpenAI-compatible API endpoint. The most popular choices for hosting support are:

• Llama 3 (70B/8B): Meta’s open-source model, excellent for technical support
• Mistral (7B/8x7B): Fast inference, good balance of quality and speed
• Qwen 2.5: Strong multilingual support for international providers
• DeepSeek Coder: Specialized for technical/code-related support queries
• Phi-3: Microsoft’s small-but-capable model for resource-constrained setups

Infrastructure requirements

Running a production LLM requires GPU hardware. For a 70B parameter model, you need at least 2x NVIDIA A100 80GB or equivalent. Smaller 7-8B models run on a single consumer GPU (RTX 4090) or even CPU-only (with slower inference). Most hosting providers start with the 8B model and scale up as needed.

Setup with DeskNexo

In the DeskNexo control panel (Enterprise plan), navigate to Settings > LLM Configuration and select “Self-hosted.” Enter your API endpoint URL and API key. The agent will use your self-hosted model for all ticket processing, with the same dual-LLM verification pipeline as the managed service.