llama-dash — self-hosted inference gateway
One UI for model state, request history, API keys, routing rules, and proxy metrics — fronting llama-swap and any OpenAI- or Anthropic-compatible upst...
AI Infrastructure Platform
Summary
Respan is an all-in-one engineering platform designed to unify LLM gateway management, evaluation, and observability. It provides tools to route traffic across various models, test output quality with automated evaluators, and monitor real-time metrics like cost and latency.
Highlights
- Unified gateway supporting 1,000+ models with automatic fallbacks and response caching
- Comprehensive evaluation framework using LLM judges, code checks, and human reviewers
- Real-time monitoring and alerting for cost, latency, tokens, and error rates
- Granular budget and rate-limiting controls to manage spend across organizations or customers
auto-generated
Context
Audience
AI Engineers, Machine Learning Engineers, and Software Developers building LLM-powered applications
DomainMachine Learning
FormatSaaS platform
AccessSubscription required
Source Overview
Respan is an LLM platform combining a unified gateway for 1,000+ models, output evaluation, cost and latency monitoring, and request tracing.
The gateway includes automatic model fallbacks, response caching, and per-key or org-wide spend limits. The platform holds ISO 27001, SOC 2, GDPR, and HIPAA compliance certifications.
Topics
Related
LLM ObservabilityMLOpsPrompt EngineeringAI GatewayModel Evaluation
