llama-dash — self-hosted inference gateway
One UI for model state, request history, API keys, routing rules, and proxy metrics — fronting llama-swap and any OpenAI- or Anthropic-compatible upst...
AI Infrastructure Platform
Summary
Respan is an all-in-one engineering platform designed to unify LLM gateway management, evaluation, and observability. It provides tools to route traffic across various models, test output quality with automated evaluators, and monitor real-time metrics like cost and latency.
Highlights
- Unified gateway supporting 1,000+ models with automatic fallbacks and response caching
- Comprehensive evaluation framework using LLM judges, code checks, and human reviewers
- Real-time monitoring and alerting for cost, latency, tokens, and error rates
- Granular budget and rate-limiting controls to manage spend across organizations or customers
auto-generated
Context
Audience
AI Engineers, Machine Learning Engineers, and Software Developers building LLM-powered applications
DomainMachine Learning
FormatSaaS platform
AccessSubscription required
Topics
Related
LLM ObservabilityMLOpsPrompt EngineeringAI GatewayModel Evaluation
