Mo Moustafa reports OpenRouter operational issues from running an iMessage AI assistant over 18 million messages on open-source models.

The same model weights show 20-point benchmark swings across ~20 hosts, some vision providers silently drop images while returning 200 OK, and reasoning.effort is ignored by select providers. Quantization labels (fp4 vs fp8) do not predict actual output quality.