GitHub - sutro-sh/jev-align: Build calibrated AI classifiers from human feedback using Jev and GEPA.
Build calibrated AI classifiers from human feedback using Jev and GEPA. - sutro-sh/jev-align
Welcome! Type "help" for available commands.
$
Summary
A coding agent helped build a SetFit classifier using a 149M ModernBERT encoder to label document purpose across six categories in the FinePDFs-Edu dataset.
Trained on 200 agent-labelled examples, it reached 65.8% accuracy on 120 held-out documents. Hugging Face Jobs classified 191,724 documents in 42 minutes for about $0.70, versus an estimated $13–26 for LLM API labelling of the same excerpts.
