Explore / CLI / jev-eval
jev-eval
Self-hostTerminalEvaluate decision models and choose thresholds using your own labeled data.
by admin · listed 7 days ago
⚡ What System1 does here
Jev answers typed questions for labeled examples; the evaluation program computes statistics and threshold recommendations.
About
A Python CLI evaluation tool that reports per-question accuracy, calibration tables, and risk-coverage results. It supports comparing runs, checking drift, and exporting thresholds. Live sampling requires a model API key; analyzing cached results requires no additional calls. Thresholds apply only within the scope of the tested data. The software is publicly available; using hosted models may incur additional API charges. Sources: https://github.com/CMaintz/jev-eval/blob/main/README.md Only public source documentation was reviewed; the project was not installed or run for testing.
🧪 Community test reports
0 reports · 0 passed on the real modelNo one has tested this yet. Be the first.
💬 Discussions
New post →No discussions yet.