Calibration

Formsieve's thresholds are set for jev-1.13.0. This page explains what that means, how each route reports the Jev version that answered, and what Formsieve does when a route answers with a version it does not know. Last checked: 2026-09-22.

Calibrated probabilities and calibrated thresholds

Jev returns calibrated probabilities rather than a bare yes or no. TypeSafe notes that calibration "is measured across groups of predictions; it does not guarantee that an individual answer is correct", and recommends pinning the version you tuned your thresholds on. A probability is only comparable with a threshold for the model version the threshold was chosen on. A new Jev version can shift scores, so Formsieve records the model name every response reports and checks it against a list of versions its presets were set for (the calibration identities):

A Jev version is added to this list only in a Formsieve update, after our evaluation set has been run on it.

Route by route

For the most stable behaviour we recommend TypeSafe direct first and OpenRouter second. Vercel works, but it does not report which Jev version answered. As of 2026-09-22 every route serves jev-1.13.0 (TypeSafe has no other build available), so Vercel users get the same model today; a later change on Vercel's side cannot be seen from the responses.

Test connection shows the status for your route, and the dashboard shows the status of recent checks.

When a route answers with an unknown version

If a response reports a model that is not on the list (for example a future jev-1.14.0), the status becomes Unrecognized. Formsieve then:

• keeps filtering with your current thresholds;
• shows a notice on its own screens saying that the API answered with a model version it was not calibrated on;
• lists every unrecognised model name under Settings → Advanced → Unrecognized model versions;
• records the model and its status on every log row, and in the Formsieve note of each entry that gets one ("Model … (unrecognized) …").

The notice stays until you click Hide until another new model appears (in the notice, or Hide the notice until another new version appears under Settings → Advanced); it comes back when yet another unrecognised model answers. A later Formsieve update that adds the version to its calibrated list does not remove the notice or the list by itself.

If you prefer caution, switch on Strict calibration (Settings → Advanced): while the model is unrecognised, submissions that would have been blocked go to the review band instead, so they are delivered and tagged rather than marked as spam. It has no effect on Vercel, which does not report the version.

The model can be chosen only on the Custom route (Model ID, default jev-1.13.0); the other routes always send their fixed model shown above. Entering an alias such as jev-latest there gives up the pin; expect "Unrecognized" as soon as the alias moves to a new version.

How the presets were chosen

The presets (review / block: Low 0.65 / 0.92, Medium 0.50 / 0.85, High 0.40 / 0.75) are starting points taken from TypeSafe's guidance and from public evaluation data for jev-1.13.0, which is English e-mail rather than form submissions. Formsieve's own evaluation set adds contact-form spam, legitimate enquiries that look commercial, very short messages, prompt-injection attempts and a small Spanish sample; it supports no accuracy claim until it has been run on a live route.

Before a release, the Medium block threshold must pass a certification on a frozen set of real, legitimate contact-form messages that is never used for tuning: it passes only if the one-sided 95% upper confidence bound on the false-positive rate (Clopper-Pearson) is at most 1.5%, which needs at least 315 legitimate messages. Prompt-injection examples must never be allowed, and the "hard" legitimate examples must never be blocked. The evaluation is to be re-run for every new Jev version and every change to the questions Formsieve asks. Results for version 1.0.0 are not yet available (see below).

Results for version 1.0.0

The certification results for this release will be published on this page. Formsieve makes no accuracy claim that has not been measured this way.

Non-English sites

When your first site language is not English, both thresholds move up by 0.05. TypeSafe says Jev handles other languages "but not equally well"; the shift is a precaution, not a measured correction, and costs some spam recall. Formsieve's interface is available in English and Spanish (Spain) and is translation-ready, but the translation is not an accuracy claim: we make no claim about filtering accuracy in any language other than English until it has been measured.