Nokia’s Free Jev Rival Has a Catch

Generative Large Language Models (LLMs) produce text token by token, a creative but often unpredictable process. Decision models, in contrast, return structured outputs: probabilities for typed choices, yes/no answers, or numerical scores. This distinction is critical for applications demanding deterministic, parse-free judgments over freeform text generation.

Nokia Applied Research addresses this need with AnyJev, an Apache 2.0 Python library. AnyJev adds a decision layer to open-weight models without requiring fine-tuning or labeled training data. It transforms a generative LLM into a "System One" decision engine, enabling rapid, structured evaluations.

AnyJev directly competes with TypeSafe AI’s Jev, a hosted, closed-source API. Jev processes approximately 310 tokens per call, costing $0.042 per million input tokens. AnyJev, however, runs locally on your infrastructure with zero per-call cost, leveraging your existing compute.

AnyJev supports a range of popular local inference backends:

• Hugging Face Transformers
• vLLM
• SGLang
• llama.cpp

This flexibility allows developers to integrate robust decision-making capabilities directly into their self-hosted model deployments, bypassing external API dependencies and their associated costs and latencies.

AnyJev’s core innovation lies in prompt engineering, not model training. It presents a state (the input text) and a multiple-choice question to an existing LLM, formatting options with distinct labels like "A" or "B." Instead of allowing the model to generate a natural-language answer, AnyJev directly queries the model’s next-token probabilities for these specific option labels. This directly retrieves a probability distribution across the choices.

This method completely bypasses the risks of malformed text or JSON parsing failures inherent in generative LLM outputs. A model cannot hallucinate or misformat a single character probability. However, a cleanly formatted probability is not inherently correct or trustworthy; it simply reflects the model’s raw, uncorrected judgment.

To address these inherent biases, AnyJev introduces L0 calibration. L0 tackles position bias by testing every possible ordering of options and averaging the results, ensuring no choice gains an unfair advantage from its placement. It also corrects for label bias by estimating and adjusting the model’s prior preference for certain labels, regardless of content.

L0 offers two prior estimation methods: batch prior, which analyzes answers across a batch of inputs, and content-free prior, which estimates bias from an input devoid of actual content. This second method is crucial for single-headline judgments, providing a more robust bias correction when batch data is unavailable. L0 significantly improves decision reliability without requiring any labeled training data.

Nokia's AnyJev library faced a direct challenge against Jev 1.13 and a raw Qwen 8B model. The benchmark comprised 500 headlines, evenly split between satirical content from The Onion and real news, all processed on an M2 Max MacBook Pro. This test rigorously evaluated accuracy, speed, and confidence calibration.

Jev achieved an 80% accuracy rate. Raw Qwen trailed significantly at 68.4%, with AnyJev L0 performing only marginally better at 68.6%. Jev also dominated in speed, completing all 500 evaluations in 51 seconds, while the local Qwen and AnyJev L0 combined took 4 minutes and 36 seconds, even with shared GPU resources.

AnyJev L0’s core strength emerged in bias correction, eliminating all answer flips when option order changed. Raw Qwen, in contrast, flipped answers on 20 headlines (4%), while Jev saw 18 minor shifts. For further technical details, explore the nokia-applied-research/AnyJev: Turn any LLM into a Jev-style decision model GitHub repository.

Confidence metrics revealed a critical flaw in raw Qwen. While 87% of its answers boasted ≥99% confidence, it was incorrect on 30% of those highly confident predictions. AnyJev L0 mitigated positional and label biases but did not improve calibration in this specific comparison, leaving Jev as the superior choice for calibrated confidence.

Headline trials revealed AnyJev’s catch. Qwen, with its strong bias toward satire, often misclassified real news that sounded like The Onion. AnyJev’s default L0 calibration, using batch prior, barely corrected Qwen’s extreme 0% or 100% probabilities.

Applying content-free prior to L0, however, showed a different outcome. This stronger correction helped Qwen recognize real stories that sounded satirical, reducing its lean from 69% satire calls (on a 50/50 split) to a more balanced 67% satire and 66% real news. But this also misclassified some genuine Onion headlines.

Correcting a model’s label lean helps when that lean is harmful, but can overshoot when the mix of examples changes. The method does not make the underlying model smarter; it merely adjusts its output probabilities.

AnyJev is compelling for private, self-hosted decisions and low API costs due to its Apache 2.0 license and local execution. Yet, benchmark AnyJev on your own representative labeled examples before production. The video’s results—80% accuracy for Jev versus 68.6% for AnyJev’s L0—are specific to one model, task, and test setup, not a universal verdict.

What is AnyJev?

AnyJev is an open-source Python library from Nokia Applied Research that adds decision-style outputs to open-weight language models.

How does AnyJev make a decision?

It checks the model’s next-token probabilities for answer labels such as A or B, then returns probabilities instead of generated text.

Did AnyJev outperform Jev in the video’s benchmark?

No. On 500 headlines, Jev scored 80% accuracy, compared with 68.6% for Qwen with AnyJev’s default L0 correction.

Is AnyJev free to use?

The library is free under the Apache 2.0 license. Running it locally still requires compute resources and a compatible open-weight model.