Comparing 6 Open-Source Jev Clones: Architecture, State Sharing, and Scoring

Jev is a decision model provided as an API by TypeSafe. Given a document text alongside candidate choices like billing, technical support, or other, it returns the chosen category together with probabilities for each option. It does not generate conversational responses.

Latent.Space’s “Here are 6 Clones of Jev in 2 days” highlights six projects implementing similar decision functionality. Some rely on ModernBERT for classification, others adapt generative Qwen models, and one repurposes output slots in diffusion models.
The article suggests ModernBERT and diffusion models as plausible candidates for Jev’s internal design. However, the six projects span drop-in replacements, implementations of hypothesized architectures, and lightweight experimental models, each built with different goals.

Using official documentation and public code available as of September 20, 2026, we compared which aspects of Jev’s observed behavior each architecture can explain. Performance figures cite measurements reported by each author.

What TypeSafe Has Disclosed About Jev

Jev takes a document text as state alongside queries and choices in questions. To inspect a single support ticket for department routing, urgency, and refund eligibility, a caller attaches three questions to the same body text.

The official documentation specifies three decision types: Choice for selecting from options, Score for scalar ratings, and Noul for boolean probabilities. It also states that multiple questions on the same document are evaluated in parallel while remaining isolated from one another.

Property Compared	Officially Disclosed	Not Determined from Disclosures
Output	Returns decisions and calibrated probabilities without token-by-token generation	Whether it is BERT-based or a repurposed generative backbone
Multiple Questions	Evaluates questions independently on the same document	Which layer shares the state computation
Training	Uses RLCD (Reinforcement Learning for Calibrated Decisions) to calibrate output probabilities	Specific loss functions, reward formulation, or base checkpoint
Architecture	Discloses a novel architecture with a parallel sampler	Layer configuration, parameter scale, and model weights

RLCD stands for Reinforcement Learning for Calibrated Decisions. The official machine learning primer defines calibration as the core training target. When a model predicts 80% confidence across many instances, roughly 80% of those predictions should prove correct.
Open-source clones can target the same calibration objective, but TypeSafe has not published its specific training pipeline.

The statement in the launch announcement that Jev produces probabilities in parallel also depends on the unit of computation. In the same blog’s Wikipedia navigation demo, requests with many candidates use a two-stage approach of individual scoring followed by selection. The post does not state that an entire API call finishes in a single computational pass under all configurations.

In our earlier test on Jev’s style judgment format, we examined output shifts when altering line breaks, Markdown syntax, and prompt wording. The sections below compare how clone architectures ingest and score those inputs.

Architectural Approaches Across 6 Implementations

Implementations differ in how they handle multiple questions: some recompute the document text for each query, while others share the state computation. They also diverge on whether questions can access one another’s context and how candidates are scored.

The output module that maps hidden representations to decision scores is called a head. Across the six projects, designs differ in both the backbone model and the extraction mechanism for these scores.

Implementation	Backbone & Scoring Method	Behaviors Consistent with Jev	Architectural Gaps
Laya	ModernBERT backbone + candidate scoring head	Evaluates arbitrary candidates per call without text generation	Builds separate inputs per question without sharing state computation
Kev-0.5B	Qwen2.5-0.5B + custom decision head	Implements shared state, question isolation, and candidate comparison	No evidence of matching Jev’s base model, training data, or recipe
DiffusionGemma mod	Scores fixed output token slots in a diffusion model	Evaluates multiple answer slots in parallel	Slot parallelism by itself does not isolate questions or calibrate probabilities
Bespoke Nimble	Qwen3.5-9B + answer token logit extraction	Repurposes an LLM for classification without autoregressive loops	Mac shared prefix includes all question schemas, allowing cross-question attention
SemIf (formerly OpenJev)	Qwen3.5 + 3-way NLI classification	Directly checks whether the document entails each candidate claim	Entailment score reflects candidate-level truth, not a normalized multi-choice distribution
Jevlike	Lightweight embeddings + independent candidate scorer	Implements a variable-candidate classifier with minimal overhead	Baseline scorer evaluates candidates without cross-candidate attention

Laya: Independent Inputs per Question

The English checkpoint of Laya uses ModernBERT-large. As a bidirectional encoder, each token position attends to context both before and after it.

Laya takes the question text and candidate choices alongside the document. In its model definition, two Transformer layers follow the encoder by default, extracting decision scores from special marker tokens placed before each candidate.
Callers can supply arbitrary candidate sets at inference time, supporting flexible category labels per query.

Its handling of multiple questions appears in its inference code. Laya constructs separate input sequences containing the document for each question and batches them together. As a result, adding questions increases document computation proportionally.

flowchart TD
    S[Same document state] --> I1[State + Question 1 and choices]
    S --> I2[State + Question 2 and choices]
    I1 --> B[Batched through model<br/>as separate inputs]
    I2 --> B
    B --> O1[Decision for Question 1]
    B --> O2[Decision for Question 2]

Because each batched item includes the document text, the state is encoded independently for each query.

Laya computes confidence using the entropy of the predicted candidate probabilities.
When probabilities concentrate on a single option, confidence is high. Checking whether that confidence matches real-world accuracy involves testing against labeled datasets with known outcomes.

Kev: Balancing Shared State and Question Isolation via Attention Masks

Kev-0.5B combines Qwen2.5-0.5B with LoRA adapters and a custom decision head. LoRA trains a small set of parameter matrices, updating only a fraction of the total weights.

In its model implementation, the document and multiple questions are concatenated into a single sequence, with an attention mask restricting visibility. Each question attends only to the shared state and its own tokens, preventing information leakage across queries while computing document tokens once.

flowchart TD
    S[Shared state computation] --> Q1[Question 1 + choices<br/>Attends only to state and Q1]
    S --> Q2[Question 2 + choices<br/>Attends only to state and Q2]
    Q1 --> O1[Decision for Question 1]
    Q2 --> O2[Decision for Question 2]

No attention flows between Question 1 and Question 2. Decision positions at the end of each question attend across all choices within that query, allowing candidate representations to inform the scores.

Kev’s architecture reflects findings from Archer Hume’s black-box probing of Jev’s API. Hume tested whether hints injected into one question could influence another, concluding that Jev shares document state while isolating queries.
Kev implements this hypothesis directly. However, black-box probing reveals input-output properties, while Jev’s exact mask layout, base checkpoint, and training data remain unpublished.

DiffusionGemma: Parallel Scoring Across Fixed Output Slots

The DiffusionGemma link in the original article points to a vLLM pull request adding diffusion language model support. As of September 20, 2026, the PR had not merged into main.

Diffusion language models generate text by iteratively refining noise across an output sequence. This proposal preallocates output slots and scores answer positions across questions simultaneously.
Long candidate labels map to single-token identifiers like A or B. Logits at each answer position yield choice probabilities.

The pull request includes an option to truncate denoising to a single step to retrieve scores, bypassing multi-step text generation. However, because prefix encoding runs separately, this single step applies specifically to updating output positions.

This approach demonstrates parallel scoring across multiple output slots. Ensuring question isolation, however, needs explicit attention constraints between slots to prevent leakage.
Additionally, while the PR includes confidence-based iterative refinement, it does not detail an RLCD training recipe for probability calibration.

Nimble: Next-Token Logit Scoring Without Autoregressive Loops

Bespoke Nimble fine-tunes Qwen3.5-9B using LoRA, mapping choices to single-token identifiers to read next-token logits directly. Logits are raw unnormalized scores before probability conversion; applying softmax normalizes them into a distribution summing to 1 across candidates.
The model outputs these raw scores, while surrounding Python code formats the JSON response. This design repurposes a generative LLM for classification without autoregressive generation loops.

The training pipeline uses contrastive sample pairs where altering a single factual detail flips the ground truth label. For instance, modifying the authorized approver determines whether a refund request is valid.
The training documentation relies on supervised labels. The authors state that Nimble is not distilled from Jev’s API responses.

In the parallel scoring documentation, the Mac implementation caches key-value states for a shared prefix containing the document and appends question-specific suffixes. The CUDA implementation for NVIDIA GPUs processes full sequences per item, showing that state recomputation depends on the runtime backend.

On Mac, the shared prefix includes descriptions and choices for all questions alongside the document. Consequently, scoring one item permits attention to information across other queries.
While item answers are evaluated separately, question prompts remain shared. Although both Kev and Nimble share state computation, Nimble does not isolate question contexts from one another.

SemIf: Three-Way Classification via Natural Language Inference

SemIf, introduced in the original article as OpenJev, offers models based on Qwen3.5-4B and 35B-A3B on its model repository. It frames classification as Natural Language Inference (NLI).

The model pairs the document with a candidate statement to classify the relationship as entailment, contradiction, or neutral. If a document states that a user was billed twice, the model evaluates whether the text entails a billing issue candidate.
The implementation passes the final token representation to a small classification head to output scores for all three classes.

Its rerank implementation evaluates document-candidate pairs individually and selects the candidate with the highest entailment probability. Because each probability is normalized within its own three-way NLI distribution, summing entailment scores across candidates can exceed 100%.

For illustration, suppose candidate A has an 80% entailment score and candidate B has 70%. When both claims hold true, their sum naturally exceeds 100%. While rerank compares 80% against 70% to select A, interpreting 80% as the probability of choosing A over all other candidates misrepresents the score.

The current SemIf web demo also supports token-logit scoring. The comparison above focuses on the linked NLI model and repository code, while the demo interface incorporates alternative scoring methods.

Jevlike: A Compact Implementation Scoring Candidates Independently

Jevlike is an experimental implementation that accepts a document and an arbitrary number of choices to output candidate probabilities. By default, it converts text into byte-level embeddings and computes attention over the document for each candidate. It also supports pretrained text backbones.

Its baseline scorer evaluates each candidate against the document in isolation. Holding the document and choices A and B constant while introducing choice C leaves the raw scores for A and B unchanged.

Softmax normalizes probabilities across all candidates to sum to 1, meaning adding choice C reduces the allocated percentages for A and B. Using hypothetical logits illustrates this behavior.

Candidate	Raw Score (Logit)	Probability with A & B Only	Probability after Adding C
A	2	73.11%	24.47%
B	1	26.89%	9.00%
C	3	Not included	66.52%

The unrounded ratio between A and B remains approximately 2.718 in both cases. This independent scoring property stems directly from the implementation. Evaluating relative choices (such as “All of the above” or “None of the other options”) needs attention across candidates.

Accuracy Benchmarks and Architectural Limits

Nimble’s published benchmark reports accuracy across 324 held-out evaluation examples.

Model	Correct Matches	Accuracy
Qwen3.5-9B	215 / 324	66.36%
Bespoke-Nimble-9B	292 / 324	90.12%
Jev 1.13.0	302 / 324	93.21%

Supervised fine-tuning brings Nimble’s benchmark accuracy close to Jev’s. However, the evaluation relies on synthetic data comprising 162 paired contrastive examples drawn from six domains. This test does not indicate performance on long-form Japanese text, novel business workflows, or whether Jev uses a similar backbone.

Evaluating calibration involves checking both label accuracy and predicted probabilities. In TypeSafe’s confidence documentation, confidence is described as a summary metric derived from the full probability distribution. Validating calibration means verifying whether predicted confidence matches real-world accuracy rates.

Input Conditions That Reveal Structural Differences

Differences in public codebases suggest several test conditions that highlight contrasting architectural trade-offs.

Input Variation	Observable Metric	Architectural Mechanism Tested
Scale questions on a fixed long document	Latency and memory scaling	Shared state computation vs. per-question re-encoding
Inject answer hints into an adjacent question	Decision changes in target query	Cross-question attention isolation
Add choice C alongside fixed choices A and B	Shifts in relative ratio between A and B	Independent scoring vs. cross-candidate attention
Permute candidate ordering	Output choices and probability stability	Sensitivity to candidate position bias
Evaluate predictions on out-of-domain tasks	Empirical accuracy on 80% confidence bins	Domain generality of probability calibration

In practice, post-processing candidate distributions can alter probability ratios even under independent scoring. Similarly, latency when scaling queries reflects server load and batch configurations in addition to state caching. These overlapping factors complicate empirical measurements.