One Open Source Project a Day (No. 225): NanoJev — A 0.6B Parallel Decision Model That Outputs Probability Distributions Instead of Generating Tokens

Introduction

"A decision doesn't need to write a sentence to explain itself — it just needs a probability distribution."

This is the 225th article in the "One Open Source Project a Day" series. Today's project is NanoJev.

Most LLM-driven decision-making approaches can't escape an awkward efficiency problem: the model autoregressively generates a chunk of text (something like "I think we should choose option B, because..."), and the final decision has to be parsed out of that text afterward. This requires token-by-token decoding — slow, costly, and the "confidence" the model expresses is usually just an adjective it happened to use, not a genuinely calibrated probability.

If a task's essence is really "given a state and a set of candidates, tell me the probability of each candidate," why make the model generate a wall of text first and parse it after? NanoJev's answer: skip the generation, output the distribution directly. It's a lightweight recreation of a parallel decision model system called "Jev." Built on a Qwen3-0.6B backbone with dedicated decision heads, it takes a state and a question as input and, in a single forward pass, outputs a complete probability distribution — zero output-token decoding.

954 Stars, MIT License, validated with maze navigation and Snake game benchmarks.

What You Will Learn

• How NanoJev structures a decision as a "state + question + candidate set" triple
• Three decision head types: dynamic Choice, Boolean judgment, and Ordered Score
• Why "zero output-token decoding" fits decision-making better than autoregressive generation
• The full training pipeline: build queries → organize data → train → evaluate → serve
• How NanoJev compares against original Jev and an untuned Qwen3-0.6B on game benchmarks

Prerequisites

• Basic understanding of how LLMs run inference (autoregressive generation vs. a single forward pass)
• Familiarity with basic probability and classification/regression concepts
• Optional: familiarity with reward modeling concepts in reinforcement learning

Project Background

What It Is

NanoJev's official positioning: "A 0.6B parallel decision model. States and questions in, complete probability distributions out — with zero output-token decoding."

Worth clarifying up front: this is not an image generation or editing tool. It's a structured decision model/reasoning framework — essentially it keeps an LLM's language understanding ability while re-engineering the output side from "generate text" to "output a calibrated probability distribution," aimed at use cases like game agent decision-making and probability calibration research.

Team and Background

• Author: TianyuCodings
• License: MIT License
• Base model: Qwen3-0.6B
• Related resources: model repo C-Tianyu/NanoJev, dataset C-Tianyu/NanoJev-Data (both on HuggingFace)

Project Stats

• ⭐ GitHub Stars: 954
• 🍴 Forks: 127
• 👀 Watchers: 8
• 📄 License: MIT
• 📊 Commits: 13

What It Does

The Problem It Solves

Traditional LLM decision-making approach:
  State + question → autoregressively generate text ("I think we should
  pick B, because...")
  ↓ parse the text to extract the final decision
  ↑ token-by-token decoding is slow and costly
  ↑ "confidence" is just an adjective in the text, never truly calibrated
 
NanoJev's approach:
  State + question + candidate set → a single forward pass →
  directly output a complete probability distribution
  ↑ zero output-token decoding, fast
  ↑ the distribution is trained and calibrated, usable directly for
    ranking / greedy selection / probability sampling
  ↑ a single forward pass can batch multiple independent states and questions
     (official example: "6 states · 18 questions · 44 candidate paths ·
     1 backbone forward")

Use Cases

1. Game agent decision-making

   • Maze navigation (8×8 up to 50×50 in various sizes), real-time decisions for a Snake game agent

2. Local safety judgment

   • Fast boolean judgments for scenarios like collision avoidance, combined with code-based planning

3. Probability calibration research

   • Calibration experiments using CE (cross-entropy)/Brier loss and paired proper-reward learning

4. Scenarios needing batched decisions

   • Evaluate multiple candidate paths in a single forward pass, suited to applications needing high-throughput decision-making

Quick Start

Simplest path: run the interactive demo locally (no model weights needed)

git clone https://github.com/TianyuCodings/NanoJev.git
cd NanoJev
python3 -m http.server 8080 --bind 127.0.0.1 --directory web

Visit http://127.0.0.1:8080/side-by-side.html for a three-panel comparison demo, or arcade.html (dark arena demo) and comparison.html (early benchmark viewer).

Full run: download model weights and start the service

# Install dependencies
python -m pip install -r requirements-toy.txt
 
# Download model checkpoint and dataset
python -c "
from huggingface_hub import snapshot_download
snapshot_download(
    repo_id='C-Tianyu/NanoJev', local_dir='checkpoints/NanoJev',
    allow_patterns=['best.safetensors', 'config.json', 'tokenizer/*', 'backbone_config/*'],
)
snapshot_download(
    repo_id='C-Tianyu/NanoJev-Data', repo_type='dataset', local_dir='data/NanoJev',
)
"
 
# Start the persistent service
python scripts/serve_decisions.py \
  --checkpoint-dir checkpoints/NanoJev \
  --web-root web --port 8765

Once started, the service loads the model once and accepts repeated batches of decision requests via POST /api/evaluate.

Core Features

1. Three Decision Head Types

2. Triple-Structured Decisions

Every decision is defined by a state, a question, and a candidate set. A shared decision head processes the input and returns the corresponding distribution output for the candidate set based on the decision type.

3. Five-Step Training Pipeline

Build queries
  ↓ generate states, questions, candidate descriptions, and target distributions
Organize data
  ↓ keep related maps, rules, and variants in the same data split
Train
  ↓ Qwen3-0.6B init + decision-head warmup + complete-question distribution losses
Evaluate
  ↓ measure probability quality + game controller records actual actions
Serve and visualize
  ↓ reuse a persistent model endpoint, replay full trajectories in the browser

4. Multiple Pretrained Checkpoint Variants

A Deeper Look

The Performance Implications of "Zero Output-Token Decoding"

NanoJev's core architectural decision is to abandon autoregressive generation entirely, re-engineering the LLM's output side into structured decision heads. The official documentation gives a fairly intuitive number for the performance difference this brings:

"6 states · 18 questions · 44 candidate paths · 1 backbone forward"
 
Meaning:
  6 independent states + 18 questions + 44 candidate paths
  ↓ all folded into a single backbone forward pass
  ↓ instead of 44 separate autoregressive generations

The cost of autoregressive generation accumulates token by token — generating a chunk of explanatory text might require tens to hundreds of forward passes (one per token). NanoJev compresses the evaluation of 44 candidate paths into "1 backbone forward pass" — essentially swapping "expressing confidence in text" for "expressing probability directly through the model's internal numerical output," bypassing the entire autoregressive decoding cost chain.

The Design Trade-offs Behind Choice / Boolean / Score Heads

Why not use one generic head for every decision type? The three head types actually correspond to three fundamentally different statistical problems:

Choice (dynamic selection):
  Variable candidate count (2-255) → needs "set attention"
  ↑ the model must understand "these candidates form a mutually
    exclusive set," not independent judgments
 
Boolean:
  A single proposition, binary yes/no → a single-path sigmoid is enough
  ↑ no need to be aware of other candidates
 
Score (ordered evaluation):
  Levels have an order relationship (1 star to 5 stars) → can't be
  treated as a plain classification problem
  ↑ using a probability-weighted expectation lets a non-integer
    expectation like "3.5 stars" emerge naturally

This design reflects a simple but easily overlooked principle: when decisions have different statistical structures, the loss function and output head should differ too. Forcing "which option," "true or false," and "what rating" into the same softmax loses the structural information specific to each question type.

Real-World Training Results: Small Model vs. Fine-Tuning

NanoJev was benchmarked against original Jev and an untuned Qwen3-0.6B across three game benchmarks, and the results are fairly compelling:

A few points worth noting:

1. The untuned Qwen3-0.6B lags significantly behind (35%/15% vs. NanoJev's 95%/90%), showing that the purpose-trained decision heads deliver a real improvement — not just a rebranded model
2. NanoJev requires far fewer attempts than original Jev on the 50×50 maze (244 vs. 2,738), suggesting the lightweight recreation is actually more efficient on this specific task, likely due to more focused training data
3. NanoJev's overall accuracy is slightly lower than original Jev (e.g., 95% vs. 100% on the navigation benchmark), which is expected — recreating a system with fewer resources involves a reasonable trade-off in performance

Positioning: A Research Recreation, Not a Commercial Product

NanoJev explicitly positions itself as a "lightweight, reproducible" research alternative. That positioning is clear: it doesn't aim to outperform original Jev, but rather to let more researchers validate and extend the "parallel decision model" idea at a smaller training cost with a simpler deployment path. Roadmap items like expanding data scale and RLCD (contrastive distillation reinforcement learning) expansion all point toward "continuing to explore the boundaries of this architecture" rather than "polishing it into a product."

Project Links and Resources

Official Resources

• 🌟 GitHub: https://github.com/TianyuCodings/NanoJev
• 🤗 Model: C-Tianyu/NanoJev (HuggingFace)
• 🤗 Dataset: C-Tianyu/NanoJev-Data (HuggingFace)
• 📄 License: MIT License
• 🐛 Issues: GitHub Issues

Related Resources

• Qwen3 — The 0.6B backbone network NanoJev is built on
• Brier Score — One of the calibration scoring methods used in NanoJev's training

Summary

Key Takeaways

1. Zero output-token decoding: abandons autoregressive generation, using decision heads to output probability distributions directly, sidestepping the performance overhead of token-by-token decoding
2. Triple-structured decisions: state + question + candidate set clearly bounds every decision query
3. Three decision heads, each fit for purpose: Choice/Boolean/Score correspond to distinct statistical problem structures rather than being forced into a single softmax
4. Real benchmark validation: significant improvement over untuned Qwen3-0.6B on maze navigation and Snake game benchmarks, with a reasonable performance trade-off versus original Jev
5. Research-oriented positioning: explicitly a lightweight recreation rather than a commercial product, serving further exploration of the "parallel decision model" direction

Who This Is For

• Game AI / agent decision-making researchers: need a lightweight, reproducible baseline for parallel decision models
• Researchers focused on LLM efficiency: interested in architectural changes that skip autoregressive generation in favor of directly outputting structured distributions
• Probability calibration researchers: want to study the practical effect of CE/Brier loss and paired reward learning in decision-making scenarios
• Teaching/reproduction-oriented learning contexts: want to understand a "decision head + backbone" architecture without training a large-scale system from scratch

One-Line Verdict

NanoJev raises a question worth sitting with: if a task's essence is producing a probability distribution, why detour through generating a paragraph of text at all?

Check out PrimeSkills — a curated marketplace of AI agents and skills that have been validated in real-world, enterprise-grade workflows. No fluff, just what actually works.

Find more useful knowledge and interesting products on my Homepage