Choice
Pick a class.
Mutually exclusive candidates for diagnosis, organ or tissue classification.
P(c₁), P(c₂), …, P(cₖ)
Medical multimodal decisions · System One
Confidence that
means what it
says.
OmniMed-Jev turns bounded medical predictions into native decision distributions — so a model can answer correctly, and tell you how likely it is to be correct.
one state · many decisions
15source datasets
8modality labels
155,729decision targets
126×lower reliability error
01 / Motivation
Token probability is not decision probability.
The number a medical model reports should be a number a decision rule can trust.
Today
Diagnoses, finding lists, counts and measurements are all serialized into text. The model's probability is about the next token, not directly about the medical decision.
Risk
Overconfidence lets errors pass with too little scrutiny. Underconfidence sends cases that could be handled automatically to unnecessary review.
Shift
Make the candidate set part of the input and train the model to return a normalized distribution over the decision itself.
02 / Method
One medical state.
Many typed decisions.
The shared MedGemma-1.5-4B backbone reads the image, question and runtime candidate set. A shared scorer returns a probability distribution for each decision in one forward pass.
Noul
Find a finding.
A separate No / Yes distribution for each radiographic finding.
P(No), P(Yes) × 14
Score
Measure a value.
An ordered grid for counts, ejection fraction and retinal scores.
Σ p(vₖ) · vₖ
Figure 1 · The backbone stays multimodal; the output object becomes a candidate-conditioned decision distribution.
03 / Results
Calibration is where
the interface changes.
Under an interface-controlled comparison, point predictions remain comparable while reported probabilities track observed correctness much more closely.
126×lower multi-label reliability error
20×lower multi-label selective risk area
9.1×lower classification reliability error
| Task family | Metric | Gen-SFT | OmniMed-Jev | Direction |
|---|---|---|---|---|
| Classification | Balanced accuracy ↑ | 0.6469 | 0.6892 | comparable |
| Multi-label | Macro F1 ↑ | 0.5449 | 0.5618 | comparable |
| Counting | MAE ↓ | 2.1429 | 2.5159 | baseline ahead |
| Regression | MAE ↓ | 3.3400 | 2.9488 | comparable |
The point-prediction comparison establishes no general accuracy advantage. Counting is the boundary case: folding a distribution over a wide ordered grid to its expectation can pull extreme values toward the centre.
Figure 3 · OmniMed-Jev moves all four families to the side where the reported number is more reliable than the baseline's reconstructed confidence.
Figure 4 · Better confidence ranking supports lower-risk selective prediction, especially for multi-label findings.
04 / Decision workflow
From image to an auditable decision.
The same state can carry a diagnosis, a set of findings and measurements without collapsing them into one generated string.
01 / READ
Medical state
Image, modality and question enter the shared multimodal backbone.
S + Q + C → hidden states02 / SCORE
Candidate space
Each candidate is read as semantic input, not as a fixed output index.
zⱼₖ = f(hstate, hcⱼₖ)03 / ACT
Decision rule
Use the distribution for ranking, shortlisting, folding or escalation.
p(c) → review / act
Figure 2 · The two interfaces learn differently across checkpoints; the comparison uses step 2424.
05 / Open source
Reproduce the comparison.
Inspect every metric.
The repository ships the decision interface, training and evaluation scripts, frozen splits, figure data and reproduction notes.
Released code
Python package, configs, launch scripts and the two-stage MedJev trainer.
Released splits
23,558 train UIDs, 600 validation UIDs and 726 held-out test states.
Released weights
LoRA adapter plus decision head on top of google/medgemma-1.5-4b-it.
Transparent scope
Metrics, reconstruction caveats and every not-like-for-like detail are documented.
Citation
@article{tang2026omnimedjev,
title = {OmniMed-Jev: Calibrating LVLM Confidence for Trustworthy Medical Multimodal Decisions via System One},
author = {Tang, Luyao and Chen, Cheng},
journal = {arXiv preprint arXiv:2610.00381},
year = {2026}
}