Let me draw the boundaries first, so this piece does not overlap with older site posts or the three sibling articles in the same batch. The October 23, 2026 review Open Weight, But How Open? Five Model Licenses Compared is a license-tiering exercise, grading terms, territory, and revenue thresholds. This article is not a license comparison; licensing gets one sentence and a link near the end. The August 29, 2026 review Tencent's 770B Flagship Activates Only 49B: Five Open-Weight Flagships Compared covers 700B-to-2.8T scale and VRAM thresholds; this article moves to the 280B-to-310B band, to Base checkpoints at the end of pre-training, and asks how to read one self-test table. In the same batch, the hotspot piece covers the November 5, 2026 release and license boundary, and the setup SOP covers installation. This piece does methodology and selection only: no release recap, no command lines, no license breakdown. Those three answer what it is, whether you may use it, and how to install it; this one answers why the score table is worth trusting, and why it still is not the whole story.
1. Put the four base models and the table on the desk
On November 5, 2026, Baidu released ERNIE 5.5 Flash Base as open weights, and the Hugging Face model card shipped with a 30-item evaluation table, placing four Base checkpoints under one process: ERNIE-5.5-Flash-Base, DeepSeek-V4-Flash-Base, Hy3-preview-Base, and MiMo-V2.5-Base. Up front: every one of the 30 scores is from Baidu's unified evaluation pipeline, self-reported, with no third-party reproduction published. They are vendor self-reported numbers, not an objective ranking, and support no absolute claim about who is strongest. Every figure below carries that qualifier.
| Model | Total params | Activated per token | Activation ratio |
|---|---|---|---|
| ERNIE-5.5-Flash-Base | 280B | 14B | 5.0% |
| DeepSeek-V4-Flash-Base | 284B | 13B | 4.6% |
| Hy3-preview-Base | 295B | 21B | 7.1% |
| MiMo-V2.5-Base | 310B | 15B | 4.8% |
Setup, per the official blog: all four use their own publicly released checkpoints, one shared evaluation framework, identical prompts and few-shot examples, and the same judge model for free-form answers. The 30 items span seven groups: Exam, Knowledge, STEM, Reasoning, Multilingual, Code, and Long Context. Inside this table, ERNIE takes the top score on 24 items and is not top on 6. "Inside this table" means a relative position from Baidu's unified evaluation, self-reported, without third-party reproduction; swap the framework or prompts and the order could reshuffle.
2. What the shared ruler actually standardizes
The value is not the scores; it pins down the variables that inflate benchmark numbers. The same model scores differently with a different harness, prompt, few-shot set, or judge model, and a heavier scaffold inflates the score, so vendor leaderboards often measure packaging rather than capability. Baidu folds four models into one framework, one prompt and few-shot set, and one judge, so the remaining gap can be attributed to the models, not the scaffold. But "unified" and "trustworthy" differ; the next section takes the second apart.
3. Five things the ruler still does not fix
First, it is still vendor self-reported: Baidu's unified evaluation, self-reported, with no independent lab re-running all four checkpoints. Self-reported is not fabricated, but it stays self-reported data, never an objective ranking.
Second, all four are Base checkpoints at the end of pre-training: no mid-training and no post-training, a starting point for domain adaptation, instruction tuning, and RL research, not a chat model. The 30 items measure a foundation for further training, not chat-box experience or agent task completion. Endorsing an assistant with them is a category error.
Third, scores are comparable only within this table; cross-leaderboard comparison is forbidden. Terminal-Bench, Agent Arena, and the AA Intelligence Index have their own task sets, scaffolds, compute budgets, and scoring rules; their setups cannot be horizontally aligned. Being ahead on 24 items here says nothing about placing first on some agent leaderboard, nor the reverse. This is the reddest line in this batch.
Fourth, the table has scores and no costs: no throughput, latency, VRAM footprint, parallelism config, and no architecture, context length, license, training data, or deployment cost for the other three. The first two rows give only total and activated parameters; this article copies exactly that and speculates about nothing outside it.
Fifth, 24 first places is not comprehensive dominance. The 6 misses cluster on sharply defined capabilities, making "leading" a structured claim. The honest reading: a unified, but still self-reported, and Base-only reference starting point.
4. Read the six misses one by one
| Item | ERNIE-5.5-Flash-Base | Top scorer in this table | Its score |
|---|---|---|---|
| ChineseSimpleQA | 73.0 | DeepSeek-V4-Flash-Base | 75.3 |
| BBH (CoT) | 89.2 | Hy3-preview-Base | 90.0 |
| INCLUDE-lite | 79.5 | DeepSeek-V4-Flash-Base | 79.8 |
| FullStackBench | 54.6 | Hy3-preview-Base | 56.7 |
| HELMET-Rerank | 56.2 | DeepSeek-V4-Flash-Base | 60.0 |
| InfiniteBench | 46.5 | DeepSeek-V4-Flash-Base | 48.0 |
Qualifier as before: all six are from Baidu's unified evaluation, self-reported, with no third-party reproduction; the opponent score is the highest value for that item within this table, not any cross-leaderboard comparison.
ChineseSimpleQA tests Chinese factual QA. Top is DeepSeek-V4-Flash-Base at 75.3, ERNIE-5.5-Flash-Base at 73.0. These hinge on precise recall of Chinese world knowledge, so ERNIE is not uniformly ahead here; "best at Chinese" does not follow from 24 first places.
BBH (CoT) tests long-chain reasoning. Top is Hy3-preview-Base at 90.0, ERNIE at 89.2, the smallest gap. In a CoT setting, changing the prompt or dropping chain of thought can reshuffle the order, so one item cannot become a verdict on reasoning.
INCLUDE-lite tests multilingual coverage. Top is DeepSeek-V4-Flash-Base at 79.8, ERNIE at 79.5, again a hair apart. For a multilingual scenario, total wins do not help; this cell does.
FullStackBench tests full-stack code. Top is Hy3-preview-Base at 56.7, ERNIE at 54.6. Inside the Code group ERNIE is top here on HumanEval+ 82.3, MBPP+ 81.5, LiveCodeBench 40.1, and RepoQA 91.7, yet trails on full-stack. "Strong at code" must be broken into sub-capabilities.
HELMET-Rerank tests reranking in long context. Top is DeepSeek-V4-Flash-Base at 60.0, ERNIE at 56.2, the largest gap. In the Long Context group ERNIE leads on HELMET-ICL 85.9, RepoQA 91.7, and LongBench-v2 48.7, but trails on reranking, so long-context ability cannot be one number.
InfiniteBench tests very long documents. Top is DeepSeek-V4-Flash-Base at 48.0, ERNIE at 46.5. With HELMET-Rerank it points to one thing: ERNIE's 128K training context with 1M for fine-tuning and inference does not push every long-text benchmark to the top of this table.
The misses cluster on Chinese factual QA, CoT reasoning, multilingual coverage, full-stack engineering, reranking, and very long documents, while the 24 first places sit mostly in Exam, STEM, basic code, and most of Long Context. Leading is not leading everywhere; every digit remains self-reported with no third-party reproduction.
5. Read the wins: ten checkable samples
| Item | ERNIE-5.5-Flash-Base | DeepSeek-V4-Flash-Base | Hy3-preview-Base | MiMo-V2.5-Base |
|---|---|---|---|---|
| MMLU | 93.3 | 89.4 | 87.5 | 87.6 |
| MMLU-Pro | 78.4 | 69.2 | 66.9 | 65.7 |
| HumanEval+ | 82.3 | 69.5 | 78.0 | 74.4 |
| MBPP+ | 81.5 | 67.2 | 78.3 | 74.3 |
| LiveCodeBench | 40.1 | 31.9 | 33.5 | 32.4 |
| GPQA | 60.2 | 55.9 | 50.2 | 51.4 |
| RepoQA | 91.7 | 81.0 | 91.3 | 56.5 |
| LongBench-v2 | 48.7 | 44.5 | 39.8 | 44.6 |
| HELMET-ICL | 85.9 | 83.8 | 83.0 | 80.2 |
| PIQA | 96.0 | 95.0 | 95.2 | 93.6 |
Whole table: Baidu's unified evaluation, self-reported, no third-party reproduction; values as on the official model card, showing relative position under one shared ruler, not an objective ranking.
Even one capability is uneven internally. The gaps between MMLU 93.3 and MMLU-Pro 78.4, and between HumanEval+ 82.3 and LiveCodeBench 40.1, come from item difficulty and evaluation format, not a model shifting by mood. On RepoQA, ERNIE 91.7 and Hy3-preview-Base 91.3 are nearly tied while MiMo-V2.5-Base is 56.5, so the same group has different strengths on different sub-items. Those highs and lows came from one framework, one prompt set, and one judge model, so relative positions are clean. Still self-reported; re-run comparable items from your own tasks rather than copy a total.
6. Selection: six accounts beyond the scores
Account one is activation ratio and per-token compute. The four total/activated pairs are 280B/14B (about 5.0%), 284B/13B (about 4.6%), 295B/21B (about 7.1%), and 310B/15B (about 4.8%). Activated parameters set per-token compute, total parameters set weight size and VRAM footprint; the two must be computed separately.
Account two is KV cache and the long-context implementation, applying to ERNIE only. Per the official account, 37 of 43 layers use HCA heavily compressed attention, compressing every 128 tokens into one cache entry, while 6 layers of MLA-DSA keep token-level full-context access; training context is 128K, and fine-tuning plus inference support 1M. That sets VRAM and throughput on long sequences, but applies to ERNIE alone; the official table offers no comparable detail for the other three, and this article speculates about none. To check the implementation or wire up inference, start from PaddlePaddle/FlashMask and baidu/vLLM-ERNIE.
Account three is the 128K-to-1M extrapolation boundary. Training at 128K with 1M supported for fine-tuning and inference is ERNIE's official account, so 1M is extrapolated, not a natively pre-trained length; at that scale verify with your own task runs rather than trust a point such as LongBench-v2 48.7 or InfiniteBench 46.5.
Account four is post-training cost. All four stop at the end of pre-training; turning one into a usable assistant means mid-training, instruction tuning, and alignment on your side, covered in the Base fine-tuning SOP. The table measures a starting point for further training, not an out-of-the-box product; waiting for the official post-trained version is steadier.
Account five is the serving engine and parallelism config. The VRAM budget and sharding of 280B-class weights need their own calculation; see the VRAM planning guide. The table carries no throughput, latency, or hardware config, so a high score does not mean it runs cheaply.
Account six is license territory, in one sentence: ERNIE 5.5 Flash Base is open weights, not unconditionally open source, and its custom license carries territory and revenue thresholds; for clause-level checks see the license comparison. The official table gives no license information for the other three either, and nothing is inferred.
7. Conclusion
For continued training or domain adaptation, choose a Base whose provenance is clear and whose evaluation methodology is written down, and treat this table as a reference starting point under one shared condition. For ready-to-use behavior, wait for the official post-trained version; do not let pre-training-endpoint scores stand in for product experience. End on your own tasks: run a few real sample types blind a few times and take the mean, which beats staring at 24 first places. Once more: every number here is from Baidu's unified evaluation, self-reported, with no third-party reproduction; it is not an objective ranking, it does not align across leaderboards, and it cannot be packaged into an absolute cross-leaderboard claim.
FAQ
Q1: 24 first places, so can we say ERNIE won? A1: No. These are from Baidu's unified evaluation, self-reported, with no third-party reproduction, so self-reported data, not an objective ranking; 6 of the 30 items are not the highest here; all four are un-post-trained Base models, so scores do not equal chat or agent experience; and cross-leaderboard comparison is forbidden.
Q2: Can this table be compared with Terminal-Bench, Agent Arena, or the AA Intelligence Index? A2: No, cross-leaderboard comparison is forbidden. Those leaderboards have different task sets, scaffolds, compute budgets, and scoring rules; their setups cannot be horizontally aligned, so a position here neither implies a position there nor can be inferred back.
Q3: All four are Base models, so do the scores represent real conversational experience? A3: No. All four are checkpoints at the end of pre-training with no mid-training and no post-training, a starting point for further training rather than a chat model. For ready-to-use behavior, wait for the official post-trained version or do post-training yourself.
Q4: Does this table give the other three models' context, license, or deployment cost? A4: No. The first two rows give only total and activated parameters; this article copies that and speculates about nothing. 128K training, 1M for fine-tuning and inference, 37 HCA layers with one cache entry per 128 tokens, and 6 MLA-DSA layers apply to ERNIE only and cannot be carried over.
Q5: So what should I actually do with this table? A5: Treat it as a reference starting point under one shared condition. It unifies framework, prompts, few-shot examples, and judge model, good for shortlisting candidates and seeing relative strengths. But it remains self-reported with no third-party reproduction, so re-run on your own tasks and fold in activation ratio, KV cache, the 128K-to-1M extrapolation, post-training cost, and deployment thresholds.
Related reading
- Open Weight, But How Open? Five Model Licenses Compared: license-tiering
- Tencent's 770B Flagship Activates Only 49B: Five Open-Weight Flagships Compared: trillion-scale VRAM thresholds
- ERNIE 5.5 Flash Base open-weights hotspot: release and license boundary
- ERNIE 5.5 Flash Base setup SOP: installation steps
- Base fine-tuning SOP: continued training and alignment
- VRAM planning guide: deployment thresholds