Pull a 280B weight set onto a local machine, run a few commands, and watch the first token appear. What blocks a first deployment is usually not compute but three earlier questions: where the license lets you use these weights, in what capacity, and whether the request is shaped right. After Baidu released ERNIE 5.5 Flash Base on November 5, 2026, community commands mostly covered serving, while the license boundary and input-format traps rarely got explained together. This article follows the official model card and LICENSE text in five steps, from a pre-download self-check to the first verification request. Commands are quoted as given, without inventing hardware numbers.
Division of labour. The hotspot piece covers the release event, architecture claims and license boundary; this one sticks to procedure. The comparison piece covers how to read that self-reported benchmark table and how to choose among four base models; this one compares no scores. For general memory sizing, see llm-local-deploy-vram-sop; this article is about one weight set and its official example configuration. For general LoRA and QLoRA fine-tuning, see llm-fine-tuning-sop; here I only point at the official continued-training entry point. Put simply, this answers whether the steps get you a running service, not whether you should use the model or how to squeeze VRAM.
One reminder: a responding service and commercial clearance are different things. Step 1 restates LICENSE clauses, not legal advice; whether your use falls inside the license is yours to judge.
Step 1: Read the LICENSE before you download, three checks
Do not click download yet. These weights do not use a common open-source license; they ship under a custom ERNIE 5.5 Flash Base Model License Agreement v1.0, and the Hugging Face repo is tagged license: other. The clauses below are restated from the LICENSE text retrieved on November 6, 2026, with that verification point noted, as clause restatement only, not legal advice. Walk through three checks before pulling the weight set.
Check one, is the place of use inside the Territory. The agreement defines Territory as all geographical areas worldwide other than the European Union, the United Kingdom and South Korea. Section 5.3 states that the licensed materials, derivative models and their outputs must not be used, reproduced, modified, distributed or displayed outside the Territory. This only restates the clause's scope.
Check two, is Annual Revenue of Related Products or Services above US$50 million. Section 1.7 defines it as the same Operator's aggregate revenue over the 12 months preceding any assessment time from all related products or services using the licensed materials or derivative models; it is assessed continuously. Section 4.2 permits commercial use without a model license charge at or below US$50 million, provided you fulfill the Section 3.2 attribution obligations and send a one-time use notice; revenue exactly equal to 50 million falls here. Section 4.3 requires separate prior written commercial authorization before starting commercial use above US$50 million, and states that a use notice, application, pending reply or ongoing negotiation does not confer above-threshold rights. The one-time notice to Baidu_ERNIE@baidu.com under 4.2 is not that written authorization.
Check three, can your product prominently display Powered by ERNIE and send that notice. Section 3.2 requires any commercial product or service using the licensed materials or derivative models to clearly and prominently display the exact wording Powered by ERNIE in at least one readily accessible location on the user interface, official website, about page or product documentation; for APIs, backend services or model deliveries without an end-user interface, prominent display in developer, API or delivery documentation suffices. Section 6.3 grants only limited permission for that wording. The one-time notice goes to Baidu_ERNIE@baidu.com before the first commercial use, identifying the Operator, model name and version, and related products or services. It states plainly that this notice does not constitute the 4.3 written authorization.
Other clauses are easy to skip. Section 5.5 prohibits using the licensed materials, any derivative model, or any of their outputs or results to improve any other AI model or machine-learning system other than the Model or its derivative models, the distillation ban. Section 1.4 states that unreleased training code, data, inference code or other materials do not become deliverables, so you receive weights, parameters, configurations, tokenizers and accompanying materials, not a training recipe. Section 8.3 requires that after termination you cease affected use, copying and distribution and delete or destroy copies of the weights and derivative models. All of the above is clause restatement, not a legal conclusion. For a comparison across five agreements, see openweight-license-comparison-review.
Step 2: Pick the weights and the mirror, do not grab the wrong build
The official model card download table lists two builds: the FP8 version baidu/ERNIE-5.5-Flash-Base and the BF16 version baidu/ERNIE-5.5-Flash-Base-BF16, both safetensors, published simultaneously on Hugging Face and ModelScope. Official figures are 280B total parameters and 14B activated per token. Choose by precision and hardware: FP8 saves bandwidth and VRAM footprint, while BF16 expresses numbers more directly. The official side gives no recommendation that one build is better, so do not write this trade-off as an official conclusion.
One boundary has to hold: the official model card publishes neither a minimum VRAM nor a minimum GPU count, so do not fill either in from experience. Memory sizing is a general method, not an official figure here. Verifying file hashes after download is likewise general engineering common sense, not a step in the model card's download procedure. On the mirror, Hugging Face and ModelScope use the same repo names; pick whichever is more stable. The real trap is the suffix: the BF16 repo name ends in -BF16, so do not mix the two builds, and do not configure torch.bfloat16 against the FP8 build. Keep the repo name and local cache path straight, because both serving commands below reuse the same model identifier.
Step 3: The smallest runnable check, load the weights with Transformers
Before reaching for a serving framework, confirm the weights load and generate with a few lines of Transformers. The official Quick Start reads:
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_name = "baidu/ERNIE-5.5-Flash-Base-BF16"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16, device_map="auto")
model_inputs = tokenizer(["ERNIE 5.5 Flash hybrid attention"], add_special_tokens=True, return_tensors="pt").to(model.device)
print(tokenizer.decode(model.generate(**model_inputs, max_new_tokens=256)[0], skip_special_tokens=True))Loading uses AutoModelForCausalLM.from_pretrained, precision is set with torch_dtype=torch.bfloat16, device_map="auto" spreads layers across available devices, and max_new_tokens=256 caps generation length. The example model name is baidu/ERNIE-5.5-Flash-Base-BF16, matching bfloat16; switching to the FP8 build means adjusting dtype rather than copying this line verbatim.
The key line is add_special_tokens=True. The official text states this is a Base model and every input must begin with a BOS token; keeping the flag True adds the start marker automatically. Set add_special_tokens=False and the input loses its BOS, generation drifts, and the continuation stops being coherent. It looks like a broken model but is really an unmet input convention. Output is a prefix continuation, not an answer.
Step 4: Serve with SGLang or vLLM, then verify over completions
Both paths come from the Deployment section of the official model card. Pick one.
Start with SGLang. The official prebuilt image is lmsysorg/sglang:dev-ernie-55-flash; the docker command mounts the local Hugging Face cache into the container, with ernie55 as the example container name:
docker run -d -it --name ernie55 \
--gpus all \
--network host \
--ipc host \
-v ~/.cache/huggingface:/root/.cache/huggingface \
lmsysorg/sglang:dev-ernie-55-flashLaunch the server:
sglang serve \
--model-path baidu/ERNIE-5.5-Flash-Base \
--tp 4 \
--attn-dp-size 4 \
--context-length 131072 \
--max-running-requests 1024 \
--host 0.0.0.0 \
--port 30000The values --tp 4, --attn-dp-size 4, --context-length 131072, --max-running-requests 1024 and --port 30000 come from the official example. Note that --tp 4 is an official example configuration, not a minimum hardware requirement; the card does not say four GPUs are required.
Now vLLM. Create a Python 3.12 virtual environment, install the pinned version, then install the official plugin repository vLLM-ERNIE:
uv venv --python 3.12 ernie5-venv
source ernie5-venv/bin/activate
uv pip install vllm==0.32.0
uv pip install git+https://github.com/baidu/vLLM-ERNIE.gitLaunch the server:
vllm serve baidu/ERNIE-5.5-Flash-Base \
--kv-cache-dtype fp8 \
--enable-expert-parallel \
--data-parallel-size 4 \
--max-model-len 131072 \
--port 30000 \
--moe-backend hummingHere --kv-cache-dtype fp8, --enable-expert-parallel, --data-parallel-size 4, --max-model-len 131072 and --moe-backend humming are quoted as given, and --data-parallel-size 4 is likewise an official example configuration rather than a minimum hardware requirement. Both commands set context length to 131072, while the model card states both servers accept up to 1048576 tokens for long-context inference, so do not mistake the command-line value for the model ceiling.
Once the service is up, verify it. Because this is a Base model, the server exposes the completions endpoint, not chat/completions; the request body uses prompt rather than messages, and no chat template applies. The official curl reads:
curl http://localhost:30000/v1/completions \
-H "Content-Type: application/json" \
-d '{
"model": "baidu/ERNIE-5.5-Flash-Base",
"prompt": "Large language models are",
"max_tokens": 256,
"temperature": 0
}'The request hits /v1/completions, passes the prefix through prompt, and sets max_tokens to 256 and temperature to 0. A coherent continuation means weights, framework and request format line up. For general API integration, see hy4-tokenhub-api-integration-sop; here the point is the completions plus prompt convention of a Base model.
Step 5: The right entry point for continued training
For continued training, the official blog points at examples/models/ernie55/ in the NVIDIA-NeMo/Megatron-Bridge repository. A GitHub API snapshot of that directory on November 6, 2026 shows four files: README.md, benchmark_d_layer_indexer.py, convert_sft_jsonl.py and slurm_train.sh. The official side provides a continued-training example entry point and a script list, not a turnkey full training recipe. How to assemble the cluster, how many nodes, and which accelerators are not verified in that directory listing, so do not fill in specific cluster-scale numbers from press retellings.
General LoRA and QLoRA are generic parameter-efficient fine-tuning methods, not an official recipe for this weight set; they are covered in llm-fine-tuning-sop and treated as general methods here. Before starting, go back to Step 1 and confirm the license covers your intended use, especially the Section 5.5 ban on using outputs to improve other AI systems.
Three traps that catch almost everyone
First, the missing BOS. Setting add_special_tokens to False leaves the Base model input without its start token and the output drifts. Keeping that flag True in the Step 3 tokenizer call is the most easily overlooked convention here.
Second, treating the Base build as a chat model. Its official positioning is a pretraining-end checkpoint, without mid-training or post-training, and it is not a chat model. Use completions plus prompt, do not copy messages out of a chat template, and do not invent one. The official text says a post-trained ERNIE 5.5 Flash will follow but gives no date; do not invent a timeline or force conversation behaviour out of the Base build.
Third, misreading territory and revenue. The Territory excludes the European Union, the United Kingdom and South Korea. Revenue above US$50 million requires separate prior written authorization, and a one-time notice is not that authorization. The Section 5.5 ban, Section 1.4 training-data limits and Section 8.3 weight deletion are often overlooked together. Equating it runs with I can ship it commercially is the most common and most expensive misjudgement.
Platform and scope boundaries
This stack is Linux, Docker and CUDA, and Windows cannot run these SGLang and vLLM commands natively. Using WSL2 to host a Linux environment and then install Docker is a general workaround, not a step written in the official README, so do not look to the model card when troubleshooting that environment.
To say it once more: a running service only proves the weights, framework and request format line up; commercial eligibility is decided by the LICENSE clauses in Step 1. This article restates clauses, reproduces steps, and is not legal advice. Check version pins such as vllm==0.32.0 against the repository on download day.
FAQ
Q1: The service runs, so can I use it commercially? A1: No. Running it is engineering; commercial eligibility is licensing. The three checks in Step 1 still apply: is the place of use inside the Territory, is annual revenue above US$50 million, and can you prominently display Powered by ERNIE while sending a one-time notice to Baidu_ERNIE@baidu.com. Above the threshold you need separate prior written authorization; an application or notice does not constitute it.
Q2: FP8 or BF16, and did the official card publish a minimum VRAM? A2: Decide by precision and hardware. FP8 saves bandwidth and VRAM footprint, BF16 is more direct numerically, and the official side gives no recommendation. The model card publishes neither a minimum VRAM nor a minimum GPU count, so do not fill either in. Hash verification after download is general engineering common sense, not an official step.
Q3: I sent messages like a chat template, why did it error or produce odd output? A3: Because it is a Base model. The server exposes /v1/completions, the body uses prompt rather than messages, and no chat template applies. Keep add_special_tokens=True so every input carries a BOS token; turn it off and the output drifts. These are two sides of the same mistake.
Q4: Does --tp 4 mean I need at least four GPUs, and what is the context ceiling? A4: Both --tp 4 and --data-parallel-size 4 are official example configurations, not minimum hardware requirements, and no minimum GPU count is published. Both launch commands set context length to 131072, while the model card says both servers accept up to 1048576 tokens for long-context inference. The command-line value is not the model ceiling.
Q5: Can I run this on Windows, and when does the post-trained version arrive? A5: This Linux, Docker and CUDA stack is not supported natively on Windows; WSL2 is a general workaround, not an official README step. For the post-trained version, the official text only says it will follow, with no date, so do not invent one. If you need dialogue before then you must do continued training or adaptation yourself, which leads back to the license and the Step 5 entry point.
Related reading
Pick by whichever step you are stuck on: the hotspot piece for the release event, architecture claims and license boundary; the comparison piece for reading the self-reported table and choosing among four base models; the October 23, 2026 piece for placing this license among five agreements; the general VRAM SOP for memory sizing; the LoRA and QLoRA SOP for parameter-efficient tuning; and the API integration SOP for wrapping the service as a public API.