Put three inference engines on one table and the striking detail is not which one runs faster. It is a license clause: one of them forbids offering the software to third parties as a hosted or managed service. TrnRT, Kindi, and Baseten are three repositories that appeared on GitHub in late October 2026, yet their hardware bets sit a full generation apart. One is tied to the AWS Trainium2 accelerator, one targets only next-generation data-center GPUs, and one is built for decade-old graphics cards and x86-64 CPUs. More important, the three licenses are not the same: Apache License 2.0, Elastic License 2.0, and GNU AGPL v3.0 coexist, and One License forbids a Managed Service offering to third parties, a restriction the other two do not carry in the same literal form. As of November 15, 2026, laying out the README files, the LICENSE files, and the GitHub API fields of the three repositories yields no performance leaderboard, only three non-overlapping hardware routes and three different licensing boundaries. This piece checks only verifiable fields, cell by cell; it makes no selection recommendation, assigns no scores, and runs no benchmarks.
Three engines, three hardware generations
The three coordinates are these. RunanywhereAI/trnrt is an inference engine for AWS Trainium, basetenlabs/deepseek-ultrafast is a Baseten engine for DeepSeek-V4.1-Flash, and ahtx/kindi is a local engine aimed at low-cost GPUs and CPUs. Their addresses are RunanywhereAI/trnrt, basetenlabs/deepseek-ultrafast, and ahtx/kindi, all readable on github.com down to the README and LICENSE text. Cumulative star totals from the GitHub API on November 15, 2026 are 1584, 331, and 168; these are totals, not single-day increments, and this piece does not rank them.
| Dimension | TrnRT | Baseten | Kindi |
|---|---|---|---|
| License (per LICENSE text) | Apache License 2.0 (spdx_id: Apache-2.0) | Elastic License 2.0 (license.key: other, spdx_id: NOASSERTION) | GNU AGPL v3.0 (spdx_id: AGPL-3.0) |
| Target hardware | AWS Trainium2, one trn2.48xlarge | 4 x GB300, 4 or 8 x B300, 4 or 8 x B200, sm_100a and sm_103a, CUDA 13 | Quadro K420 (1 GB and 2 GB), K1200, K6000, Tesla P100, GTX 1080, RTX 3080, AMD Radeon 780M, 840M, 8060S, x86-64 CPU |
| Supported models | GLM-5.3-Flash first, more open models Coming soon | DeepSeek-V4.1-Flash | Prism ML ternary Bonsai 2B, 4B, 9B, 27B, AH-Nanbeige4.3-33B, Nemotron 3.5 Lightning 30B-A3B, DeepSeek-V4.1-Flash (552B) |
| Context and cache | 1M-token context, sparse attention (top 2,048) plus linear attention, fixed-size state; prefix cache as a radix tree over FP8 KV pages plus a host-memory tier | 1M context, multi-server KV cache sharing; each server keeps recent KV in host memory, fetched host-to-host over RDMA, no prompt recompute; a Redis instance for coordination | prompt cache, tool calls, thinking mode, speculative decoding; OpenAI-compatible server at http://localhost:8093/v1 |
| OpenAI-compatible shape | chat and completions, streaming, tool calls, reasoning output | OpenAI-compatible /v1/chat/completions | OpenAI-compatible server |
| Performance framing (README self-reported) | NKI megakernels: one fused kernel per layer half for decode, KDA attention, fused DSA, mHC, MoE FFN | README self-reported: 4 x GB300 TP4 decode 1,740 tokens/s, 8 x B300 TP8 2,060 tokens/s; prefill 1M tokens in 13.2 s (76,000 tokens/s) and 11.7 s (85,000 tokens/s) | README self-reported: DeepSeek-V4.1-Flash 552B at 18.7 tok/s on one RTX 3060 12 GB with a dual-Xeon DDR4 server |
| Language and cumulative stars | Python, 1584 stars, 18 forks, created 2026-10-30, branch development | Rust, 331 stars, 17 forks, created 2026-10-27, branch main | Python, 168 stars, 38 forks, created 2026-10-20, release v0.6.0 on 2026-11-12 |
| Restriction on serving third parties | No dedicated hosted-service ban in Apache-2.0 text | LICENSE states verbatim: You may not provide the software to third parties as a hosted or managed service; forbids managed service, GitHub returns NOASSERTION | Governed by AGPL-3.0 text |
Each cell is written from LICENSE text, README wording, or a GitHub API field. The performance row is labeled README self-reported across all three, not third-party measurements or any test run here. The table aligns the two axes, license and hardware route, rather than declaring a winner; reading down one column gives a single engine's full cell instead of scattered fragments.
The license layer: three values, one clause that forbids a Managed Service
The pivot of this piece is the license. The three repositories each carry a different one. The TrnRT LICENSE and its GitHub API spdx_id both read Apache-2.0. The Kindi LICENSE opens with GNU AFFERO GENERAL PUBLIC LICENSE Version 3, 19 November 2007, and its GitHub API spdx_id is AGPL-3.0. The basetenlabs/deepseek-ultrafast LICENSE opens verbatim with Elastic License 2.0, followed by Licensor: Baseten and URL: https://www.elastic.co/licensing/elastic-license; the GitHub API reports license.key as other and spdx_id as NOASSERTION. Three values coexist here. This piece writes each one from its own LICENSE text and GitHub API field, picks no single value over the others, and offers no explanation of why each maintainer chose its license.
The sharpest gap sits in the middle one. The Elastic License 2.0 text includes a restriction verbatim: You may not provide the software to third parties as a hosted or managed service. The word forbids in the title and the words Managed Service point at exactly that line. Under the LICENSE text, providing the software to third parties as a hosted or managed service is prohibited, and GitHub classifies that repository as NOASSERTION, meaning the license was not mapped to a standard SPDX identifier. Stating those two facts is enough; this piece adds no official characterization that is not in the repository and supplies no motive for the choice of Elastic.
What must be kept separate is source availability and license restriction. Source availability says the repository can be read, downloaded, and built from its README on github.com. License restriction says whether the software may be wrapped into a service offered to third parties. These are two different questions. Visible source code does not by itself settle whether hosted serving is allowed. The Apache-2.0 and AGPL-3.0 repositories follow their own LICENSE terms and obligations. The Baseten one is source-available with a license that carries a managed-service restriction, and the two facts should be read separately rather than folded into one shared label.
The restriction on serving third parties is therefore not wordplay. Anyone thinking of wiring one of these engines into a hosted product should read that LICENSE line before the README performance table; a build that runs and an authorization to sell a service to third parties are two different things. TrnRT under Apache-2.0 and Kindi under AGPL-3.0 carry different obligation clauses; deepseek-ultrafast writes the hosted or managed service ban as an explicit prohibition. This piece draws no conclusion about which is more permissive and infers nothing about commercial boundaries the maintainers did not spell out.
The hardware and implementation layer: from NKI megakernels to old cards
TrnRT is tied to a purpose-built chip. Its README says it serves an OpenAI-compatible API from one trn2.48xlarge, runs decode as NKI megakernels at about one fused kernel per layer half, and keeps a prefix cache across device and host memory. The Highlights name NKI megakernels with KDA attention, the fused DSA layer, mHC, and the MoE FFN; the 1M-token context line describes sparse attention with a top 2,048 selection plus linear attention and fixed-size state; the Prefix cache line describes a radix tree over FP8 KV pages plus a host-memory tier for evicted pages and state checkpoints. The supported-hardware table marks AWS Trainium2 trn2.48xlarge as supported and lists GLM-5.3-Flash as the first supported model, with more open models marked Coming soon. The OpenAI API line covers chat, completions, streaming, tool calls, and reasoning output.
basetenlabs/deepseek-ultrafast sits at the other end, on next-generation data-center GPUs. Its README describes an ultra-fast inference engine for DeepSeek-V4.1-Flash on sm_100a and sm_103a, supporting 1M context length and multi-server KV cache sharing. The Requirements line lists 4 GB300 (sm_103a), 4 or 8 B300 (sm_103a), or 4 or 8 B200 (sm_100a) GPUs, the CUDA 13 toolkit, and a Redis instance for multi-server KV cache coordination. Every server keeps recent conversations' KV caches in host memory, and when a turn lands on a different server, that server fetches the cache directly from the holding server, host memory to host memory over RDMA, instead of recomputing the prompt. In the README self-reported performance table, 4 x GB300 (TP4) decode reads 1,740 tokens/s, 8 x B300 (TP8) reads 2,060 tokens/s, and prefill reads 1,000,000 tokens in 13.2 s (76,000 tokens/s) and in 11.7 s (85,000 tokens/s). Those are README self-reported figures, not third-party measurements. The API is OpenAI-compatible /v1/chat/completions.
ahtx/kindi reverses the direction toward low-cost GPUs and CPUs. Its README describes a custom inference engine for language models on low-cost GPUs, running Prism ML's ternary Bonsai models (2B, 4B, 9B, 27B), AH-Nanbeige4.3-33B, Nemotron 3.5 Lightning 30B-A3B, and DeepSeek-V4.1-Flash. It supports NVIDIA cards from the 1 GB Quadro K420 (Kepler) up to the RTX 50 series and the CMP 170HX, plus AMD Radeon 780M, 840M, 8060S graphics and x86-64 CPUs. The Requirements tested list reads Quadro K420 (1 GB and 2 GB), Quadro K1200, Quadro K6000, Tesla P100 16 GB, GeForce GTX 1080, GeForce RTX 3080. Its OpenAI-compatible server has a prompt cache, tool calls, a thinking mode, and speculative decoding, exposed at http://localhost:8093/v1. The README also says that, with the exact head, the greedy output is the same as the output of llama.cpp for the same model file, token for token.
Seen together, a purpose-built Trainium2 chip, a rack of next-generation data-center GPUs, and decade-old cards with CPUs are three different deployment premises, none a drop-in replacement for the others. The cache cell differs too: TrnRT uses a prefix cache of FP8 KV radix tree plus a host-memory tier, deepseek-ultrafast uses KV cache sharing fetched over RDMA across machines, and Kindi uses a local prompt cache plus speculative decoding. This piece does not subtract one throughput figure from another, because the three README self-reported numbers use different GPU counts, models, and context shapes and do not form a comparable win-loss table.
What can and cannot be checked
The verifiable part is in the open: GitHub API fields for the three repositories (license spdx_id and license.key, cumulative stars, forks, language, default branch, creation date, release), README wording, and LICENSE clauses all live on github.com and can be re-read by anyone. Stars are read as cumulative totals captured on November 15, 2026; this piece writes no single-day star increment and cites no derived growth score. The first commit of the Baseten repository is dated 2026-10-30 in the GitHub commit history, which is a commit record rather than an open-source announcement date. Cumulative totals and commit dates are static fields, read once and written once, not reinterpreted as a timeline of declarations.
The parts that cannot be checked, and that this piece does not fill in, are equally clear. There is no third-party benchmark here, and README self-reported numbers are not treated as measurements. Unstated official explanations of commercial boundaries are not inferred. Wider model-support timelines are left as TrnRT writing Coming soon and Baseten writing more open source releases are coming, with no dates added. The reasons for the license differences are not explained in any verifiable field, so this piece does not explain them. The DeepSeek-V4.1-Flash in that README is only a supported model name; this piece does not expand any cloud API capability of it and does not connect it to announcements elsewhere. Cell-by-cell comparison means filling the cells that can be checked and leaving the others blank, not covering a gap with imagination.
Related in-house pieces take different angles. The kinference routing article covers one project's routing, the local deployment comparison leans toward deployment selection, and the openweight license comparison is a review of weight licensing; this piece is a cell-by-cell comparison of three new engines on license and hardware route, and does not reuse the single-project conclusions of those pieces.
FAQ
Q1: Can all three engines be used freely for a Managed Service? A1: Not as a blanket answer. TrnRT uses Apache-2.0, Kindi uses AGPL-3.0, and basetenlabs/deepseek-ultrafast uses Elastic License 2.0; the latter LICENSE includes verbatim You may not provide the software to third parties as a hosted or managed service, and GitHub returns NOASSERTION for it. Restrictions on serving third parties have to be read from each LICENSE text separately, and this piece draws no unified permissiveness conclusion.
Q2: What is the basis for saying One License forbids a Managed Service in the title? A2: It is the restriction clause in the LICENSE text plus the NOASSERTION classification by GitHub. This piece writes source-available with a license carrying a managed-service restriction, adds no official characterization, and supplies no motive on Baseten's behalf.
Q3: How should the star counts be read? A3: As cumulative totals from the GitHub API on November 15, 2026: TrnRT 1584, deepseek-ultrafast 331, Kindi 168. This piece writes no single-day star increment, cites no growth score, and makes no cross-project ranking.
Q4: Can the throughput numbers in the READMEs be treated as measurements? A4: No. Those figures are labeled README self-reported, come from each repository's own README rather than third-party measurements or any benchmark run here, and use different hardware, models, and context shapes across the three. This piece does not subtract them to declare a winner.
Q5: Does this piece give a selection recommendation? A5: No. The axis is a cell-by-cell comparison of license and hardware route. Third-party benchmarks that could support a selection verdict, official explanations of commercial boundaries, and wider model-support timelines are all absent here and are not filled in on the maintainers' behalf.
Related reading
The local deployment VRAM planning and the inference framework article are adjacent reads.