tryAGI/Tiktoken
High-performance .NET BPE tokenizer — up to 618 MiB/s, competitive with Rust. Zero-allocation counting, multilingual cache, o200k/cl100k/r50k/p50k encodings + HuggingFace tokenizer.json support.
Health Breakdown
Should you contribute to tryAGI/Tiktoken?
tryAGI/Tiktoken has a FoundDev health score of 86/100, which puts it in the active-and-maintained tier. The maintainer team is shipping recently, issues are being closed, and a PR you open this week has a realistic chance of being reviewed.
Last push was 0 days ago — that signals an actively maintained project. New issues are likely to get a maintainer response within days. The project is written primarily in C#, so prior C# experience will shorten ramp-up.
Licensed under MIT, a standard OSI-approved license — safe to contribute to under normal employer IP policies.
Community
