Brooooooklyn c7ddd679fd feat(server): use native tokenizer impl (#6960) ... ### Benchmark `yarn workspace @affine/server-native bench` ``` ┌─────────┬────────────┬─────────┬────────────────────┬──────────┬─────────┐ │ (index) │ Task Name │ ops/sec │ Average Time (ns) │ Margin │ Samples │ ├─────────┼────────────┼─────────┼────────────────────┼──────────┼─────────┤ │ 0 │ 'tiktoken' │ '5' │ 176932518.76000002 │ '±4.71%' │ 100 │ │ 1 │ 'native' │ '16' │ 61041597.51000003 │ '±0.60%' │ 100 │ └─────────┴────────────┴─────────┴────────────────────┴──────────┴─────────┘ ```		2024-05-16 07:55:10 +00:00
..
index.js	feat(server): use native tokenizer impl (#6960)	2024-05-16 07:55:10 +00:00