Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I kept having to debug prompt issues with open models

So I built OpenAI's Tokenizer page for all tokenizers on HuggingFace: Llama, Mistral, GPT2, MPT, Persimmon, T5 etc

http://tokenwiz.rahul.gs



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: