Has it? It's my understanding that GPT-3 was 12,288 and GPT-4 went down to 3072. | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

causal on May 15, 2024 | parent | context | favorite | on: Gemini Flash

Has it? It's my understanding that GPT-3 was 12,288 and GPT-4 went down to 3072.

WhitneyLand on May 15, 2024 [–]

I’m not aware that internal embedding dimension size has been made public for Gpt4 et al.

In general for models we know its trended upward, but for sure it’d be interesting to know what they’re using now.

Consider applying for YC's Fall 2026 batch! Applications are open till July 27.
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact