OOMs even in quantized mode on a 3090. What's a better option for personal use? ... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		jbellis on Oct 13, 2024 \| parent \| context \| favorite \| on: Omni SenseVoice: High-Speed Speech Recognition wit... OOMs even in quantized mode on a 3090. What's a better option for personal use? > torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 43.71 GiB. GPU 0 has a total capacity of 24.00 GiB of which 20.74 GiB is free.

yellow_lead on Oct 13, 2024 [–]

Not sure if you mean in general, or options for this particular project, but Whisper should work for you.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact