Actually day and a half :). I'm all for it but unfortunately I have pretty old hardware.

No it shouldn't but since it neural net there is a small chance though.

Yes and yes you are correct!

Just fixed it.

No I didn't it yet. I would be grateful if you could advise me such a benchmark.

Not sure, havent done so myself but I think you can use MTEB maybe. Or otherwise a llm benchmark on large inputs (and compare your chunking with naive chunking)