Databricks built a proprietary vectorized accelerator for Spark they call Photon... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		sagarm on May 31, 2023 \| parent \| context \| favorite \| on: Databricks acquires serverless Postgres vendor bit... Databricks built a proprietary vectorized accelerator for Spark they call Photon. It's not just that they've tuned OSS Spark especially well.

RBerenguel on May 31, 2023 [–]

Back when I was a customer (before Photon was released, also during) they had a very good tuning, in the order of around 2x faster for the workloads we had at the time (very large graph computation and a “simple” filtering)

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact