sqlite extension that uses database tables for indexing HNSW values. This is something I wanted to build after trying to build my own in pure Go.

Warning: Was vibe coded with my instructions of architecture, testing preferences (LuaJIT for FFI), and how I wanted to be able to accept stories.

ncruces helped me with some code I made for VFS. It uses [zstd seekable](https://github.com/jtarchie/sqlitezstd) for reading a file. I thought it would be really well-suited for S3.

- Support for HTTP range queries - "Fast" read times - No disk required

I was wrong.

It turns out that for specific SQL queries, it might be fine, but not fast. For queries that do aggregations, like `COUNT`, sqlite loads the whole database anyway.

There are two sections that describe the same thing.

- Try: turning off synchronous mode

- Try: synchronous “OFF”

Was this AI generated?

Note: We've crossed paths on another thread (Reddit?).

I have an experiment going (as part of a talk) of how compressed of a sqlite database I can get all of English Wikipedia with full-text search.

So far, the smallest with full articles and titles is 28 GiB.

I'm glad it was helpful. It was a list of things I had known before I used it. So it was meant to be a quick getting started like doc.

Someone on Reddit mentioned in Norwegian, it means "nothing."

I've used Pocketbase for some other projects. A quick small web app, but it was so easy.

I did see that. I'm going to let the fork simmer for awhile. I didn't require module support for my runtime. Since its meant to be very basic ETL runtime.

Fair enough! Though they're quite clearly a much more serious/focused/resourced team than the goja dev, and they've fully integrated it into the K6 codebase now. I expect it'll have a bright future.

Since you're doing ETL, did you consider something like Conduit, which is written in Golang and also allows for JS processing in the pipeline? Or is that overkill for what youre doing?

https://conduit.io/docs/processors/builtin/custom.javascript

Not doing it all, Go allows my API endpoint to accept sandboxed javascript code to enable users to run more complex queries against my data. It is also faster because they don't have to make individual API calls for each data set. Therefore, more data is required to be sent over the wire to them.

Thanks. Ive been exploring all of this for the past many hours and definitely see the appeal and various use cases for it now.

Thanks for opening me up to it all - it will definitely solve a problem that I'm about to start working on!