It is not mentioned in the README of the repository, but SQLiteData wraps GRDB to access the database and get notified of database changes (the meat and butter).
GRDB is by itself a solid "toolkit for SQLite databases, with a focus on application development", with both high levels APIs for everyday coding, and expert SQLite features for the demanding developers. Many apps rely on GRDB alone.
Oh hi -- Thanks for your work! Just finished replacing SwiftData in an app with GRDB and it was a pleasure to use.
After struggling with some issues in SwiftData, GRDB really hits the nail on the head in terms of providing a solid dev experience for the common cases, but allowing you to drop into the more advanced features when you need them.
Sorry about that, just opened a PR to update the readme and docs landing page! Previously it was very clear that GRDB was being used under the hood, but now we will say it explicitly.
Thank you for GRDB! I am using it in a project now and it’s been great. About the benchmarks in this repo though, how can SQLiteData be faster if it uses GRDB under the hood? Are they doing something inefficient or are they bypassing GRDB in some way?
GRDB appears to encourage Codable which is very slow. It does not require it though and there are some alternatives, some of which are also slow. ("Slow" can still mean "fast enough", depending on the user.)
The alternative I've seen for doing this with GRDB seemed more cumbersome and lacks community adoption: https://github.com/Jasperav/GRDB-ORM You must define schema and queries in an unusual external file that a Rust tool transforms for you.
Yes. GRDB encourages Codable because the user can profit from the code generated by the compiler, and this implies that database values are accessed by column name, on top of the Codable runtime, and those layers have a high cost. When necessary it is possible to access database values by position, and in this case GRDB achieves speed of light (performance nearly identical as raw SQLite).
So it appears that there is lightning-fast and lighting-faster.
Of course aside from comparing the dev ergonomics (138 vs 33 lines for the respective benchmarks), either may be fast enough depending on the use case.
BTW I did also see some discussion in swift-evolution about a future replacement for Codable but haven't tracked its progress. I hope they do because Codable is very convenient but tragically slow.
Yes, GRDB even w/ manual help is obviously not as fast raw SQLite. As much respect I have for the author "performance nearly identical as raw SQLite" is incorrect. Lighter also achieves some of the performance characteristics by avoiding allocations for bound parameters (such being statically generated).
I didn't look into SharingGRDB yet, but it seems like those macros could accomplish similar performance, the other way around (Lighter works on the SQLite DB schema).
What I'm not entirely sure of yet is why it even sits on top of GRDB in the first place, instead of just doing the SQLite parts itself. Marketing I suppose.
Nope. And not sure where you get that idea. This release even involved a rename away from including "GRDB."
When 0.1 of the library was released, it was a simple adapter between our Sharing library and GRDB, thus the name SharingGRDB. As our needs grew, the tool evolved significantly, and both the Sharing component and GRDB have become more of an implementation detail. In the future we will consider supporting any SQLite adapter, even your libraries ;)
I do, for JSON columns. I store UTF8 strings in SQLite, so that it is easy to see JSON values with a plain `SELECT *`). And I load blobs, because I code in Swift and the standard JSON decoder eats raw UTF8 memory buffers, not strings.
This avoids two useless conversions:
- first from a C string loaded from SQLite to a Swift Unicode string (with UTF8 validation).
- next from this Swift Unicode string to a UTF8 memory buffer (so that JSONDecoder can do its job).
SQLite is smart enough to strip the trailing \0 when you load a blob from a string stored in the database :-)
It is always safe, and by "safe" I mean "safe for data". You won't have to deal with data corruption. Precisely, see "How To Corrupt An SQLite Database File": https://www.sqlite.org/howtocorrupt.html
Now concurrent accesses from different processes/connections can lead to runtime errors (SQLITE_BUSY), because the database happens to be locked by one connection.
Those errors are greatly reduced by the WAL mode (https://sqlite.org/wal.html) which provides ultra-robust single-writer/multiple-readers semantics:
- Writes can not happen concurrently (SQLITE_BUSY).
Oh excellent, I wasn't sure the "snapshot isolation" level existed (the WAL kind of implied it, but I wasn't sure). SQLITE_BUSY only happens if a write lock is kept for longer than the readers' configured timeout, right? It shouldn't happen for short writes?
One should only expect SQLITE_BUSY for writes (if a writer is already holding the lock, and the busy timeout expires before the other writer releases the lock). So yes, prefer short writes, or adjust your timeout. Generally speaking, SQLITE_BUSY can not be 100% prevented for writes.
Yes. Now, many short writes look exactly as one very long write from the point of view of an enqueued write that is waiting for its turn :-) I don't quite remember how fair is SQLite scheduling, in practice.
I don’t think there is any scheduling. Each connection polls to see if the write lock is available up to the max busy timeout setting.
The connection polls at these intervals: static const u8 delays[] = { 1, 2, 5, 10, 15, 20, 25, 25, 25, 50, 50, 100 };
So, if you are using the default 5 second timeout, and you are trying to acquire a lock while an exclusive lock is held, you will wait 1 second, then 2 seconds, then 5 seconds, and timeout. I’m not sure if you timeout after 3 total seconds have elapsed, or sometimes after the 2 and sometimes after the 5.
If you have a thread running many fast queries in a loop you can deny access to another thread that needs a lock. The other thread may get lucky and poll for the lock at the exact moment in between locks from the other thread, but it might not.
We start to see ORMs like Diesel (https://diesel.rs) or GRDB (http://github.com/groue/GRDB.swift) that radically simplify their domain by removing traditional ORM features like implicit uniquing, lazy-loading, or auto-updates. This of course gives more work to do on the client side. But this work is positive and often simpler than expected. With traditional ORMs, there is still some work to do, which often looks like fights: against misunderstandings, against leaky abstractions, against configuration levels that don't match the needs of application, and generally speaking against the framework.
