While the author mentions that he just doesn't have the time to look at all the databases, none of the reviews of the last few years mention immutable and/or bi-temporal databases.
Which looks more like a blind spot to me honestly. This category of databases is just fantastic for industries like fintech.
What we do is range types for when a row applies or not, so we get history, and then for 'immutability' we have 2 audit systems, one in-database as row triggers that keeps an on-line copy of what's changed and by who. This also gives us built-in undo for everything. Some mistake happens, we can just undo the change easy peasy. The audit log captures the undo as well of course, so we keep that history as well.
Then we also do an "off-line" copy, via PG logs, that get shipped off the main database into archival storage.
People are slow to realize the benefit of immutable databases, but it is happening. It's not just auditability; immutable databases can also allow concurrent reads while writes are happening, fast cloning of data structures, and fast undo of transactions.
The ones you mentioned are large backend databases, but I'm working on an "immutable SQLite"...a single file immutable database that is embedded and works as a library: https://github.com/radarroark/xitdb-java
The fastest approach is to just zero out the data. Alternatively you can rebuild the entire database while preserving only the data accessible in the latest copy of the db (kinda similar to SQLite's VACUUM command).
I see people bolting temporality and immutability onto triple stores, because xtdb and datomic can't keep up with their SPARQL graph traversal. I'm hoping for a triple store with native support for time travel.
XTDB addresses a real use-case. I wish we invested more in time series databases actually: there's a ton of potential in a GIS-style database, but 1D and oriented around regions on the timeline, not shapes in space.
That said, it's kind of frustrating that XTDB has to be its own top-level database instead of a storage engine or plugin for another. XTDB's core competence is its approach to temporal row tagging and querying. What part of this core competence requires a new SQL parser?
I get that the XTDB people don't want to expose their feature set as a bunch of awkward table-valued functions or whatever. Ideally, DB plugins for Postgres, SQLite, DuckDB, whatever would be able to extend the SQL grammar itself (which isn't that hard if you structure a PEG parser right) and expose new capabilities in an ergonomic way so we don't end up with a world of custom database-verticals each built around one neat idea and duplicating the rest.
I'd love to see databases built out of reusable lego blocks to a greater extent than today. Why doesn't Calcite get more love? Is it the Java smell?
> it's kind of frustrating that XTDB has to be its own top-level database instead of a storage engine or plugin for another. XTDB's core competence is its approach to temporal row tagging and querying. What part of this core competence requires a new SQL parser?
Many implementation options were considered before we embarked on v2, including building on Calcite. We opted to maximise flexibility over the long term (we have bigger ambitions beyond the bitemporal angle) and to keep non-Clojure/Kotlin dependencies to a minimum.
Btw Datomic is free now that Nubank supports it (and runs a large bank on it).
There's also a fantastic kind of mini, FOSS, file-based Datomic-style Datalog DB that's not immutable called Datalevin. Uses the hyper-fast LMDB under the hood. It's called Datalevin. https://github.com/juji-io/datalevin
Destructive operations are both tempting to some devs and immensely problematic in that industry for regulatory purposes, so picking a tech that is inherently incapable of destructive operations is alluring, I suppose.
I would assume that it's because in fintech it's more common than in other domains to want to revert a particular thread of transactions without touching others from the same time.
Most of the musings on enforcing invalid states with ADTs are impractical when working with real data that needs runtime validation when it enters the system.
Using a schema/spec/type library at runtime that is even more powerful than ADTs is a better investment - or to be less controversial - is an additional investment on top of types.
Yes, it means the compiler can't help you as much, but who has time waiting for a compiler anyways ;)
I find the "make illegal state unrepresentable via types" idea great for software that needs to fly a plane, but for enterprise software not so much. The cost/benefit is not justifiable.
Can someone who's adapted HTMX for a larger app report about front-end-server costs?
HTMX serves fully baked HTML that needs to be created on the back-end (or front-end-facing servers)
That is more processing than sending the raw data to the front-end and baking the HTML there.
It is also more bandwidth (unless the JSON is more verbose than the HTML generated).
Lastly, I can generate different HTML fragments on the front-end from the same client-side state with the data only being transported once.
How is that working out?
Not necessarily. Often it is less. Template engines can be very, very efficient and fast.
Returning JSON almost always requires data/object serialization on the server, this is often slower than direct data/object -> template rendering.
Further, it's not your server but keep in mind the client must de-serialize JSON and render some HTML every time.
Modifying layouts as a result of non-persistent state (light/dark mode, sorting, etc) can usually be handled relatively easily with styles and data- attributes and sprinkles of JS. HTMX works very well with Alpine.JS and similar libraries for more complex situations.
HTMX isn't for every scenario, but it works very very well in many scenarios.
AI - as a discipline - has been around forever (1956), essentially since the birth of Lisp - both with staggering successes as well as spectacular failures that ushered in 2(3?) so-called AI winters.
The author probably just means LLMs.
And that's really all you need to know about the quality of this article.
One could get the impression that the only really really important non-functional requirement for such a thing is to absolutely ensure that you can only call the "good" functions with the "good" payload.
If that is just your team, use whatever tech gets you there quick.
However, if you need to provide some guarantees to a second or third party with your API, embrace standards like JSON, even better, use content negotiation.
This is the first wave of a new generation of Apps which has the potential to replace/enhance the majority of e-commerce websites.
I wouldn't underestimate it - it's the hammer can break up the information silos we've built up around websites/apps.
Why prompt Gertrude(tm) on Ramazon for a specific thing you need, if you can ask ChatGPT to find you said thing along with UIs to purchase it across all e-commerce platforms that agree to play in this market?
because the secret is that the web runs on advertising/targeted recommendations. Brezos(tm) wants you to actively browse Ramazon so he can harvest your data, search patterns etc. Amazon and most sites like that are very not crawl friendly for this reason. Why would Brezos let Saltman get all the juicy preference data?
when PayPal tells you that they already know you and don't require you to log in: that's fingerprint.com behind the scenes.
There are pros/cons.
It should be obvious by now that using any free service of scale is being paid for by your interactions which are made more valuable through fingerprinting.
Trying to circumvent that just makes it more expensive for the rest of us.
>when PayPal tells you that they already know you and don't require you to log in: that's fingerprint.com behind the scenes.
Why use a third party service when cookies can do exactly that? They load their .js from the same domain they set up a cookie and there's no limitation to read that cookie, correct?
reply