Even an unpaid report like this involves weeks of work: reading docs, designing tests, executing and refining them, filing issues, writing the report, and editing it. Each report costs thousands of dollars in hardware and and editing. I'd love to do an FDB test! It's just that between contracting, volunteer work, and research, I can only do so much.
What are your most tempting/daunting databases that you haven't got a chance to put through their paces?
And, a second question, if you step back and think about the various APIs you've had to use, have you personally developed favourite styles of API to use?
I'd love to do more work with predicates in general. That's an open research problem I've been noodling on for years. Pretty much any SQL DB would be a good candidate for that work!
I'm gonna be a weirdo and say I actually loved Fauna's FQL. A little Lisp-ish functional language for queries is a great way to interact with document-structured data. SQL is fantastic for sheer breadth, though its specification is a nightmare and actually writing portable SQL is real challenging. One of those places where a stronger spec and conformance tests would have really helped.
Thanks for the shoutout. At some point if you find yourself with some spare time you can check out our new FQL version. It's closer to JS in terms of syntax now, but still a small, relatively functional language.
Your reports have reached such a huge notoriety, i'm surprised you don't already have a dozen of people working for you fulltime on benchmarking techs for wealthy customers.
If you allow me : how many people you work with can actually perform those Jepsen Report ? Or is it only you ?
It's not a huge market. I usually have a queue of clients, but both deal flow and actual scheduling are wildly variable--sometimes I'll go without income for six months or more. I've considered hiring one or two people, but I couldn't offer the kind of stability people need from an employer. I do subcontract though! Editing, legal, finance, occasional code, that sort of thing.
There's a lot of folks out there who can do basic testing work with Jepsen. I've taught... I dunno, maybe a few hundred people directly in Jepsen workshops. A couple people have worked alongside me, and I'm sure lots more have learned from the docs online. Writing a report is a more involved problem--certainly not intractable, but for me it involves testing, experiment design, lots of reading, doc review, writing, editing, finding reviewers, and of course all the business stuff.
> There's a lot of folks out there who can do basic testing work with Jepsen. I've taught... I dunno, maybe a few hundred people directly in Jepsen workshops.
I don't know why all the DBMS vendors don't just have a guy on the QA team whose job is to run and interpret Jepsen tests for every new version. It's certainly a better option than eventually getting a damning report written by you.
Thanks, that's very interesting. I would have assumed that with today's world of distributed database being so common, and with the various technologies available, lots of people would be interested in hiring experts in ensuring database work as advertized. But i guess people trust the product documentation too much.
As an anecdote i was surprised to discover mongodb had a second life in the corporate world as a standard , certified technology to store critical documents. So yeah, maybe people aren't really that aware of the kinds of nasty gotchas that lure in their systems.
