The Spring RTS FOSS engine is an engine, but does not provide the game assets or game-specific code.
Recoil is a fork of the Spring engine (background: Spring made backward incompatible changes; Recoil forked to retain backward compatibility). Beyond All Reason uses the Recoil engine and supplies its own game code, shaders, and assets.
I applaud the work that’s been done on Dingo (I also really like the name and inspiration, i.e. Dingo is a language that broke free from Google’s control). However, I don’t think Dingo is Typescript for Go, because it is too limited in scope.
Dingo adds Sum types and associated pattern matching elimination thereof, it adds a ‘?’ syntax for propagation of Optional types, and exhaustiveness checking for those pattern matching statements. There is no type system expansion, or syntax alterations that would make the Typescript comparison more appropriate.
I think Dingo probably addresses a lot of the common complaints with Go, but it is not nearly as far from Go as a baseline as I would assume a language positioned between Go and Rust.
Amundsen has two databases and three services in its architecture diagram. For me, that's a smell that you now have risk of inconsistency between the two, and you may have to learn how to tune elasticsearch and Neo4j...
Versus the conceptually simpler "one binary, one container, one storage volume/database" model.
I acknowledge it's a false choice and a semi-silly thing to fixate on (how do you perf-tune ingestion queue problems vs write problems vs read problems for a go binary?)..
But, like, I have 10 different systems I'm already debugging.
Adding another one like a data catalog that is supposed to make life easier and discovering I now have 5-subsystems-in-a-trenchcoat to possibly need to debug means I'm spending even more time on babysitting the metadata manager rather than doing data engineering _for the business_
For me, it was Elixir pattern matching + GenServers + SupervisionTrees + Let It Fail mentality.
I've fought null pointer exceptions longer than I want to in C# servers and Python code. Usually because we're introspecting into a message that a system sent us, we expect it to have a field on the message, we failed to check for the existance of that field / the field to not be null, and bam null-pointer exception.
Sure, add another guard clause... Every time you run into it. Every time the server crashes and it's a serious problem and someone forgot to set a restart policy on the container... Sometimes you don't know exactly what cascade of messages caused the whole thing to unwind. So tedious. So repetitive.
With GenServers, you've carefully defined that those parts of your system have active, in-memory state, and everything else must be a pure function. Messages come in a FIFO queue to your GenServer.
With pattern-matching, you define exactly what shape of message / fields on the message, and if it doesn't match that, you can either log it, or crash that GenServer.
Which normally would be catastrophic, but since you tied your GenServer to a SupervisionTree with a restart policy (which is trivial to write), if anything happens that you didn't expect, you can Let It Fail. The GenServer crashes and is immediately restarted, with a clean slate, with the correct peer-services in the correct state. Sure, the message that crashed your server was "lost" (there are options to push it to a dead-letter-queue on exit) and you have a bad-state and bad-message to debug in the crash-dump, but your server is still going. It hasn't stopped, it just dropped unexpected messages and has a deterministic recovery pattern to clear all bad-state and start again.
So instead of your message handling code starting with a bunch of defensive sanity checks, with the real meat of the function in the last 5 lines.... you just say "My GenServer is here in this SupervisorTree, I expect an inbound message to look roughly like this, and I will do this with it, done". Suddenly the server will handle the messages it knows how to handle, and everything it cannot handle it will drop on the floor.
Think of this! The server just stays up and keeps going! Anything it cannot handle is not a fatal exception, but a log message. And you didn't have to bend the code into an unnatural shape where you trapped and logged all exceptions and bubbled them up in specific error checks... it's just designed to do it that way out of the box.
Neato! I might just self-host this at home and explore using it for my 3d printing needs...
Declarative constructed solid geometry sounds like how OpenSCAD works. I was curious if you took any inspiration from that project, or if you had found it but didn't suit your needs for some reason...
A friend of mine who ran a print farm actually told me about OpenSCAD when I shared a screenshot of my first design (a ball joint with armature). So I didn't take inspiration, but I plan on learning it just to figure out how they handle things like fillets. Because currently my fillets are blowing up. I contemplated just faking the fillets using an extrusion with a cylinder cut out of it, but if I can define edges in code and fillet them that would be better.
They have to be done manually, usually using the Minkowski feature iirc.
There's another similar tool called implicitcad that handles them better (it's also the only useful piece of software written in Haskell I've ever encountered) https://implicitcad.org/
> So I didn't take inspiration, but I plan on learning it just to figure out how they handle things like fillets. Because currently my fillets are blowing up.
They don't. So save yourself that trouble. You design the fillets right into the extrusions doing them after the fact is prohibitively expensive.
I keep thinking I have a possible use case for property -based testing, and then I am up to my armpits in trying to understand the on-the-ground problem and don't feel like I have time to learn a DSL for describing all possible inputs and outputs when I already had an existing function (the subject-under-test) that I don't understand.
So rather than try to learn to black boxes at the same time , I fall back to "several more unit tests to document more edge cases to defensibly guard against"
Is there some simple way to describe this defensive programming iteration pattern in Hypothesis? Normally we just null-check and return early and have to deal with the early-return case. How do I quickly write property tests to check that my code handles the most obvious edge cases?
> [...] time to learn a DSL for describing all possible inputs and outputs when I already had an existing function [...]
You don't have to describe all possible inputs and outputs. Even just being able to describe some classes of inputs can be useful.
As a really simple example: many example-based tests have some values that are arbitrary and the test shouldn't care about them, like eg employees names when you are populating a database or whatever. Instead of just hard-coding 'foo' and 'bar', you can have hypothesis create arbitrary values there.
Just like learning how to write (unit) testable code is a skill that needs to be learned, learning how to write property-testable code is also a skill that needs practice.
What's less obvious: retro-fitting property-based tests on an exiting codebase with existing example-based tests is almost a separate skill. It's harder than writing your code with property based tests in mind.
---
Some common properties to test:
* Your code doesn't crash on random inputs (or only throws a short whitelist of allowed exceptions).
* Applying a specific functionality should be idempotent, ie doing that operation multiple times should give the same results as applying it only once.
* Order of input doesn't matter (for some functionality)
* Testing your prod implementation against a simpler implementation, that's perhaps too slow for prod or only works on a restricted subset of the real problem. The reference implementation doesn't even have to be simpler: just having a different approach is often enough.
But let's say employee names fail on apostrophe. Won't you just have a unit test that sometimes fail, but only when the testing tool randomly happens to add an apostrophe in the employee name?
Hypothesis keeps a database of failures to use locally and you can add a decorator to mark a specific case that failed. So you run it, see the failure, add it as a specific case and then that’s committed to the codebase.
The randomness can bite a little if that test failure happens on an unrelated branch, but it’s not much different to someone just discovering a bug.
You can either use the @example decorator to force Hypothesis to check an edge case you've thought of, or just let Hypothesis uncover the edge cases itself. Hypothesis won't fail a test once and then pass it next time, it keeps track of which examples failed and will re-run them. The generated inputs aren't uniformly randomly distributed and will tend to check pathological cases (complex symbols, NaNs, etc) with priority.
You shouldn't think of Hypothesis as a random input generator but as an abstraction over thinking about the input cases. It's not perfect: you'll often need to .map() to get the distribution to reflect the usage of the interface being tested and that requires some knowledge of the shrinking behaviour. However, I was really surprised how easy it was to use.
As far as I remember, hypothesis tests smartly. Which means that possibly problematic strings are tested first. It then narrows down which exact part of the tested strings caused the failure.
So it might as well just throw the kitchen sink at the function, if it handles that: Great, if not: That string will get narrowed down until you arrive at a minimal set of failing inputs.
> Which means that possibly problematic strings are tested first.
Hypothesis uses a the same probability distribution for all the 200 (or so) random cases it generates for a test. The first case has the same distribution as the 200th.
However, Hypothesis gives a pretty large weight in the probability distribution to inputs that are generally 'problematic'. Of course, that's just a heuristic: eg empty lists and 0 and empty strings or strings with apostrophes in them and NaN or infinity often are problematic, but that's just a guess: hypothesis doesn't know anything specific about your code.
The heuristics work remarkably well in practice, though.
Once Hypothesis has found a failing test case, then it tries to shrink it down.
If you know it will fail on apostrophe you should have a specific test for that. However if that detail is burried in some function 3 levels deep that you don't even realize is used you wouldn't write the test or handle it even though it matters. This should find those issuses too.
> But let's say employee names fail on apostrophe. Won't you just have a unit test that sometimes fail, but only when the testing tool randomly happens to add an apostrophe in the employee name?
If you just naively treat it as a string and let hypothesis generate values, sure. Which is better than if you are doing traditional explicit unit testing and haven’t explicitly defined apostrophes as a concern.
If you do have it (or special characters more generally) as a concern, that changes how you specify your test.
Either your code shouldn’t fail or the apostrophe isn’t a valid case.
In the former, hypothesis and other similar frameworks are deterministic and will replay the failing test on request or remember the failing tests in a file to rerun in the future to catch regressions.
In the latter, you just tell the framework to not generate such values or at least to skip those test cases (better to not generate in terms of testing performance).
I think what they meant is, "won't Hypothesis sometimes fail to generate input with an apostrophe, thus giving you false confidence that your code can handle apostrophes?"
I think the answer to this is, in practice, it will not fail to generate such input. My understanding is that it's pretty good at mutating input to cover a large amount of surface area with as few as possible examples.
Hypothesis is pretty good, but it's not magic. There's only so many corner cases it can cover in the 200 (or so) cases per tests it's running by default.
But by default you also start with a new random seed every time you run the tests, so you can build up more confidence over the older tests and older code, even if you haven't done anything specifically to address this problem.
Also, even with Hypothesis you can and should still write specific tests or even just specific generators to cover specific classes of corners cases you are worried about in more detail.
> But by default you also start with a new random seed every time you run the tests, so you can build up more confidence over the older tests and older code
Is it common practice to use the same seed and run a ton of tests until you're satisfied it tested it thoroughly?
Because I think I would prefer that. With non-deterministic tests I would always wonder if it's going to fail randomly after the code is already in production.
You can think of property based tests as defining a vast 'universe' of tests. Like 'for all strings S and T, we should have to_upper(S) + to_upper(T) == to_upper(S+T)' or something like that. That defines an infinite set of individual test cases: each choice for S and T gives you a different test case.
Running all of these tests would take too long. So instead we take a finite sample from our universe, and only run these.
> With non-deterministic tests I would always wonder if it's going to fail randomly after the code is already in production.
You could always take the same sample, of course. But that means you only ever explore a very small fraction of that universe. So it's more likely you miss something. Remember: closing your eyes doesn't make the tiger go away.
If there are important cases you want to have checked every time, you can use the @example decorator in Hypothesis, or you can just write a traditional example based test.
the more the test runs the less likely it is there is an uncovered case left. So your confidence grows. Remember too anything found before release is something a customer won't find.
Yes, you should note the seed you use for a run in the logs, but you should use a new seed each run (unless trying to reproduce a known bug) so you cover more of the search space.
Hypothesis is a search algorithm for finding ways that we have misunderstood our code (whether that be in the codebase, or the claims we have made about it in the spec/tests). Conceptually, that's the inverse of "a unit test that sometimes fails": it's a search that sometimes succeeds at finding out that we're wrong. That's infinity% more effective than a search procedure which is hard-coded to check a single example; especially when that example is chosen by the same dev (with the same misconceptions!) that implemented the code-under-test!
Now, as for what would actually happen in that situation, when using Hypothesis:
- As others have indicated, Hypothesis keeps a database of the failures it's found. Committing that database to your project repo would act like a ratchet: once they've successfully found a problem, they will continue to do so (until it's fixed).
- If you don't want to commit your database (to avoid churn/conflicts), then the same would happen per working-copy. This is fine for dev machines, where repo directories can live for a long time; though not so useful for CI, if the working-copy gets nuked after each run.
- Even if we disable the failure database, property failures will always show the inputs which caused it (the "counterexample"), along with the random seed used to generate it. Either of those is enough to reproduce that exact run, giving us a ready-made regression test we can copy/paste to make the failure permanent (until it's fixed). As others have said, Hypothesis properties can be decorated with `@example(foo)` to ensure a particular input is always checked.
- Even if we disable the failure database, and don't copy the counterexample as a regression test, property checkers like Hypothesis will still "shrink" any counterexamples they find. Your example of apostrophes causing problems is trivial for Hypothesis to shrink, so if/when it happens to find some counterexample, it will always manage to shrink that down to the string "'"; essentially telling us that "employee names fail on apostrophe", which we could either fix right now, or stick in our bug tracker, or whatever.
Sibling comments have already mentioned some common strategies - but if you have half an hour to spare, the property-based testing series on the F# for Fun and Profit blog is well worth your time. The material isn’t really specific to F#.
The simplest practical property-based tests are where you serialize some randomly generated data of a particular shape to JSON, then deserialize it, and ensure that the output is the same.
A more complex kind of PBT is if you have two implementations of an algorithm or data structure, one that's fast but tricky and the other one slow but easy to verify. (Say, quick sort vs bubble sort.) Generate data or operations randomly and ensure the results are the same.
> The simplest practical property-based tests are where you serialize some randomly generated data of a particular shape to JSON, then deserialize it, and ensure that the output is the same.
Testing that f(g(x)) == x for all x and some f and g that are supposed to be inverses of each other is a good test, but it's probably not the simplest.
The absolute simplest I can think of is just running your functionality on some randomly generated input and seeing that it doesn't crash unexpectedly.
For things like sorting, testing against an oracle is great. But even when you don't have an oracle, there's lots of other possibilities:
* Test that sorting twice has the same effect as sorting once.
* Start with a known already in-order input like [1, 2, 3, ..., n]; shuffle it, and then check that your sorting algorithm re-creates the original.
* Check that the output of your sorting algorithm is in-order.
* Check that input and output of your sorting algorithm have the same elements in the same multiplicity. (If you don't already have a datastructure / algorithm that does this efficiently, you can probe it with more randomness: create a random input (say a list of numbers), pick a random number X, count how many times X appears in your list (via a linear scan); then check that you get the same count after sorting.
* Check that permuting your input doesn't make a difference.
Speaking for myself — those are definitely all simpler cases, but for me I never found them compelling enough (beyond the "it doesn't crash" property). For me, the simplest case that truly motivated PBT for me was roundtrip serialization. Now I use PBT quite a lot, and most of them are either serialization roundtrip or oracle/model-based tests.
> The absolute simplest I can think of is just running your functionality on some randomly generated input and seeing that it doesn't crash unexpectedly.
For this use case, we've found it best to just use a fuzzer, and work off the tracebacks.
That being said, we have used hypothesis to test data validation and normalizing code to decent success. We use on a one-off basis, when starting something new or making a big change. We don't run these tests everyday.
Also, I don't like how hypothesis integrates much better with pytest than unittest.
For working with legacy systems I tend to start with Approval Tests (https://approvaltests.com/). Because they don't require me to understand the SUT very well before I can get started with them, and because they help me start to understand it more quickly.
I think the easiest way is to start with general properties and general input, and tighten them up as needed. The property might just be "doesn't throw an exception", in some cases.
Thanks, the "does not throw an exception" property got my mental gears turning in terms of how to get started on this, and from there I can see how one could add a few more properties as one goes along.
Yes. One of the first things you have to do when writing any property based tests, no matter what you actually want to test, is defining your input generators. And once you have those generators, you might as well throw together the "doesn't crash unexpectedly" test together.
That not only tests your code, but also exercises your just written input generators.
I've only used it once before, not as unit testing, but as stress testing for a new customer facing api. I wanted to say with confidence "this will never throw an NPE". Also the logic was so complex (and the deadline so short) the only reasonable way to test was to generate large amounts of output data and review it manually for anomalies.
The simplest ones to get started with are "strings", IMO, and also gives you lots of mileage (because it'll definitely test some weird unicode). So, somewhere in your API where you take some user-entered strings -- even something "open ended" like "a name" -- you can make use of Hypothesis to try a few things. This has definitely uncovered unicode bugs for me.
The only real downside is that a Hypothesis-heavy test-suite like the above can take a while to run (but you can instruct it to only produce one example per test). Obviously, one example per test won't catch everything, but is way faster when developing and Hypothesis remembers "bad" examples so if you occasionally do a longer run it'll remember things that caused errors before.
Yes, when I saw eg Golang people use table driven tests like this, I was wondering why nobody seems to have told them about generating these tables automatically..
When I'm "up to my armpits in trying to understand the on-the-ground problem", I find PBT great for quickly find mistakes in the assumptions/intuitions I'm making about surrounding code and helper functions.
Whenever I find myself thinking "WTF? Surely ABC does XYZ?", and the code for ABC isn't immediately obvious, then I'll bang-out an `ABC_does_XYZ` property and see if I'm wrong. This can be much faster than trying to think up "good" examples to check, especially when I'm not familiar with the domain model, and the relevant values would be giant nested things. I'll let the computer have a go first.
Recoil is a fork of the Spring engine (background: Spring made backward incompatible changes; Recoil forked to retain backward compatibility). Beyond All Reason uses the Recoil engine and supplies its own game code, shaders, and assets.
https://recoilengine.org/articles/choose-recoil/
https://recoilengine.org/ (list of games powered by Recoil)
https://github.com/beyond-all-reason
Source: been in the BAR Discord for about a year, have contributed tiny bits of server code to the project, and read a few pull request comments.
reply