The novice said: "I will save my working files, but not my system and application files, as they can be always be reinstalled from their distribution disks."
The master made no reply.
The next day, the novice's disk crashed. Three days later, the novice was still reinstalling software.
If that's your situation, it might be worth backing up applications. However, there are better solutions that make this irrelevant. For instance, using NixOS, you can literally back up a single text file describing your system setup, and you can recreate the system from it automatically in minutes, guaranteed to be bit for bit identical.
In my experience that works in box like 80% of the time. Which is great! But a backup is great in case it doesn’t.
If you are wondering ‘How are you getting that much failure’, it’s because I like to freeze older versions and they often decay for one reason or another (can’t get files for example)
The single file bit-by-bit replication is only possible if those bits still exist and you have guarantees of it. Those bits are a hidden dependency unless you mirror them. Without a mirror it seems like replication will work most of the time, but will eventually fail in ways you don't want.
I mean I could but I find a full drive backup isn’t that much bigger than a more selective backup and whenever I try to back up part of a drive I never grab the right bits.
If your NixOS setup is not of hello-world kind, you typically have sops in it (or another secret management tool) and want to backup your age keys as well. They're necessarily separated from the main config tree so they don't end up leaking to the globally accessible nix store, and in the most typical case they're located in a 600 file outside the git repo. Forgetting to back it up is a classic NixOS gotcha and also a generalized version of this koan.
Wouldn't you just re-encrypt the secrets with a new host key? That's what I usually do (boot once to generate keys, update the secrets, rebuild and switch remotely to the machine)
At the same time it is significantly more "hardware-change" resilient, since the same backed up config file (minus perhaps some very system specific lines that you may or may not have) can bring up a functionally equivalent working system on quite distinct other hardware as well.
Like, you can do the same with any distro and say Puppet or Ansible but only to within that distro packages, any 3rd party stuff (and people use more and more stuff that's 3rd party to the distro) might just have dead link for the download.
It does, actually! Because only the package definitions are needed, and you can get those from the tarball of the relevant nixpkgs repo commit. And since building+installing is the same Nix command as downloading+installing, whether the binary caches still have the old package (or are even still up) is only the difference between a fast download and a slower build. This is different from most package managers, where trying to install anything old means spinning up a complex build environment and learning a bunch of new tools.
Of course, if even the original sources are no longer available, you'd have to change the broken URLs. And if the source has somehow been scrubbed from the Internet you're SOL
If bare metal recovery takes shorter time than reinstalling, then you should make bare metal backups. When bare metal updates, so should your backups.
If you're using virtual machines, make sure to backup the hypervisor config and setup as well. You never know which part of the infrastructure starts to fail first.
If you think this is historical, you have a lot to learn about production servers in the wild! =)
Just as a not-insignificant portion of the world still relies on a handful of ancient AS/400 systems, a pretty big portion of society depends on the good running of hand-configured Windows Server.
You've missed the point. It doesn't matter that you can rebuild the system exactly, the example in question had little to do with how accurate the restores were, it had to do with the time to reinstall.
I too can entirely rebuild many systems simple from kickstarting a new one and applying an ansible playbook. Do I want to? No, that's too slow, and I might be doing this operation for tens or hundreds of systems.
We have multiple levels of backups. We use veeam for VM and physical backups of the entire system to a disk store that's also shipped off-site to an immutable store, but we also run an older system based on rsync which rsyncs the system to directory on a server (sans some data directories) that uses snapshots to keep space usage minimal for the purpose.
In many cases, if we want to restore we can literally just rsync data back to the host, whether one or a few files or most of the system.
I can restore many types of failures within seconds or single digit minutes at most, and the time required is mostly unattended time. Time to restore is often an important and overlooked metric, until you've been bitten by it and realized that an hour or more of work from a person to restore something doesn't scale well in some scenarios, and those are often the most important ones.
But it's not actually slower. When NixOS loads you get a list of all previous states to chose from. Booting to any one of them is just as fast as booting to the most recent one.
Restoring some backup via copying bits around is much slower because you have to wait for those bits to copy, whereas booting to previous configs is just rejiggering references to bits which are already in place and need no manipulation.
Immutability buys you quite a lot here, although some up-front discipline is necessary to take advantage of it, which might be a dealbreaker for some.
That's useful if the system is still up and you can trust the state information, but when you have a disk failure, or there's enough system state corruption that you don't want to rely on that state tracking, you're then going to need something else.
Also, are you pinning your package versions in your config? For every single case of that, is the original package still available? Were they coming from remote sources which may not keep prior versions always available?
Did you build anything from scratch? Did you supply a source or binary archive to install from? Is that source still available, of the exact same version?
There are numerous things you need to worry about when rebuilding from what the state should be based on a description of that state (which may include arbitrary actions), as opposed to restoring from a recorded state that includes everything needed in the recorded data.
The more you can record the better. Should you also record nix configuration state and use that to restore when it makes sense? Sure! If it's simple and easy and doesn't use an inordinate amount of space, why wouldn't you? Just don't confuse nice to have with sufficient based on the specific requirements and goals your backup system needs to meet.
We have entire system images in an immutable remote store, but that's not what we go to first, it's what we go to when our quicker and easier systems are insufficient. If we were running NixOS, we'd probably keep everything we're doing now, and then also just add the NixOS state as a separate thing that's backed up and saved on every change (and in fact, along these lines, we also run etckeeper on every host and track all changes to /etc with daily commits if something is changes but also an automatic commit at the beginning and end of every ansible run we do that applies our host configuration).
More is better. If I want to see configuration changes, I can usually just look in etckeeper on the box, if I want to restore a tree to the exact state it was at a specific point that a backup was made through rsync, I can do that easily enough (or just navigate the directory tree on the backup server, if I just want to see what it was, or diff between different states at different times). If I need to restore the whole system image from a known good backup, I can use Veeam and the local repository for that. If the local repository is dead, I can use the replicated off-site repository. If an attacker has destroyed both those, I go to the immutable off-site backups, which I'm probably doing to use after rebuilding my backup infra from scratch to restore from. Each of these protects from slightly different scenarios, and offers different trade-offs for security and integrity and ease of use.
Sure, you may lose out of the "faster" part if you have to pull from a remote cache (both inputs and outputs) instead of your local one. And maybe just blindly relying on the community maintained caches isn't right for you, so now you're still in a position maintain a cache and back it up through traditional means. It's not a magic bullet, probably doesn't pay off for a lot of use cases.
But what it buys you is a sort of verifiability that you can't really get any other way. Anyone can build any part of the system from its declared inputs and say "hmm, I got a different hash than you, maybe something is up." If you're restoring from an image which somebody has tampered with, then the image becomes the source of truth, rather than the source code, and it gets a lot more difficult to scrutinize its validity because the pool of available scrutinizers is just the people who care about your particular image--that's likely to be a lot smaller than the set of people who care about whatever sources you're relying on.
I don't feel like this is responsive to what I said when taken as a whole. I obviously wasn't just advocating for something because it's faster.
...but, if your goal is really verifiable state, why don't you just use image mode linux with bootc, since it provides much of what you're looking for, except to a higher level. All your arguments for NixOS so far apply to that as well, but there's even more immutability and verifiability.
There's at least one possible point, which is where the income is taxed. If data is not considered valuable, the German branch of company X can send data to its Irish branch, and the profit realized in Ireland (with very low tax rate), while this value was generated in Germany but the German state gets no income. And if Ireland raises its tax rate, send it to another fiscal paradise.
Potentially, if data was accounted as valuable, the income would be accounted for where it is generated, each country gets its fair share, and tax avoidance gets harder.
The route to the Clay problem through a
smooth force, options c and d in Fefferman’s statement of the problem, is the
route Luis and Diego opened and the one Levent and I had quietly chosen to
attack. Almost nobody else I know of was working on it. It is not the direction
one arrives at in a few days by giving a model the problem statement. When I
heard “forced,” it was a bright red flag.
This is much more than the knowledge than the problem can be solved, it's also the specific, non-obvious approach to solving it. That's much more damning for OpenAI, if confirmed.
That's a stretch. The Luis and Diego paper was published in 2023 and is included in every frontier model's training dataset. An AI model could independently choose the same path route as Luis and Diego, without access to Buckmaster and Alpöge’s work.
And the article states "an insane amount of compute had been used," which implies OpenAI brute-forced their way to a solution. I.e. they searched for every paper published on Navier-Stokes and exhaustively attempted every approach. Such an approach would lead them to a solution.
There is not enough information at this time to reach a conclusion. The best option is to wait for statements from both sides, then reevaluate.
> An AI model could independently choose the same path route as Luis and Diego, without access to Buckmaster and Alpöge’s work.
the post you were replying to quotes Buckmaster specifically denying this: "It is not the direction one arrives at in a few days by giving a model the problem statement."
> And the article states "an insane amount of compute had been used," which implies OpenAI brute-forced their way to a solution. I.e. they searched for every paper published on Navier-Stokes and exhaustively attempted every approach. Such an approach would lead them to a solution.
"implies" is a surprising choice of word here. that's certainly one interpretation of "an insane amount of compute had been used". what came to my mind, considering Buckmaster's statement that the AI would not head down this specific path on its own, is, though, that they prompted it in this specific direction and then used an insane amount of compute. this seems consistent as well with these other statements:
> Over the course of the call, as members of their team sent Sebastien corrections and details over their internal chat, it emerged that an entire team had been working on the problem, that this was one of a number of things that was tried, that work had started on the unforced problem, that the team first set the model on easier problems, including Euler (...)
i don't know if i understand what you're saying. but i'm no mathematician. here's the full statement in question once again:
> The route to the Clay problem through a smooth force, options c and d in Fefferman’s statement of the problem, is the route Luis and Diego opened and the one Levent and I had quietly chosen to attack. Almost nobody else I know of was working on it. It is not the direction one arrives at in a few days by giving a model the problem statement. When I heard “forced,” it was a bright red flag.
so, you're saying that "the direction one arrives at in a few days" is (A) "the route through a smooth force, options c and d in Feffermann's statement of the problem", and no more specific than that, i.e. does not necessarily include (B) "the same path route as Luis and Diego" (quoted from the post i was replying to) (which, as i understand, is a subset of A -- directly from Buckmaster's quote: "the route A is is the route Luis and Diego opened")?
but the post i was replying to claims that "An AI model could independently choose the same path route as Luis and Diego, without access to Buckmaster and Alpöge’s work", i.e. that the AI model could independently chose B. but choosing B implies choosing A, since B is a subset of A. and in this case it is irrelevant whether Buckmaster claimed that an AI could not independently choose A or B -- the point which you seem to be contesting.
EDIT: my understanding is that "solving the Navier-Stokes existence and smoothness problem" consists of proving at least 1 of 4 precise statements ("options (a) through (d) of Fefferman's statement of the problem"), and Luis and Diego's work were developments towards a proof of statements (c) and (d), which have been recently further expanded by Alpoge and Buckmaster
if you assume the choice of route is a uniformly distributed random variable, yes. but this assumption does not seem consistent with "Almost nobody else I know of was working on it", from Tristan's quote. nor with "It is not the direction one arrives at in a few days by giving a model the problem statement".
It doesn't really make sense what he said. Everyone expects N-S to have blow-up. If you are going to try to show blow-up, C and D are strictly easier than A and B. So Tristan was not clear about what detail of "route" he is talking about, because what he literally said can not be it.
yeah, it looks like i don't have enough knowledge of this subject to discuss this. but i'd be interested in reading what more knowledgeable people have to say about it -- Tristan's work, how different from others' it was (they claim no one else was following the same path), and how it compares with the AI's
Well, no one arrived in this direction, because no one was sitting and prompting a model. It was 10000 agents working 24/7 for several days, trying millions of different directions.
Buckmaster did also mention, for example, that a team of people was employed to solve the problem, which supports this claim. but that is another claim whose veracity could also be questioned. but at some point we must trust other people, unless we can be satisfied with only believing what we personally see.
(also, IMO, the coincidence of both discoveries in time is pretty suspicious. this one doesn't need you to trust many people i guess)
Please don't say brute forced. It sounds like some form of denial or something. Compute for hard problems drops with models--it just means they threw a huge amount of compute. There's (idk about NS specifically so maybe it's exception) no real way to "brute force" a math proof [ok you can enumerate proofs if you can wait until heat death ]
Sorry for random rant but I don't think these statements help your point
I want to point out that almost all previous AI discoveries in math were made in almost the same way. The ideas were there in the community, but weren't considered mainstream/worth pushing forward. Read Tao's comments on the unit distance problem, for example (sry I can't find a link right now).
OpenAI said there [1]:
> The method by which the problem was solved is also notable. The proof brings unexpected, sophisticated ideas from algebraic number theory to bear on an elementary geometric question.
> Invest in formal verification, fuzzing and property testing, and memory-safe languages. LLMs are good at writing Lean and fuzz tests. I don't care whether you use Go or Rust but for the love of god please don't use C or C++ for new code.
How accepted is this thinking in your respective domains?
A lot, I am only writing C or C++ for new code when it is unavoidable, like existing code bases, bindings or tinkering with runtime implementations that aren't bootstraped.
Mobile platforms, distributed computing have long moved the spotligh away from C and C++, other than language runtimes or existing products from the 90's like SQL servers, and naturally UNIX like underlying OS, which most userspace developers aren't writing new code for.
Naturally there are domains like LLVM/GCC, console game dev, HPC/HFT where they are unavoidable for new code.
Not sufficiently. Companies aren't keen on taking on new ideas unless it pays. Buzzword matchers only match old buzzwords. People in the trenches are too busy to try new stuff and/or don't want to push new things because of management or other obstacles.
Most products are built on top of legacy code after all. You stick to the tech that those contain.
The propaganda police are lying. There is nothing wrong with C/C++, you are just too lazy to handle your own memory, and you accepted that propaganda that "managing your own memory is hard" without even trying.
The idea that this terrible advice floats at all tell you how terrible educations are these days. The idea is ridiculous and yet nobody calls it what it is: it is stupid and those that follow that advice out of fear are dumber than rocks.
The propaganda police are lying. There is nothing wrong with assembly, you are just too lazy to manage your own register allocations and stack layout, and you accepted that propaganda that "manage your own register allocations and stack layout" without even trying.
Moats are obstacles against something you're trying to avoid. It can be competitors coming in, or customers going out. A moat could surround a prison to keep prisoners in. Some companies just choose to treat their customers as prisoners.
Putin is under an arrest warrant from the International Criminal Court for war crimes. Rapes and killings are widely documented as part of the war he started. I don't think the parent comment was overstating the crimes in question in its simile.
The biggest thing wrong is Russia's gold isn't in the EU. It's actually the other way round - Russia has kind of stolen Spain's gold, at least has it and won't return it.
> I don't think the parent comment was overstating the crimes
Said comment was understating the crimes, but also misrepresenting them in context.
Those crimes are exceptional in ferality, not exceptional as abstracted within the difficulties of trust building. (Which is also gained diplomatically, and diplomacy has clearly failed - bilaterally).
The end of that post came out in such a mess, it must be a typo for something - "compatriot" maybe? I like the other standout neologism in there though, I can see it in a dictionary now:
