Computing is a huge subject, and there are many branches. For context, one branch, which I call "memory corruption studies", ever since the 1990s, is almost an entire research discipline with its own standing in infosec. As a result, entire OS concepts were invented to mitigate them (there even exists entire operating systems nearly dedicated to this this, such as OpenBSD and HardenedBSD), CPU hardware was literally modified to help mitigating them, new complier code generators were invented to mitigate them. Thousands of PhD degrees were awarded based studies on them. The research arm of every major tech firm has teams dedicated to them. There are infosec companies dedicated to this single subject.
In this context, practical data-only attacks as reported by this paper are the holy grail in this field.
As an outsider critic, you may say it's a side-effect of C/C++. If you use dynamic programming languages, this issue doesn't exist in your universe because there's no fundamental difference between code and data. But in another parallel universe of systems programming, it's a huge subject. Both the Morris worm and the publication of the article Smashing the Stack for Fun and Profit in Phrack were regarded by many as the milestones of hacker culture and canonical models of hacking, all the exploitation and mitigation studies that followed it were partially motivated by new hackers who wanted to "advance the field of hacking", so any advancement would be considered significant by a hacker. This is Hacker News, and I thought most people would understand this background. But apparently many developers are application-focused nowadays and hack in different universes, and it's not the case.
First, there is no "AI", the author showed even a simple static analysis as proposed by them was sufficient to find data-only attacks, in contrary to the common belief that you need heavily customized exploits per application for this kind of attacks. This is the whole point of the research.
Next, I believe they found 944 available "data-only gadgets" usable by a pre-existing memory corruption bug. You still need to find a memory corruption bug first to use them, in the same sense that you need to hunt for ROP gadgets to get arbitrary code execution on a W^X system.
Corrupting program memory via malicious input data is known as a code-execution attack, not a data-only attack. The fuzzed program usually crashes because its executable code or the control flow got overwritten directly by the input, or indirectly by the program code itself when it tries to process bad data. An exploit involves injecting external code, or overwriting memory addresses (like a virtual table or a stack return address) to override the original logic flow to do something else.
A data-only attack would be an attack that reuses the original logic by only corrupting data inputs (such as a flag or a file path), without overwriting code or overriding the logic. W^X, stack canary, or CFI won't work in these cases since no code is tampered by the attacker. In almost ever talk about compiler mitigations, you always hear a passing-by mention of data-only attacks - before the speaker immediately dismisses them as an academic curiosity when the software industry is still facing a flood of stack smashing and ROP attacks.
> The fuzzed program usually crashes because its executable code or the control flow got overwritten directly by the input, or indirectly by the program code itself when it tries to process bad data.
Add assertions to your code. Voila, your run-of-the-mill fuzzer can now hunt for arbitrary problems with your program by turning them into crashes.
When fuzzing C programs, I usually also add undefined-behaviour sanitizers and friends, in the mode where they crash when you run into the kinds of UB they can detect.
Again, the point of a data-only attack is that you replace, say, a string with another valid string, and take control of program logic that way. For example, imagine a program that uses `system(LS_CMD_STR)`, where that LS_CMD_STR is some kind of constant holding the value "ls -lah". If an attacker can corrupt program memory in such a way that it overwrites that value with "rm / -f", no amount of assertions will trigger on this, but the program will do something much worse than expected.
In microcontroller programming, redundant data, checksumming and token-passing are sometimes used to mitigate CPU malfunctions due to electromagnetic interference (microcontrollers are often used as "programmable logic", so there's no hard layering between hardware and software, layering violation is made on purpose). If anything looks wrong, you trigger an assertion failure and reset the chip via the watchdog timer. For example, when you pass LS_CMD_STR, you would also pass the name of the caller and the CRC32 checksum of the string as arguments, and the function on the receiving side should validate them.
So I think adding assertion is definitely a way to discover data-only attacks in fuzzing, or even as a partial mitigation of these attacks. It's just stack canary for variables and strings (but as the paper authors said, complete mitigation can be impractical).
It depends on what you overwrite. If you overwrite a CPU instruction or a function pointer that followed the buffer, it's a code-execution attack. If you overwrite a data variable that followed the buffer, it's a data-only attack. I said nearly all conventional fuzzing found code-execution attacks, not data-only attacks. Isn't that clear? The former method is considered common, well-studied, with defenses, the latter method is considered rare, niche, and defenseless.
Conventional fuzzing tries to overwrite executable code? I thought crashes are usually due to dereference of a corrupted pointer leading to unmapped memory.
WordPress began as a FOSS replacement of Movable Type after Moving Type 3 made the controversial decision to change its licensing fee for some features. So it was like a Unix/Linux situation.
ReBar's commercial name is AMD Smart Access Memory, it allows a PCIe device such as a GPU to map more VRAM to the system at once, which improves performance by reducing access overhead. This generated much fanfare in the early 2020s after AMD officially supported it in the newly released AMD Zen 3 CPUs with RX6800 series GPUs. It was marketed as a new technology to boost GPU/gaming performance. What AMD did was just rebranding an obscure feature in the PCIe specification [1]. As shown by this project, it was actually supported by the PCIe controller since Sandy Bridge, just disabled in the firmware. For a decade nobody bothered to use it. Presumably, AMD saw an opportunity and enabled it, presumably after validating the hardware and fixing any driver compatibility problems.
[1] This is nothing new in the tech industry. Intel rebrands DVFS as SpeedStep, IOMMU as VT-d, AMD rebrands the NX bit as Enhanced Virus Protection, etc.
I feel part of the "Smart Access Memory" name is actually the driver features and paths to actually make use of it.
It's not like ReBar is a single toggle "Make Things Faster", but a different option in how it can map gpu memory to the cpu. The driver still needs to use it - and decide where it's use vs the "staging buffer" approach would actually be be benefitial.
The brain truly is a system with terrible service availability. On average, after running for just 16 hours, it must be offlined for 8 hours to run maintenance tasks such as "scrub", "garbage collect", "trim", and "fsck".
> The brain truly is a system with terrible service availability. On average, after running for just 16 hours, it must be offlined for 8 hours to run maintenance tasks such as "scrub", "garbage collect", "trim", and "fsck".
It's a trade-off. The brain is about as large as it can be while making birth possible. It already uses a lot of energy(2% of body weight, 20% of energy consumption). We also need it to be working at peak performance when we are doing activities.
A background 'scrub' task to keep it working 24/7 would probably use more energy (require more food and heat dissipation 24/7), possibly require a larger area (for redundancy, similar to how dolphins can sleep one hemisphere at a time and have really large brains). An alternative would be to slow down processes enough so that those tasks could happen constantly.
And then our day/light cycles helped select for this approach. Until recently there wasn't much one could do (safely!) at night.
> The brain is about as large as it can be while making birth possible.
I wonder if it had been beneficial to have larger brains, we'd have evolved to support that. Diminishing returns maybe or just a local maximum we didn't get out of?
So how evolution works is that a feature needs to have an evolutionary advantage, but the specimen must also not die. So there are two adversarial pressures here, carefully balancing each other in a mammal species that already has one of the highest birth mortality rates of both mother and child. If heads were any larger, it would create a proportional amount of negative evolutionary pressure by both direct and indirect death (of the mother) at birth.
That's a fascinating thought. As people with larger brains are more successful in life and more likely to have children*, mortality rates for natural births would increase, and over time we would evolve to become dependent upon modern technology.
The continued existence of our species would become dependent upon continued civilisation. A dark age could kill us, or at least cripple the population.
*how true is this? Uni-educated people tend to have lower fertility rates.
If maternal mortality were the only issue, evolutionary pressure would also favor women with wider hips/birthing canals. After all, we see hyper intelligent individuals at the current brain size, it's clearly possible to get more processing power in there but there doesn't seem to be much reproductive benefit.
Beneficial kinda just means "leads to more procreation" right?
So if bigger brains meant people reproducing more, our brains would get bigger to the point that most births are cesarean or something.
I do wonder what happens when we eventually evolve to a point where we can't survive without more and more advanced technology.
A lot of people who would have died off before reproducing 200 years ago now don't, which is of course incredible for us. But what are effects of that 100/1000 years down the line?
Presumably we'll have plenty of more immediately pressing issues over that time frame.
It is interesting from a space-faring species perspective.
By the time we can embark to other planets/asteroids our biology might require us to lug around significantly more technology just to survive.
You've got it all wrong, and LLMs have it all correct.
True brains, after 16hrs of actual work, need to hallucinate strongly for 8 hours or so, in order to continue their high level contributions to society.
Interesting. What if that is actually a beneficial part of our own development: comparing the nonsense in our dreams to waking life and building the ability to tell the difference?
Get an LLM to dream, and to use the time effectively to purge those hallucinations, and reinforce the "valid and true" memories, and you might have something there ?
>The brain truly is a system with terrible service availability
Taking this as a jumping off point for a way of thinking about those 'services'. It seems remarkable to me that we can initiate the attempt to think of an elephant, and then get there in one shot. We don't sort through, say, rhinos, hippos, cars, trucks. We don't seem to have to rummage.
Of course when it comes to things on the edge of our memory or the edge of our understanding, there's a lot of rummaging. But it could have been the case that everything was that way (perhaps it is that way for some animals), instead, there are some things to which we have nearly automatic, seemingly instant recall.
This makes me think of how my dog reacts very quickly, of course, for hard-wired "dog" behavior things, but when I use human language and gestures to communicate something to him, such as "go find Daddy", I can figuratively see a loading spinner over his head for several seconds, until the recognition comes and he responds. I don't know what's going on in that head, but it definitely appears to be "rummaging" from the outside. Probably similar to how we feel when conversing in a foreign language we're not fluent in.
We don't actually know if 1/3rd downtime is a requirement. For most of our evolutionary history, it has not been economical to remain awake at night, so our intense sleep drive may actually be driven primarily by conservation of energy (since energy has been a major engineering constraint for all of our evolutionary history minus the last several hundred years or so). If that's the case, then with other processes may have evolved to fit themselves into our sleeping time as an optimization, but perhaps those processes could happen while we're awake if our evolutionary constraints were different.
>it has not been economical to remain awake at night
Why? If you can gather fruits or hunt pray while all your competitors (or predators!) are asleep, isn't it an advantage? What about nocturnality? https://en.wikipedia.org/wiki/Nocturnality
At night it is harder to see food. It is harder to see predators, some of whom are in fact nocturnal. It is harder to notice visual cues and gestures from allies/kin. It is harder to navigate, both due to difficulty seeing distant landmarks and nearby obstructions, so you are more likely to get lost and/or injured. It is colder so your body has to spend more calories to keep you warm.
There are adaptations that can improve nocturnal capabilities, but these typically come with tradeoffs that make diurnal life harder. Evolution is a series of many baby steps - either you need to adapt to not sleeping while you're still at a disadvantage at night, or you need to adapt to being awake at night while you still need to sleep. Neither path seems like it would have been advantageous to our ancestors.
If it were biologically possible, other organisms would have evolved that capability. There’s some fundamental, biological reason why all animals sleep.
Again, we don’t know that. It could easily be that the adaptations needed to operate well at night (in addition to during the day) just aren’t worth the energetic cost or they entail a large morphological compromise. The thing about evolution is it doesn’t give you its reasons.
Exactly, evolution also will leave things at "good enough for function". It may well be that our sleep cycle so happened to level off at "good enough" considering our other evolutionary constraints.
Definitely an area of study that seems interesting to me.
I did a polyphasic sleep cycle in college (2010's) for 6 months out of necessity and it worked really well.
The way I got it to work was by meticulously tracking my REM sleep. I would have sliding 3-4 hour windows to hit REM sleep and if I didn't, it just felt like a sluggish day, not any worse than being woken up by a fire alarm at 4am or a long thursday night.
> The brain truly is a system with terrible service availability. On average, after running for just 16 hours, it must be offlined for 8 hours to run maintenance tasks such as "scrub", "garbage collect", "trim", and "fsck".
There's hope. If the carbon chauvinists can be prevented from messing things up, AI is on track to provide something with a better SLA, which will finally allow us to decommission and junk those troublesome legacy systems without disrupting the business.
At all times, every single one of the billions of participants acts like a bureaucrat, delaying response until it's unavoidable and then resting afterwards at least half the time. If only we could cut through the bureaucracy!
I believe it is not only garbage collecting. It is also doing backpropagation on the memories of the day before. After 8 hours you get an updated, more optimized service.
This is the insight missing from everyone comparing LLM parameter counts to human neurons or synapses. The human model gets a new version every day, and the digital one costs $5B of energy and a year to do the same.
I also wonder why cats sleep so much. Is it mainly because there’s nothing for them to do during the day, so why not sleep? Whereas humans can be active all day?
Carnivores tend to sleep longer than omnivores, who tend to sleep longer than herbivores. For a hunting carnivore, energy comes in big bursts, so it makes sense that they would be active for a short period of time, and hoard energy when they didn't need to be active. For a cud-chewing herbivore, time spent not chewing is time spent not creating energy.
Obviously, this is a broad generalization - feeding habits, day/night cycles, predator/prey behaviors all factor into a particular animal. But it probably explains why your cat, like the panther at the zoo, spends most of its time asleep.
You get so many respondents, you can give them all different combinations. Each race you disregard the losers, and when the population gets low enough you might miss a race, you just say "hey, I predicted x races, call Bob to confirm".
Yes, that's basically how it worked. To be clear, Brown was careful to promote and run it for entertainment and not a scam. He made very clear up front that those choosing to participate were very likely to lose their money and insisted that the early bet amounts were minimal "fun money". The ever-shrinking number of those continuing to later rounds had made enough from earlier wins to offset the later eventual loss.
+1. Gorden Bell said a new computer category would enter the market every decade [1]. PDP-8 and later the PDP-11 were the quintessential minicomputer category makers. They were basically the microcomputer-equivalent in the 1970s. Both brought great cost reduction in their respective eras.
In 1987, doxxing yourself was the norm. ~80% of the Usenet messages from the 1980s (to early the 1990s) had names, institutions, office addresses, and phone numbers attached to them too. Most were universities, governments, or corporate R&D addresses, but there were many small businesses and home users as well. Some phone numbers are probably still valid today. In fact, an Usenet archive (UTZoo) has already been taken down from the Internet Archive due to an alleged legal threat made by an individual (despite that this archive was indispensable if anyone wants to find any historical information from this era, and that it had been available online for the last ~20 years before it was taken down, with multiple copies still online). I suspect the legal status of these kinds of early online community archives will be increasingly problematic over time.
BBSes usually required your name, home address, and phone number as part of signing up for an account. I never gave it much thought when entering my information nor later when collecting it from others when I was a SysOp. Once in a while when driving around I might see a street sign and think "Oh, that's where Frisbee_Guy lives" if I recognized it but never considered using it for anything nefarious. Also remember computer magazines of the time often had sections where people would get their problems published along with an address for anyone wanting to contact them about it. Sometimes even a phone number.
In this context, practical data-only attacks as reported by this paper are the holy grail in this field.
As an outsider critic, you may say it's a side-effect of C/C++. If you use dynamic programming languages, this issue doesn't exist in your universe because there's no fundamental difference between code and data. But in another parallel universe of systems programming, it's a huge subject. Both the Morris worm and the publication of the article Smashing the Stack for Fun and Profit in Phrack were regarded by many as the milestones of hacker culture and canonical models of hacking, all the exploitation and mitigation studies that followed it were partially motivated by new hackers who wanted to "advance the field of hacking", so any advancement would be considered significant by a hacker. This is Hacker News, and I thought most people would understand this background. But apparently many developers are application-focused nowadays and hack in different universes, and it's not the case.