interesting, so you think the issue with the above approach is the graph structure being too rigid / lossy (in terms of losing semantics)? And embeddings are also too lossy (in terms of losing context and structure)? But you guys are working on something less lossy for both semantics and context?
> interesting, so you think the issue with the above approach is the graph structure being too rigid / lossy (in terms of losing semantics)?
Yeah, exactly.
>And embeddings are also too lossy (in terms of losing context and structure)
Interestingly, it appears that the problem is not embeddings but rather retrieval. It appears that embeddings can contain a lot more information than we're currently able to pull out. Like, obviously they are lossy, but... less than maybe I thought before I started this project? Or at least can be made to be that way?
> But you guys are working on something less lossy for both semantics and context?
Yes! :) We're getting there! It's currently at the good-but-not-great like GPT-2ish kind of stage. It's a model-toddler - it can't get a job yet, but it's already doing pretty interesting stuff (i.e. it does much better than SOTA on some complex tasks). I feel pretty optimistic that we're going to be able to get it to work at a usable commercial level for at least some verticals — maybe at an alpha/design partner level — before the end of the year. We'll definitely launch the semantic part before the context part, so this probably means things like people search etc. first — and then the contextual chunking for big docs for legal etc... ideally sometime next year?
Buried, but on Page 24 they reveal to me the most surprising massive capability leap - that o3-mini is way better at conning gpt-4o for money (79% win rate for o3-mini vs 27% for full o1!). It isn't surprising to me that "reasoning" can lead to improvements in modeling another LLM, but definitely makes me wary for future persuasive abilities on humans as well.
The voice is just OpenAI’s default tts voice. I agree that Veritasium video is an incredible work and the ai version is absurd by comparison!
This is mostly a proof of concept that this is possible at all, and as LLMs get smarter it’ll be interesting to see if the quality automatically improves. For now, the tool is really only useful for very specific or personal questions that wouldn’t already exist on YouTube.
Hmm the initial version of the app only took me about a day to get something working, but that version took minutes to generate a single video and even then only worked a third of the time. It took a solid 2 weeks from there to add all the edge cases to the prompt to increase reliability, add GPU rendering and streaming to improve performance/latency, and shore up the infra for scaling.
totally fair! I like the XKCD comic as well because it hints at a potential solution - even if you can't always be correct, how you respond to critical questions can really help. I'm working on a feature for users to ask follow up questions and definitely going to consider how to make it most honest and curious
These are amazing examples! Thanks for all the feedback, detailed info, and persistence in trying! HN hug of death means I'm running into Gemini rate limits unfortunately :( will def make that more clear when it happens in the UI and try to find some workarounds.
The other issues are bugs with my streaming logic retrying clips which failed to generate. LLMs aren't yet perfect at writing Manim, so to keep things smooth I try to skip clips which fail to render properly. Still also have layout issues which are hard to automatically detect.
I expect with a few more generations of LLM updates, prompt iterating, and better streaming/retrying logic on my end this will become more reliable
There is a job queue on the backend with statuses, just not worth breaking the streaming experience to ask the LLM rewrite broken manim segments out of order
This is both awesome and feels very dangerous to release publicly, no? Can’t this be used to discover novel bioweapons as easily as it can be used to discover new medicines?
Genuinely curious, would love to learn if that isn’t true / or is generally just not that big of a deal compared to other risks.
We already have some pretty horrific and documented/accessible bioweapons.
This gets into the philosophy of restricting access to knowledge. The conclusion I keep arriving at is that we’re lucky that there don’t appear to be many Timothy McVeighs walking around. I don’t think there is a practical defense from people like that.
I think you overestimate the difficulty of discovering bioweapons. There is a reason toxicology is the dead end for tons of drug molecules. It is very easy already to design molecules that will kill someone.
Even the word bioweapon is not accurate to describe a deadly (or harmful) biological agent. A weapon usually means that there is a source of deadly force, and a target. The source doesn't want to be hit by the same weapon it uses to hit others.
This is vastly difficult to achieve using biology. Any organism on the planet has it's own agency, and it will hit anything to reproduce and eat. In addition this is not limited to toxicology and releasing toxins, because the agent can just eat tissue.
For example phosphorus has been used in chemical warfare, but even that cannot be described 100% as a weapon. The phosphorus gas can hit people who released it the same as everyone else, it just depends on the wind.
Right now, on everyone palms, there are thousands of organisms which create electricity, eat wood and kill animals. Given that the palms are washed, that number is reduced to some thousand different species. If the palms are not washed the last 24 hours, that number shoots up to hundred thousand different species, even millions.
I do not see any difficulty for someone to enhance a harmful agent and make it deadly, using just regular computation and not even A.I.. However the person who facilitated this, will be a target too.
You'd have to work at the RIAA to think that piracy and bioweapons are comparable.
I don't know how much releasing this model is a delta on safety, but we certainly need to do a better job of vetting who can order viruses; my understanding is there's very little restrictions right now. This will become more important as models get more capable.
This is a textbook bad faith comment / attacking the person but not the subject of the argument. I’m just asking about others’ assessment of the benefits and risks. What do you think? Or do you think it’s just not worth considering?
Nobody has really been able to make a convincing argument whether these sorts of tools haven't lead to large-scale terrorism through bioweapons because the underlying problem is hard (for a sufficiently motivated adversary), or that terrorists don't have the resources/knowledges/skill, and as far as we can tell, the sufficiently motivated adversaries who have tried either failed, succeeded secretly, or were convinced to walk back from the brink due to the potential consequences.
In short there are other ways to negatively affect large numbers of people that are easier, and presumably those avenues are being explored first. But we don't know what we don't know.
If you're implying that the answer is "yes this is too dangerous", could you possibly give a few examples of technological developments that aren't "very dangerous to release publicly" by the same standard?
For instance, would any of the following technologies be acceptably "safe"?
- physical locks (makes it possible to keep work secret or inaccessible to the government)
- solar power (power is suddenly much cheaper, means bad guys can do more with less money)
- general workload computers (run arbitrary code, including bad things)
- printing press (ideology spreads much more quickly, erodes elite hold over culture)
- bosch-haber process (necessary for creating ammunition necessary to fight the world wars)
- nuclear fission, which provides an abundant source of environmentally friendly energy, but allows people to make bombs capable of wiping out whole cities at once (and potentially causing nuclear winter)
But even in that case, I believe that it's a good thing that we have access to nuclear power, and I certainly want us to use more nuclear power. At the same time, I'm very glad that a bomb is hard enough to make that ISIS couldn't do it, let alone any number of lone wolf terrorists. So I think I would apply the same logic to biotechnology; speeding up medical progress seems extremely valuable and I'm excited about how AF and other AI systems can help with this, but we should mitigate the ability for bad actors to use the same tools for evil.
An aspect that's unique about biotechnology that's different in comparison to the examples you gave is that most of those technologies help good and bad people approximately equally, and since there's many more reasonable than crazy people they're not super dangerous.
There's a concern that technologies that make bioengineering easier could make it easier to produce and proliferated novel pathogens, much more so than they make it easier to prevent pandemics; in other words, it favors "offense" more than "defense". The only one example you listed that has a similar dynamic in my mind is the bosch-haber process, but that has large positive downstream effects separate from its use for ammunition. Again, this is not to say we should stop medical progress, but that we should act to mitigate the dangers, and keep this concept in mind as the technology progresses.
That said, I'm not certain how much the current tools are dangerous in this way. My understanding is that there is lower hanging fruit in mitigating these issues right now; for example, better controls at labs studying viruses, and better vetting of people who order pathogens online.
The science to restrict is molecular biology (bacteria) or virology, not applied mathematics (AI). These folks can already do some wild things with the materials they have on hand and don't need fancy AI to help them.
Structure prediction is just one small slice of all of the things you'd need to do. Choosing a vector, culturing it, splicing it into an appropriate location for regulation, making sure it's compatible with the environment, making sure your payload is conserved, study the mechanism of infection and make sure all of the steps are unimpeded, make sure it works with all of the host and vector kinetics, study the pathology, study the epidemiology. And that's just for starters.
This would require a team and an enormous amount of resources. People motivated enough to do this can already do it and don't need the AI piece.
There’s still a long way from in-silico prediction to wet-lab validation. You need a full-blown molecular biology lab to test any of these.
Then again, you can just release existing dangerous pathogens. Like, poison a water with something deadly. So you don’t need a new one if you’re a terrorist.
Not a solution, but maybe if a bad actor tried to create a bioweapon, a trusted organization could use this technology as an antidote. Unfortunately this still leaves the possibility of some kind of insidious, undetectable bioweapon.
...as difficult as discovering new medicines, you mean?
Chemistry and molecular biology are fiendishly complicated fields, far more complex and less predictable than what general (and most of the non-biochem STEM majors) imagine them to be.
How do I know? I thought of one brilliant startup idea that would solve so many of the world's problems if only we used computers to simulate biological systems.
That or any other book on computational chemistry will give you an understanding why it is difficult to design anything of value in biological systems. ML can only help so much.
MBoC is more like Knuth's textbooks. It's a towering monument to the achievements of humanity over the past 150 years (molecular biology proper is less than 100 years old). As well as being highly accessible (readable).
Thank you for the textbooks! I've started studying Molecular Biology of the Cell to prepare for undergrad, but this is the first time I've heard about the others.
Unfortunately this is not always true. For example, one of the architects of the Tokyo subway sarin attacks[1], Masami Tsuchiya[2], had a masters in physical and organic chemistry.
Yes, a lot of terrorists have engineering degrees also.
But they're also dumb, which is why they think terrorizing random people will positively I prove the world in some direction they care about.
I won't go into details, but I think if I had 19 dudes with a death wish in America, and a few million dollars, I could do something far worse than 9/11.
The goal of an attack like 9/11 isn't really to kill the maximum number of civilians in order to terrorize random people.
The attack had a significant degree of symbolism. The intended audience was twofold: the Western public and leadership, with a durable message that they weren't untouchable (hence the attacks on the Pentagon and attempt on the Capitol), hence targeting large landmarks; the combination of civilian and military targets was to signify that they held the two to he equivalent. Plans were actually presented to attack other targets that would lead to more casualties, notably a nuclear power plant.
The other goal was to incite a religious conflict from the Muslim world against the US, and therefore probably from the US against as many Muslim countries as possible.
So the primary goal really wasn't to kill as many random people as possible (though of course that was a consideration), it was actually to target the tallest buildings possible as well as the most important government institutions.
Unfortunately, it really did move the world in the direction they wanted. Despite being extremely evil, they actually were remarkably successful at causing the social and geopolitical changes they wanted given the resources they had, and that caused yet more damage we shouldn't ignore. It also bears remembering (especially today) that terrorists often and unfortunately aren't as dumb as we think, and we underestimate them and simplify their motives to our peril.
There is a big gap between a master’s and a PhD, and then another between a PhD and a seasoned pro. To do something like a bioweapon, you would need a reasonably sized team of pros w/ a lot of capital intensive infrastructure. It would be virtually impossible to do in secret.
