Explaining why there is a conflict of interest, while interesting, doesn't mitigate it. You and I might know the details, but my conversations with friends/family/colleagues indicate that anyone who isn't deeply in the AI space does not. The way METR communicates information, and the impact it has on public discourse around AI is therefore concerning. The author of this article might be a whack job judging by the other comments, but that doesn't mean he is wrong.
That's true, but his first few lines of the post are so hyperbolic that it's borderline deceiving. There's no need to call for a Congressional investigation when you find very public information that one company is investing in another that it does business with.
The presupposition of guilt is hyperbolic, yes. But let's take anthropic's own hyperbolic public statements (10% chance of human extinction?) at face value. It follows then that any organization that claims to evaluate or regulate such an entity for safety ought to fall under rather heavy scrutiny. It is not unreasonable to ask questions like "is ant/oai/whatever buying off a non governmental regulator of a field that, by their own admission, has a 10% chance of killing everyone?", nor is it hyperbolic to want Congress to step in.
Congress needs to regulate AI, not investigate METR. It makes no sense for Anthropic to "buy off" METR when they are giving them full access and funding them 100% voluntarily.
Again, if we take the frontier labs' statement about AI risks at face value, Congress needs to do both.
Hypothetical: Several biological research labs are working on a bacteriophage that could permanently end all bacterial diseases. People from one of them routinely go on twitter and talk about how the phages they're making could actually just kill all life on Earth. The only "regulatory" agency is funded by these labs, and rather conveniently seems to say whatever is best for the IPO of these labs. Are you really telling me you have zero problem with this setup?
As someone who spent a few weeks trying to get into C++ (pre-LLM, so maybe it's better now?) and kind of just gave up out of confusion, I really found it challenging to deal with the multiple different ways to do something. I mean, which pointer should I use? How should I iterate? I'm sure there's an answer to this, but I just found it easier to go back to python and hit up rust when I needed it.
It seems like part of the concern is that the feature needs to be shipped right the first time because it'll live forever. Perhaps allowing features to be deprecated and then removed could help?
But idk, I haven't written anything that lasted 36 years, so maybe the current approach is for the best!
Does the existence of PEP 572 not similarly result in being paralyzed and overwhelmed at the number of ways to do something being greater than 1?
C++ definitely has complexity, but I'm surprised by the two examples specifically chosen. Which pointer? C++ only has 2: unique_ptr and shared_ptr (or 2.5 if you want to include weak_ptr but that necessarily means you already chose strong_ptr first). While this might be a complicated choice if you're just comparing to Python, but it shouldn't be if you're familiar with Rust since Rust has you make the same choice.
And then "how should I iterate" is similarly the same as any other language? Either a for or a for-each loop?
There's definitely baggage from 38 years of lessons learned, to be sure, just those examples are odd choices
When I started C++ nearly 15 years ago this was already a huge problem. I bought some books and looked up online references and after a while I discovered that both were already out of date.
C++ is a chimera. And it hasn't improved since.
Other languages are far more stable, conceptually. New features rarely undermine or reinvent the older ones.
C++ is unique in the way it keeps modifying its own core. There is no formal deprecation or EOL process, so the language is more like an archaeological dig than a living standard.
Yes there is or we'd have been talking about std::auto_ptr as well.
> New features rarely undermine or reinvent the older ones.
Many languages have mistakes they've later tried to sweep under the rug. JavaScript arrow functions trying to convince everyone that bind(this) wasn't ever a thing, for example.
C++, for whatever reason, is consistently the only language where people are determined to always dig up the skeletons and pretend they are at all still relevant. It's not unique in having skeletons. It's not unique in having language spec mistakes that can't realistically be changed, just ammended with a newer, better alternative.
But like the saying goes, there are only two kinds of languages: the ones people complain about and the ones nobody uses
C++ is not unique in having historic spec mistakes but it has by far the most of them which are still relevant today of all languages I have used. Maybe Perl or PHP come a bit close, but I doubt it.
Perl hasn't evolved in a while but PHP is definitely full of historic mistakes, dating all the way back to the first versions using strlen as a hash for function name lookup.
for i in list, enumerate(), for i in range(len(list)), while loop, list comprehension as a map, generator expressions, map() function, iter(list) + next()
The only thing it seems to be missing is a for_each function, assuming you aren't doing a map() in which case then that exists.
Don't forget * (true pointer) and & (called a reference, more honestly a pointer that can't be repointed, except with UB it can _totally_ get repointed).
Knowing the story behind a decision like that is worth a thousand spec clauses.
When the language got operator overloading there was the question of how to convert the arguments from the operator itself to its implementation. You wanted to write a+b, not &a+&b but most versions of operator+ were going to want const pointers for efficiency. One option was to make the & implicit, another was to introduce a type of argument that didn't need & but acted like a pointer. So references were born.
I ignore people who advise to use references when something can't be null. A null reference is UB just like a null pointer, it just shifts where the UB is, but the compiler is likely to produce the same code either way and the program is just as likely to crash. Instead, I use pointers to make it obvious that something is a pointer, and references when it should be invisible, just like with operator overloading.
The source of the UB is dereferncing a null pointer. I don't think it's possible to obtain a null reference otherwise. I, on the other hand, ignore people who advise against using references for parameters that can't be nullable. Particularly const references, which provide potentially massive performance improvements, and are a core idiom in C++.
Somewhere you have a null and somewhere you want to call a function on it (by mistake). Whether you call it through the pointer or convert the pointer to a reference and then call it through the reference makes no difference, the program still has UB. You can't even argue that the UB occurs at a different point in the program - that makes no difference because UB is allowed to time-travel.
Didn't they say they use references when the pointer/reference should be invisible?
> Particularly const references, which provide potentially massive performance improvements
My understanding was that const references don't provide that much in the way of optimization opportunities (if any?) due to aliasing and the existence of const_cast.
It's not about that, it's about avoiding copying to a function parameter. If you want a function that adds a bunch of integers, you should write it as "int sum(vector<int> const&)" rather than "int sum(vector<int>)" because the latter will copy the whole array of numbers just to sum them and delete the copy, when you could have summed the original array with no downside. Arguably it should be span<int> though now that exists.
If you just want to avoid a copy without adding a null state then it's the reference bit that makes all the difference. Whether that reference is a const reference or not does not matter for optimization, I think.
Sure, but I think that has less to do with optimizations and more to do with avoiding potential footguns. The standard could, in principle, allow temporaries to bind to non-const references (and IIRC older versions of Visual Studio did do such a thing).
And even without such a change, such a restriction is pretty easily worked around without a const reference with something like T&& temp = get_daily_prices(); sum(temp);. I don't think that'll result in different codegen than either passing a temporary to a const ref or allowing binding a temporary to a non-const ref.
Things I read recently makes me think that default target for stuff like that is a manky compiler that's half a step above macro assembler. AKA it's much faster if your compiler is dumb as rocks and usually not with actual compilers.
One of the first production C++ codebases I worked on had helpers:
Type& GetReference(Type*);
(Type is a placeholder, it was not templated). It worked exactly how you'd fear it would, and as a result we had to chase down null dereferences of our "can't be null" references all over the codebase.
EDIT: by "exactly how you fear it would", it just did a dereference and return. It didn't check for null, didn't throw, and couldn't otherwise signal a failure.
C++ has to be approached as a grab bag of useful parts, not a coherent whole. It's somewhat akin to Python if Python modules were C++ language features. It's said that nobody knows all of C++, even its creators.
Do you know C? Everything that works in C works in C++, modulo a few minor syntactic changes (void* doesn't implicitly convert). You can get acquainted to C++ features on an as-needed basis. Other people will surely disagree with me on this.
With pointer types in particular, I'd say use C-style pointers long enough to know how frustrating they are but also how they work. Then you'll know what unique_ptr and shared_ptr are doing to help you. (And those are the only two you need 99% if the time. weak_ptr is a spin-off of shared_ptr that you'll know if you need it. observer_ptr is just stupid ivory-tower stuff.)
Explanation: shared_ptr is an implementation of "reference counting". All shared_ptrs to the same object collaborate to know how many of them exist (it's just a ++ on creation of one, -- on destruction of one) and when the last one is destroyed it also destroys the object that's pointed to.
weak_ptr is like shared_ptr but doesn't add to the count. When the last shared_ptr is destroyed it destroys the object even if there are still weak_ptrs. Any weak_ptrs pointing to that object become null. I won't dive into how that's actually implemented. Obviously it costs some CPU cycles to make that happen, so you don't see shared_ptr or weak_ptr a lot in code that's written with a performance mindset. Although you're still getting 90% of the performance benefit by writing C++ instead of JS even if it's "inefficient" C++.
I agree with GP's point and add, if this isn't something you need to do, then don't even think about it. C++ is like a toolbox with a bunch of tools. I don't study all the tools before building a table, I look up how to build a table, and then see if the tools are in my toolbox. Studying the triangle screwdriver would be a waste of time and only confuse me more, unless I'm trying to drive triangle screws. Only when I need to drive a triangle screw then I can look for the triangle screwdriver.
Anyone that thinks Python is "simple", to come back to some of the suggestions on how it is easier than C++, they really should learn the language reference, standard library, and CPython FFI manuals.
>
Anyone that thinks Python is "simple", to come back to some of the suggestions on how it is easier than C++, they really should learn the language reference, standard library, and CPython FFI manuals.
Python is more "simple" in the sense that
- it is more forgiving with respect to badly written code,
- it has collected less historical baggage than C++.
It really isn't less forgiving, it just happens that as a dynamic language, it blows up at runtime instead of failing to compile.
Yeah, we all know how cleaning up Python has worked out, and unfortunately a few ecosystems failed to learn from it as well, e.g. Java 8 => 9, .NET Framework => Core.
The things that make C++ unforgiving is not the things that blow up at compile time. Those are trivial to deal with, comparatively, since you have the error right there. It's the things that compile, but then blow up in weird, and often unpredictable ways that make C++ unforgiving. Or to put it another way, it's the lack of undefined behavior in most situations that makes Python more forgiving.
Yes, Python may still blow up at runtime if you don't get it exactly right, but it'll blow up in very predictable ways: Off by one errors accessing arrays doesn't work some of the time, maybe, except on Tuesdays, instead you always get an IndexError. And some cases, like overflows/underflows, doesn't even blow up, instead you just get bigger/smaller numbers.
Type-hints also does a lot (but still not enough, IMO) to reduce the amount of blowing at runtime up that you might experience when working in Python. Though that obviously wont help with poorly written code, which is unlikely to have proper type hints
> The things that make C++ unforgiving is not the things that blow up at compile time. [...] It's the things that compile, but then blow up in weird, and often unpredictable ways that make C++ unforgiving.
Features are deprecated and removed. It's uncommon but it happens when a new feature makes all use of the old feature unnecessary. Usually it's deprecated for a few versions and then removed some years later. You can always compile old code against an old standard.
C++ does have many ways to do anything. Most of that is historical, or due to features and library facilities with overlapping capabilities, or inherent to multi-paradigm programming languages.
Every C++ project is written in its own language. Its own "house flavor." This is true of all languages, but is particularly stark for a language as old, complicated, and powerful as C++.
I share the same sentiment and experience from trying to get into C++. I bought "A Tour of C++" (2022, Stroustrup), I read it back to back, and I thought to myself "it's not so bad, seems quite nice actually, specially the modules part"... oh boy. After 2 weeks writing a HTTP requests CLI tool I was feeling really demotivated, I gave up and went to rust, for no special reason, I just had it queued up after reading "The Rust Programming Language" book.
> pre-LLM, so maybe it's better now?
I came back to the toy tool some time later and used GLM 4.x to implement it further. I had a good experience with it and I understood the code it generated. From my experience, in general, LLM is a good mentor, if you are willing to ingest the content step by step. For me, it is no different from when I used StackOverflow (and other forums) to try and ask how a thing is done.
"I mean, which pointer should I use? How should I iterate? I'm sure there's an answer to this, but I just found it easier to go back to python and hit up rust when I needed it."
You can of course use Python. But we are talking about languages with different applications. You want fast development hand have no expectations on speed? Use Python! You want to control your computer to get maximum performance with a safe type system? Use C++!
Since you are already talking about pointers: In Python you are not even given the chance to do all the stuff you can do in C++. You have to rely on garbage collection (and "del"). That is an entirely different thing than C++ where you exactly control how long your objects should live.
For the pointers (order I would suggest):
1) No pointers, use references whenever possible
2) Use std::unique_ptr
3) Use std::shared_ptr
You can also try to use shared_ptr all the time if it makes your transition from Python more easy. It is not optimal, but it may get you started.
And do not start wrapping your head around rvalue references right from the beginning. This is a very special topic, which you can ignore as a beginner.
> As someone who spent a few weeks trying to get into C++ and kind of just gave up out of confusion
what should be conclude from this? this is like saying you tried to become a professional trapeze artist for a couple of days. C++ is complex and deep. what did you seriously hope to accomplish in a "few weeks"?
i started writing C++ professionally 4 years ago, i.e., i write it every single day at $JOB. there are still large parts of the language i'm not familiar with.
I studies for some descent while with the objective of using unreal engine, then i found out everyone was just using blue prints so not even them liked CPP
You're very diplomatic (yet downvoted), C++ is an abomination of a language. It just puts too much accidental cognitive load on developers. I understand that
- it shines for some tasks where Rust isn't capable enough
- it's impossible to get rid of because of all the code so we may as well modernize it
- there's an active community of people who built a career of being C++ experts and have no interest to see the language disappears.
Even in the LLM era, I find it's difficult to code in C++. The language has so many corner cases that LLMs make mistakes. My experience 6 months ago was that the code written by my agent had tons of complex linter warning (exceptions, lambda leaking references, futures whatever... that I just can't avoid because the codebase mixes all of this). Whenever possible, it's probably time to either modernize it or port it to Rust, component by component, until the C++ part is small or inexistant.
Maye you need a better AI. Just sayin'. My AI (Claude) doesn't make mistakes. Ok. It doesn't always think strategically, or factor well. But it certainly doesn't struggle with language mechanics.
Modern best practice for C++ is to crank the warnings to maximum, and to treat warnings as errors. You absolutely should treat linter-like warnings as an error, because they invariably are. So you know.
Discrete dimensions can exist in Pareto considerations. Not that I'm too fond of this game, but given that "Origin" is definitely part of the calculus of model deployment, it makes sense to consider it in a Pareto calculation.
It's just a different pareto frontier because you're filtering out items that don't fit your criteria. "Model was made by Anthropic" would be an equivalent dimension. You would never plot this, you're just reducing the subset of models you're comparing.
By that logic, any model is a frontier model. My GPT-2 clone is a frontier model when you compare it against models that were trained on my homelab.
That's literally the point. It's a different frontier. If you say "frontier model" and don't specify which frontier you're either being deceptive or naive.
I'll give you an example from my field (nuclear fusion). I have a small team working on an idea which I think would really advance the field. Our initial concept paper was published in 2019. Progress on it has been slow because we don't have enough manufacturing capabilities, budget, or time on the facility. Will AI solve that? Don't get me wrong, I'm pretty bullish on AI, and it's been phenomenally useful so far (I definitely feel like a 10x physicist in the last 6 months), but I don't see a swarm of agents putting fusion energy on the grid tomorrow.
Also, fwiw I have a personal test problem I give to every frontier model on release, which is to ask it to final a realatively obvious solution (in hindsight) to a minorly important open problem in my field. I know the answer, but havent gotten around to writing it up yet (it's really not that important). Every frontier model (so far) just regurgitates existing solutions.
How much effort do you put into prompting it? Are you asking it one shot with a simple prompt? Are you giving it to multiple agents and giving them a workspace to collaborate in? You might give it to a trusted colleague who does not know the solution along with an agreement to jointly publish, ask them to go through the process of solving it with agentic support, and then compare your solutions.
Edit: to be clear, I completely believe your experience - just pointing out that it might be interesting for you to actually “try” to solve it thoroughly with the tools, though it is obviously hard to do that in a fair way when you know the solution.
Enough! I think it's a reasonably fair test of the models. Let's put it this way, I use the same process to resolve problems outside my peak expertise, and I find it works well.
I think the solution is easy enough that any competent colleague would come up with it. Some variant on it has probably even been published (hence why I'm not chomping at the bit to publish). I think it comes down to the models being extremely spiky, and the problems they're trying to solve being easily verifiable in code.
With my personal little test problem, that never made a difference.
However current frontier models got good enough at websearch and thinking through the problem, that they can finally piece together the answer from my own public writing (eg on stackoverflow). And recently someone wrote up an answer to the problem and put it on Arxiv.
So, there's a very niche topic that I was generating some fiction about using AI, and as part of that I gave one of the characters an unusual name. A few months later I again experimented with using AI for this fictional niche, and the AI just randomly chooses to use the same unusual name.
"Forebears" is a site that ranks the popularity of names. They rank the name I used at 324,668.
This is why I believe the AI can be significantly influenced by a single person.
I cleared the memory. I frequently do so because I want to experience what the AI is capable of from a fresh slate. I alternate which AI I use monthly, paying for a month at a time. I clear out all memories and state, as best I can, at the beginning of every month.
I'm pretty sure the name has moved to different AIs somehow too. I've seen the name used on Grok, and also on NovelAI. I've forgotten the exact details.
Iirc some of the prior math discoveries involved very very very heavy handed guidance to force the AI to discard prior assumptions, and to try many avenues with MAX effort. So ya models don’t behave like ground breaking theoreticians by default.
Ha, I had a similar little test problem, but the frontier models are now smart enough to read enough online of what I posted here and there to piece together the answer. And someone else has written the paper in the meantime and got it accepted for publishing.
Definitely. It will let some smooth-talker with a vibe-coded website raise billions for "Nuclear Fusion AI". Then they can use AI to squeeze your team with lawsuits until you agree to sell cheap, and maybe keep you on as an employee to build it.
Hah. Fusion scammers are a dime a dozen. Unlike SaaS apps or finance, though, you can't sweet talk a neutron into existence, so they fizzle out pretty fast. I'm more worried about fusion dying because nobody comes up with a good enough idea than someone stealing my ideas.
Going on a tangent: I think fusion is pretty much useless this century. We already have energy generation technology that has high capital costs and low running costs: it's called nuclear fission and works fairly well.
Fission hasn't exactly taken the world by storm.
There are a few nice advantages fusion has, but they are more important once we want to reach the stars as far as I can tell; not this century on this planet.
But I'm just an armchair analyst. You probably have a much better understanding.
Nah I agree with you. Not this century, not this planet. Fusion rockets are really efficient, though ;)
I just find it to be an interesting problem! My hope is it happens before I die. My rule of thumb is anyone who says energy on the grid in < 10 years is a blatant liar.
I agree that you can build a facility to put energy on the grid in under a decade, but the probability that it works is going to be tiny, and anyone giving you the billion or so it would take would be disappointed to say the least.
Therefore, most credible folks are talking a 2-3 facility staged approach. And probably 5 years per facility to build, and 5 years per to test.
Also, remember that even with huge cash pools, you have supply chain problems. Some of the nonlinear optics takes a fixed amount of time to fabricate.
> I agree that you can build a facility to put energy on the grid in under a decade, but the probability that it works is going to be tiny, and anyone giving you the billion or so it would take would be disappointed to say the least.
A giant water tank that you detonate H-bombs in should work, and even be energy positive. :)
I guess I'd say, let's have a swarm of agents build a metro line first and see how well that goes. Obviously, given that we have embodied intelligence (sometimes) that's capable of doing this work, it would be silly of me to say that it's impossible for it to happen in silicon. But given where the SOTA is right now I'm skeptical of this being true in the near future. But hey! If you'd asked me the same question about GPT in 2019 I'd have said I would be skeptical there too. So who knows.
I mean, you never know. If we could have simulation models with excellent fidelity, then AI could do all the research and iterations and come up with something very plausible. Cutting down physical world experimental iterations/prototyping.
So effectively, you would spend the first 1000 experiments inside a computer simulation and move to the physical world only when you had enough confidence.
I think you should realize that we already do that. In fact, that's one of the reasons computers were invented in the first place! It turns out that science is about unknown unknowns, and by definition simulations can't answer that.
But agents are really useful for doing some of the really boring stuff, like convergence checks or setting up meshes or whatever.
If you're talking about things at the equivalent level of say, finding out if positrons maintain symmetry with electrons for their magnetic moment, then yeah - I hear you. Or if you wanted to research some unknown behaviour of hot plasma at 15 million C, you can't do that in a simulation.
..or can we?
Humour me for a moment. Maybe simulating to the degree that was necessary for cutting edge work was not possible earlier, when humans had to program each and every thing in the system that could affect the result. But with current gen petaflop-on-your-desk level compute + swarms of agents that can work tirelessly to account for every known thing to humanity, maybe the threshold has been pushed far enough to produce some more meaningful discovery work in simulation?
Respectfully, finding out how scientific computation is used in fusion research is a Google click away. And yes, we can simulate these conditions.
Let's put it this way - a first year graduate student can ignite a fusion target in simulations. It took thousands of scientists the better part of three decades to do that in experiments. I think agents could do the former really efficiently and at scale, but I'm skeptical that agents could do the latter. Oh, and btw, fusion simulations of the kind you're talking are in exa-zettaflop territory ;)
I mean, it's the pareto principle, right? 80% of the applications require 20% of the intelligence. If I can drop in something without needing to create high quality training data that's good enough, that's great! Skin cancer probably isn't one of those applications.
I'm surprised that it's not very good. I'm working on an MCP for a Foss civ 5 reimplementation (unciv) and I find that 5.6 Luna is better than the built in AI, and costs about $2-4/player/game at API prices. I suspect Astra would blow humans out of the water, but I don't have $1k/player/game to spend on that lol
Maybe civ 5 is more reactionary than strategic, so simple strategy that doesn't look too much into future works reasonably well. Or maybe civ is more mainstream and models absorbed more knowledge about strategies that work reasonably well.
It's definitely not the first. I'm not convinced it's the second, either. I'm pretty sure if you Google for HOMM3 strategy, there should be enough hits! I guess what I'm trying to say is that you shouldn't give up :)
Also the speed is painful in the extreme. It's definitely not yet at the point where I can just play a game. But maybe if they get Astra on those cerebras chips and get the price down...
People used to complain about back of the napkin posts because they signaled low effort. The same is true about LLM blogs. A back of the napkin blog signals more effort than slop.
A back of the napkin post means at least one person had interest in its authorship to put down their drink and write something, the LLM made blog posts are a show that zero people had interest in its authorship enough to put down some words.
Indeed we are, but it takes very little to train a human being. In fact, several thousand unique ones finished pre training as I typed this comment! That allows for a considerable natural diversity compared to a chatbot.
reply