Huh. This reminds me of the asymptotic equipartition theorem. Samples taken from higher and higher dimensional spaces will concentrate into a typical set.
Does model performance also concentrate into a 'typical case' where things work pretty well and a non-typical case where it's completely unpredictable as the number of parameters increase?
No. The official report seems to suggest that software in IMS core (the subsystem that controls VOIP signalling in the mobile network) bugged out after the time was shifted back 20 years. That doesn't narrow down the specific failure that stopped calls from going through very much. IMS core is full of timers, but it could also have been a license server issue for all the report said about it.
I haven’t read the report, but I did read at the time that it was a certificate validation failure: clients thought the current time was before the "Not Valid Before" date. I forget now what essential service became unavailable because it was disbelieved.
(Source would either have been the Whirlpool forums – where Australian ISP and telco engineers can be found – or local Australian mass media.)
This the most common failure mode I have seen with clocks being widely out of sync. You can't access many online services as the certificates are either dated to future or expired.
Australia's bar for skilled employment is pretty low and deflates wages as well. That's before we get into the prevalence of exploiting working holiday visas for practically slave wages.
You can't really force someone to work for X years at a place they don't want to. I don't know if what you're describing is an enforceable contract term.
Sure you can, it's effectively the same contract agreement as a sign-on bonus that requires you to stay for at least 6 months. Instead it's training cost that you get saddled with if you leave before X time. Of course, the exact monetary value would have to be spelled out ahead of time.
Well, I say sure you can, but I'm not a contract lawyer. It just sounds identical to me. You're not forced to stay, but if you end the contract you have to pay.
The terms involve the person paying back the training costs or some penalty if they quit before X years.
I saw it once at the beginning of my career but that was nearly 15 years ago.
US companies are usually unwilling to enter into any contract with labor because they want the ability to renege on the deal at will, hence the at will states.
You see companies making contracts like this with other companies all the time, so I can only assume it’s a cultural value to never make a deal with labor.
I mean. That's how profiteering works for the most part. Extracting wealth from people who don't know better is many times easier than creating more wealth for everyone.
I think most people who write code are the latter and not the former. The industry has diluted the term "engineer" so much that they maybe don't even realise that traditional engineering projects are about more than just implementation work.
Each unit of work in any given feature of a Web app has been implemented 500 times yesterday alone, and nearly each time exactly the same way. I mean that’s what programming basically is right? I’m surprised these patterns that are repeated so often by developers could have been/still be automated away even without AI/LLMs.
One thing I wonder about is how applicable your statement about web apps is to something like lamps or lighting generally. How many electrical engineers have re-designed a circuit that turns on/off a light? Are EEs who wind up doing mundane engineering like this still engineers?
You don't need to be an "Electrical Engineer" to do simple design work like this.
Professional Engineers are qualified to take on more liability than just creating a design. So you don't need an Engineer to design a product, but (depending on jurisdiction) you will need an engineer to certify that your product won't hurt or kill people.
If you do a lot of design work, then you may want to hire a Professional Engineer in-house so that your company has more confidence that they will produce compliant designs with fewer iterations.
Engineers are often the best people to work with if you need to do things that are done infrequently since theoretically, they're trained in the prerequisite first principles so they can make judgments that are rooted in rigorous analysis in addition to their practical experience.
"electrical engineers" aren't usually the ones making a lamp though. If you're paying EE wages to add a light switch to an Edison socket you're massively overpaying.
Lots of things have lights on them though. I'm thinking every status light on every piece of hardware.
Edit: also, I'm not exactly filled with knowledge on new lamp design/construction, but I have seen startups/kickstarters that make new lamps that seem like redesigns from the ground up.
Aerospace engineers who build rockets do not have some certification body allowing them to be called engineers. Same with most electrical engineers working on almost everything.
If you think a government deciding who is an engineer is a *good* thing then maybe you should ask yourself why the United States which doesn't require this for the two non-software engineering disciplines has the best engineers in the world.
Really? What evidence do you have for that? Are you saying that companies like Airbus, Mercedes-Benz, Ferrari, ASML, Leonardo, Rolls-Royce, Dassault Aviation, Toyota, Bosch, Komatsu, Siemens, ABB, Mitsubishi Heavy Industries, Alstom, Vestas, Samsung, Sony, Hyundai Heavy Industries, Mitsubishi Shipbuilding etc. don't have world class engineers?
The United States has the best engineers because it pays the best. Good engineers from other places move here to capture some of that. Obviously strict licensing laws would get in the way of that.
Outside of some exceptions which don't really exist in the US (shipbuilding and heavy forging) the American companies in those industries are highly competitive and often market share leaders.
Annoyingly, the US government still sometimes does.
If you want to take the patent agent exam, your CS degree has to come from an ABET-accredited program, though many of the very good ones aren't (Stanford, CMU).
Likewise, federal jobs also sometimes seem to want accreditation but not always.
You can call yourself engineer if you feel like it, however in case it comes to some court case due to liabilities and such, there might be some issues coming up with having Eng in that contract signature.
Many here would probably say that they have never did the exam, and nonetheless use the title, which is as mentioned, not an issue as long as you don't land in court and the validation of title doesn't come up.
Also in most European countries, being an Engineer even if not professionally qualified, automatically means that the person in question took a university degree in engineering, on an university whose engineering degree was certified as such by the government organisation responsible for all engineering professions.
If you can call yourself an Engineer if you feel like it, then it's not a protected title.
"Doctor", is for instance a protected title. You can not advertise yourself or your services as a doctor unless you're a qualified and practicing medical professional. I believe some kinds of legal practices are the same.
[MIGHT HAVE INACCURATE IMPLICATIONS BUT PRESERVED FOR POSTERITY:]
In Germany I don't think that counts. "Software Engineering" as a title isn't regulated (as kcexn is asking for). The German term "Softwareentwickler" more literally translates to "Software Developer" (as opposed to "Ingenieur") but I don't see companies having problems translating that in job posts and even contracts to "Software Engineer". I've never seen "Softwareingenieur" used tbf.
Getting into Germany as a Software Engineer (for app development) is also as relatively frictionless as it gets in comparison to more-traditional engineering fields.
Verdi is the German trade union so they have to make these kinds of distinctions about which professions they can represent. Don't quote me on this one but I think Verdi represents very little of the modern "app development" software engineers. I guess those who work for more traditional German industries like auto-manufacturing can fall under their umbrella.
---
[LESS INACCURATE EDIT AND MY THOUGHTS IN LESS WORDS:]
In Germany "Ingenieur" (Engineer) is a loaded term with legal implications so "Software Engineer" is "Softwareentiwickler" (Software Developer) instead. But really this is an HR sleight-of-hand trick. At the end of the day, getting into professional Softwareentwicklung is the same in Germany as in elsewhere that calls it "Software Engineering" (and IMO is what kcexn was asking about anyway).
Well, I have been well served by I.G. Metal, by working in industries they also covered.
Also I have yet to meet anyone doing agency work, that started as Azubi, Quereinsteiger, BWL,... and would sign any document or call themselves Engineers, like it is so very common in US out of a plain bootcamp.
I don't think Software Engineers in the US would sign any documents that expose them to legal liability as a professional engineer either.
It's still not clear to me from this discussion whether the term "Software Engineer" is legally protected in Germany (or any other European country) or if it's just a cultural convention.
The distinction is basically, if a software developer started a consultancy developing custom software solutions and called the company XY Software Engineering, would they be penalized for false advertising if they didn't have professional engineering qualifications?
Your mobile phone runs on a CPU architecture designed in Europe (ARM), manufactured by a machine that only a European company can make (ASML), using a number of protocols invented in Europe (Bluetooth for example), using protocols invented in Europe to browse the internet (HTML/HTTP), based on a very long history of computer science ideas and programming languages invented in Europe or by Europeans.
Give credit where credit is do.
Having said that, I do agree that US companies tend to be better at commercializing new ideas than European ones.
For sure there are great engineers that are from Europe, I've worked with a bunch in the US, but it's a numbers game, and there's no competition there. Partly because so many of those great European engineers come here.
It's an ARM CPU - ARM is built off Berkeley RISC. ASML would never exist if not for billions of dollars of R&D spending by American chip manufacturers, much of which was directly provided.
Obviously there are good engineers in Europe. There just aren't as many as there are in the US, in large part because the US encourages immigration with much higher salaries.
It is correct that a lot of talented Europeans get educated in Europe and then immigrate to the US starting successful companies. Without those Europeans, there would be a lot fewer successful US companies.
American rules, if you start a billion dollar company in the States you're one of us :)
On a more serious note, the US awards more college degrees per capita than most other developed nations. Strict licensing requirements or not, software engineering is just what it's called here because of the many and obvious similarities between it and other kinds of engineering.
The English term "software engineer" is not a protected title anywhere, FAFAIK. However, the local language equivalent of "engineer" is protected pretty much everywhere in Europe because it was awarded solely at technical colleges and universities. The title is (mostly) equivalent to a Master of Science degree, except that technical colleges could also award it (the English equivalent being Master without field designation).
Before I retired (in the US) the organization I worked for was heavily civil engineering oriented, and they were pretty insistent that the computer folks not call themselves engineers. The state licensing board was pretty insistent too.
I was also in a consulting CE firm working in software, but had moved from an engineering position and had an engineering undergrad degree. I was strongly encouraged to get my PE so the company could advertise it, even though the PE credentials had nothing to do with software development.
I think civil engineering is the main field in the US that relies on certifications/tests for FE/PE, there's probably more I'm unaware of though.
As I pointed out above though, aerospace engineers (working on rockets/space applications anyway, IDK about planes) and electrical engineers don't need those tests. Requiring it for CEs seems like a historical artifact more than anything.
> In which countries are Software Engineers not allowed to call themselves engineers unless they are professionally qualified engineers?
I think a better question is: what is the criteria those countries use to determine if someone can use the engineer title?
In general the software industry is still in an early confused state about standards, approaches, skills, etc.
Should we be using functional techniques? Object? Both? Relational? Column store? Push? Pull? etc.
The number of ways to build a working system is enormous and we have a fad-of-the-month every 5 years which starts as a silver bullet and always settles down to be just another option for another set of use cases.
The European Council of Engineers Chambers is seeking to standardise training programmes, beginning with the civil engineering sector: https://www.ecec.net/what-we-do/common-training-framework/.
Hopefully, this convergence will extend to other engineering professions.
I think whether "taste" was ever important comes down to whether software is a commodity to be bought and sold, or something more strategic.
The company that needs a webapp because everyone else has one is never going to value "taste".
To adapt to this AI world, we need to either ship more lower-quality software, or start seeking out higher value problems. I don't see any other way around it.
Not being an expert in any of the fields OpenAI has "advanced" I don't want to prematurely downplay the significance of this contribution. However, I am worried that the language they are using in this blog post is exaggerating for the sake of marketing.
It is true there hasn't been a reliable computational approach to solving these problems before. But do these proofs contribute new ideas to the mathematical corpus, or are they simply an effective method to exhaustively search the literature for the right combination of existing tools to apply to the problem?
Essentially, did these problems seem like they had an intuitive answer and were feasible to prove before, just not high enough value targets for an expert to invest time into? Or were they fundamentally difficult prior to this point and it appears that AI has done something more than just throw the problem into a big solver.
The ones I'm familiar with are big breakthroughs, but they are both counterexamples. Examples have an advantage in that once you have the example in hand and a sketch of the proof (which they have provided), then an expert can probably work out the details themselves.
The sofic groups question was the outstanding question about sofic groups. Almost everyone thought that non-sofic groups existed, and there were plausible candidates, but proving a group was non-sofic was out of reach. Now that we know how to do it once, we can probably do it a lot more.
The Connes rigidity conjecture I think people thought was false, but it was a provocative claim to make. The significance of conjectures is frequently not that the answer to the question is "yes", but that we don't know how to answer the question. And now, apparently, we do.
Interesting. Do you have any more specific insights into where you feel AI was a big value-add to these problems? I don't want to be overly dismissive of AI, but I also feel that the AI hype engine frequently positions claims as being 'ground-breaking' when they are really just interesting incremental results.
The general consensus of developers is that AI can only do the work of a strong 'junior'. Yet as soon as we are presented with pure mathematical results, people seem incredibly ready to accept that AI can do more than what a strong student could achieve.
They are more than a strong student could achieve. I'm not equally familiar with the problems, but the ones I'm familiar with, if a student solved them people would be thinking "that's someone on track to win the Fields Medal one day".
If it works better here than for programming, then I would guess it's because you can give it a very precise prompt, so you either solve the problem or you don't. If you read the prompts people have shared for problems like this, then the instructions are basically "Solve this problem. Don't give up early. Don't solve a similar problem."
> but proving a group was non-sofic was out of reach
a colleague was telling me that the base idea for proving that something is not sofic already appeared in the literature around 2019 or so (this is the "expanders graphs" that are mentioned in OpenAI s paper. no one had managed to find a concrete example though. this doesn't make the result less impressive in any case.
The problems from CS (CVP and circuit complexity) are very important problems that have been worked on by top researchers for 30-40 years. Some of these researchers include Turing Award winners. A solution to them would be a best-paper award at many top CS conferences.
I assume you're talking about No. 5, the arithmetic circuit complexity bound? The existence of a lower bound than state-of-the-art is certainly a significant result and worth publishing.
But the wording of the result makes it sound like we don't know what the lowest possible complexity bound might be. So, prior to this result did we think there couldn't be a lower possible bound? Or did the arithmetic circuit community think there were lower possible bounds but didn't see it as a high value target for experts to tackle (maybe a problem that was instead regularly given to students to study).
Circuit complexity lower bounds (and lower bounds in general) are notoriously difficult to come across.
For example, despite our best efforts, the state of the art lower bounds on time complexity of algorithms for solving 3SAT is O(n). In contrast, our best algorithms for the task run in time roughly O(2^n). That’s an exponential gap. This is despite decades of trying to find lower bounds.
> However, I am worried that the language they are using in this blog post is exaggerating for the sake of marketing
Your worry.... is because they used the word advanced? For marketing? The word is used very appropriately here. There were PhD's who spent a big part of their career tackling these problems.
I have no idea how many PhD's have spent how much time of their careers tackling these very specific problems, and I doubt you do either.
I'm trying to understand if these specific problems were the kinds of problems that would have justified an expert investing weeks or months to solve. Or if they were the kinds of problems that would normally have been given to students to investigate.
They are significant problems which experts have spent months or years studying. I heard a mathematician say that resolving non-sofic groups and Connes's rigidity would be career-defining for a mathematician.
"Breakthrough research" can be defined (in the citation record) as research that both (1) becomes highly cited, and (2) brings together citation chains that were previously not showing up together.
Mundane incremental research is cobbled from existing citations that already appear nearby in the record.
Basically, innovative research is a measure of bridging thought and domains that were previously not bridged. It's quite concrete as a measure in the citation record.
So we can know pretty conclusively.
Puja Ohlhaver gave a talk on this[1], and ran some experiments (that I had the pleasure to support on)
Breakthrough math research is very rarely highly cited. Maybe some combination of pretraining scale, inference speed and orchestration will help, but it's telling that OpenAI is solving random math research problems rather than bedrock algorithms and their implementation. Even as cool as the tech is, there still is very much a clock that they have to outrace before they collapse.
I'm not arguing that this isn't innovative or worthy of publication. Basically any result that moves the needle meets those criteria. I'm interested in how the results that OpenAI has published here differs from finding optimality solutions for incredibly niche optimization problems by throwing the problem in an enormous solver.
They differ in that there hasn't been a solver you could have thrown them at. I guess you could argue their harness + LLM setup is a "solver", but the approach is so different from what we used that word for in the past that I don't think it would be appropriate.
It’s a marketing. They are a sham company. If this article was by Scientific American or something it would be worth a lot more. They are literally trying to keep the hype train on track.
Also on HN front page today: AI's debt binge can't last, hidden borrowing reaches $1.65T (fortune.com)
It is marketing, they are a shady company, and yet, if someone had access to these results before today's modern AI tools, they could get tenure at any university in the world.
If the prompt was crafted by an expert mathematician with pages of context and insights provided to the LLM to put it on the right path, that is a lot less impressive than a prompt that just says "find a counterexample to this conjecture".
Similarly, if they are spending millions of dollars in inference just churning on thousands of problems and these are the 10 solutions they came up with, that would be less impressive than if they chose these 10 problems specifically and were able to come up with solutions.
Does model performance also concentrate into a 'typical case' where things work pretty well and a non-typical case where it's completely unpredictable as the number of parameters increase?
reply