This isn’t news, this is exactly what we said when the don’t be evil company was pushing them in the name of page loading speed (b/c the faster they get to display an ad making time to sell another ad!)
What?!?! I have never heard anyone pronounce these terms in such a ludicrous fashion, and it’s not because I don’t have exposure I do a fair amount of work in this space.
Not all science is performed by way of academic research. Industry performs research too, and the best way to ground both biased industrial research and impractical/naive academic research is to have the two communities engaged as commingled peers, where both have standing in funding and process and are each obliged to provide justification and transparency to the other.
I think this is a common confusion. You need impractical/naive research because that's how exploration gets done. If you heavily bias the search towards areas of the search space that you've already seen a lot then you've basically killed science and its ability to deliver useful results.
The phrase "Eliminate corporations from academic research" does a lot of heavy lifting here.
If a company wants to bring a pesticide to market. We would like them to bear the brunt of safety research costs before they launch. You might say that product safety is different from pure academic research.
We're just arguing semantics and incentives at that point. Like giving them a tax break for that research. Sure that sucks. But how else shall we make it happen.
Corporations can fund research in ways that allow them to suppress results that threaten their profits. If science is conducted in these ways, corporations can fund science (for which they receive tax benefits) and if the results are positive, it gets published and the corporation wins because the research supports their business model (tails, I win). If the research results does not support their business model, they can decline to publish it (tails, you lose).
There are ways to do science that avoid this kind of corruption.
Name three academic institutions that would sign a contract that prevented them from publishing adverse results. You are at least fifty years out of date on this argument.
A) They don't have to have that in writing, it's very implicity understood that you don't bite the hand that feeds. B) Even if they do find adverse results, depending on what they are (e.g. actively harmful versus negative results) you may have a hard time publishing them since journals don't care much for negative results.
Presumably he means you can use company funds to donate to a pro-pesticide charity/NGO, and then use that donation to claim a tax credit (around 50%, subject to some limits). It's not really a "heads I win tails you lose" situation though. If you donate $100k, and claim the tax credits, you're still out $50k. You might get back more than $50k worth of monetary benefits from whatever the NGO/charity does, but that's not really guaranteed. It's like buying some junk on wish.com with a 50% coupon and saying that it's a "heads I win tails you lose" because if it turns out to be not junk, then you win, but if it's shit you still saved 50%. A far more straightforward and honest way of describing it would just be to say that they get a discount on their charity spend.
That's also money you're spending out of your "profit" instead of netting it off as an expense, so it's really a fairly minor discount on pre-allocating money over a multi-year horizon for purposes that can be structured as a charity.
> Why would eliminating corporate funding in academic research be bad?
The honest answer is because there isn't enough funding through government and philanthropy.
You'll find a really odd opinion, that many people think engineering endeavors are significantly more important than research ones. When instead the truth is that both are two sides of the same coin. After all, research funds the next generation of engineering.
I think there's an unfortunate effect of this because there is quite a bit of science that is beholden to corporate... let's say "motivations"(?) rather than more free intellectual pursuit. Then the money that is more free is much more competitive and frankly people are metric hacking to get there (publish or perish paradigm is weird when you have to frequently publish novel works). The system was fine but the environment changes and eventually all metrics are hacked.
For personal values, I think it is quite important to fund research. From the very basic low level to even higher engineering research.[0] I'd actually be in favor of 5-10xing the federal science budget. I'd argue this should be primarily funded through federal grants, because people will take that research and go make things which will then be sold (world wide) and we'll tax through that. It's like venture capital if it was less risky but had a longer time frame for ROI.
The category of "General Science, Space, and Technology" accounts for 20.5 bn dollars[1]. The problem is, people understand this to be a big number and hear about these huge costs (often of projects that last decades!), but this is actually 0.4% of the 2024 budget! 64% of that (13.2bn) is going into space flight, research, and supporting activities. The other 7.3bn is going to the rest of science! To put this into perspective, we spend 8.4bn dollars on salaries and expenses for Social Security. The Navy gets 16bn for research, Army 11.5, Air Force 8.2, and another 14 on "Defense Wide" (so a total of ~50bn).
edit: To be clear, I'm not against corporate funding of science or even corporations working with academics for research. I think it can often work out great. But I think there needs to be some balance or academia gets captured by industry. I think we can think of some where this may have happened (or is in danger of), including domains closely related to the topics HN cares about the most.
[side note] A fun thing I do is talk about the ultra wealthy's wealth in terms of "CERNs" instead of dollars. Because numbers in the billions are just unimaginable (I have a physics degree and work with numbers this large -- or the inverse -- and if you tell me you understand this more than an abstract concept, I'll call you a liar). But we can imagine a CERN (which is funded by several countries btw and not a significant part of any of those budgets. Despite being the largest if not most expensive physics experiment ever). Which is a (roughly) 10 billion dollar super project that took (roughly) 10 years to build and costs (roughly) 1 billion dollars a year to operate. This actually makes for a good comparison for people like billionaires because their money is so massive that it is often growing far faster than they can actually conceivably spend it. Maybe the best example of this is Mackenzie Scott who in 2019 got $35.6bn in Amazon stock when divorcing Bezos, has given away $14 billion (5.8 in 2020 alone!) AND Forbes has her at 34.9 billion in net worth (Amazon has done 30% better than VOO since 2019 for context. So, not counting her givings, it's the difference of about 5bn)
Ahh the ole if you can sense every particles position and velocity you can predict the future.
your comment really belies the desperation that exists now, these models are stuck where they are (hint it’s a natural limit), you are talking about exponentiation of cost to get what a 10% improvement? 5%? They are very few places for which it’s net positive to run them now, and most of those are incredibly shitty things like creating trash marketing content to drown us all in average inanity
I really feel bad for this next generation, they will just be constantly inundated with generated crap, so much of the high fidelity of conversation and meaning is and will be lost.
Desperation? We have had huge advances just from 1.5 years ago with things I wouldn't have thought would be possible in near 10 years. After decades of research with far slower progress and all of sudden we now know that we have hit a wall?
And I am not talking about predicting the future, but more predicting the next action to take based on current state, sensor data in a more seamless way. Like a human being reacting to different input, by moving their muscles etc. There would be huge amount of training data from there that could be incorporated into a single model.
> And I am not talking about predicting the future, but more predicting the next action to take based on current state, sensor data in a more seamless way.
Like self-driving cars?
Self-driving cars is an engineering problem, let alone an AI problem, and we still cannot solve it despite trillion dollar economic incentives.
Just putting together some LLMs on a fuckton of data does not work. Tesla tried that, and failed.
Tesla has been using sparse data to train their models, because they needed to prioritize fast on device inference.
Completely different solution applied to a completely different problem with completely different risk and quality tolerances with completely different mitigations.
The point is to try and see if LLM's wide general knowledge can have advantage in something like sensory data + action learning as well. Current self driving models don't have that.
I don't understand this stretch logic. It absolutely depends on the type of problem where they are inaccurate, how well trained they are in it, there is no way you can extrapolate like this.
You can ask them to do math equation which takes steps and if they are trained in that for certain problems they are accurate near 100 percent of the time.
Like ask gpt-4o to solve different variations of
"""What is the answer to 2x + 7 = 31?"""
If the numbers are of similar magnitude and simplicity, it will follow the same steps and be right 99%+ times, and I'm only not saying 100%, because I haven't tried it enough, but I don't see it being wrong.
For example """What is the answer to 2x + 4 = -6?"""
Just run a test yourself. Do random integers within 0 - 20, it will definitely not be incorrect 5% - 10% time. It will be correct 99%+ time.
Where is this number 5% - 10% even coming from? You could also keep asking it "What is the capital of France?" and it's going to be right 99%+ of the time.
There weren't really any advancements from around 2018. The majority of the 'advancements' were in the amount of parameters, training data, and its applications. What was the GPT-3 to ChatGPT transition? It involved fine-tuning, using specifically crafted training data. What changed from GPT-3 to GPT-4? It was the increase in the number of parameters, improved training data, and the addition of another modality. From GPT-4 to GPT-40? There was more optimization and the introduction of a new modality. The only thing left that could further improve models is to add one more modality, which could be video or other sensory inputs, along with some optimization and more parameters. We are approaching diminishing returns.
> There weren't really any advancements from around 2018.
Not sure what that means. Why are you marking those as "quotes".
The last actions brought so many returns. And it's unknown what the exact effect would be in adding more modalities, training data, optimisations and even just plain parameters.
Text as training data can only get you so far. Giving real time sensory data from many fields could allow LLM like system to control robots and get even more data from real life. E.g. robot hand movements, object tracking data, all of that to be fed into LLMs, and see how it would work.
My understanding (roughly) is that the way we got here was kind of by surprise. We've had a lot of the fundamental algorithms for a long time, but we ran into sort of a happy surprise when transformer models got scaled up - suddenly they started doing interesting things. Scaling them up even more made them start do do potentially useful things.
That happy discovery was never really a linear improvement path, though. We had an explosion of capability, but all along there have been active questions about how far the improvements would go with the current approach.
I think the point that a lot of researchers are making is that that we're starting to see those limits (with LLMs, at least).
There are also a lot of questions around business model and cost/value prop. Training and running these things at scale is enormously expensive. I'm seeing a lot of FOMO and gold rush mentality in the space, similar to the online streaming wars, and I'm not convinced of the long term viability of a lot of the companies. Especially once open models like llama are "good enough" and become commodities.
Of course, it's still early days and there's a ton of room for discovery, but it looks like we'll hit a limit with the current approach pretty soon.
Personally, I'd be OK with that. With the current state of things we have an interesting toy that can sometimes do useful work. It's an incremental quality of life improvement and another good tool in the chest, but it's not a civilization impacting technology.
My question still stands. How could anyone besides OpenAI be confident of there being a limit if no one has managed to even build as strong model so far as OpenAI has? Only Claude Opus seems close, but still weaker at reasoning than GPT-4o. Better at creative writing though.
And only after 1.5 years? And especially of we just had an happy surprise like you mentioned. How does it make sense to already start claiming that we have hit the limits. How do we know there is no more scaling, optimisations and happy surprises?
> That happy discovery was never really a linear improvement path, though. We had an explosion of capability, but all along there have been active questions about how far the improvements would go with the current approach.
> I think the point that a lot of researchers are making is that that we're starting to see those limits (with LLMs, at least).
The kinds of limitations we're "starting to see" are largely the same as they were a year ago. People were talking about it on here back then, but now it's becoming more apparent to more people as they get used to LLMs.
For those who saw it back then, this does look like we're hitting a limit. For others, not so much.
How do active questions about a technology imply we are approaching a brick wall?
How could researchers without having access to the latest state of the art - by OpenAI or any other unknown companies be able to even test that we could be approaching a brick wall? It seems to me that it would take trillions to find out what the exact limit is.
It's possible that we will get diminishing returns, but I don't see how we can confidently claim or know it?
> The kinds of limitations we're "starting to see" are largely the same as they were a year ago. People were talking about it on here back then, but now it's becoming more apparent to more people as they get used to LLMs.
I don't follow. GPT-3.5 was borderline useless at reasoning.
But it still seemed amazing and what I wouldn't have thought to be possible in any near future.
And then GPT-4 was a crazy advancement over that to me. And I've been using it daily since it was available, for various use-cases. Are you saying we are seeing the limitations of GPT-4 specifically? Because, sure, GPT-4 is far from AGI, but I don't see how this implies that further scaling, optimisation, training data improvements, techniques like multi modality and other potential strategies that I might not be aware of couldn't bring another explosive step?
Also the fact that GPT-4 reasoning skill hasn't been reproduced by anyone else so far seems to leave me thinking that everyone except OpenAI are clueless. Claude Opus is close, like I've said before, but not quite GPT-4 levels in specific reasoning tasks that I'm using the API for.
If you can't reproduce GPT-4, how could we trust the assessment that we have hit a limit?
Isn’t the statement “being stuck” a bit like an attempt at predicting the future? You don’t know how long something will stay stuck…
I think a very common error when it comes to personal learning or progress is confusing a plateau with a brick wall. The reason is, unless you have already walked the path, it’s not possible to differentiate them. And when it comes to progress, no one has already walked the path, hence no one knows actually.
- something that can't be modeled because there's no training data
- something that can't be modeled because it's fundamentally stochastic
- something that can't be modeled because the discrepancy in simulating the generating process, for your specific model, can, basically, be made arbitrarily large
I thought it was pretty good actually. Most of these leak disclosures usually say things like "We do not have evidence they accessed any secrets" or something like that, because they don't "know" what the hackers did once they were in. At least huggingface is saying "Yeah, they probably accessed secrets but we can't confirm it"
Any moderately well run shop will have a mechanism to get updates when a dependency of theirs has a security issues, depending on the line of business it may actually be required by a regulator or certification body (eg PCI etc)
We should probably be more afraid of the backdoors you can’t see in proprietary that would almost never be found.
This is how ALL open source used to be! Like literally ALL, this is the norm not this bullshit VC funded fremium restricted/tiered fuck the customer trap nonsense.
People built things because they loved it and wanted to help others , not to get rich. Now everyone just wants to get rich, and fast.
While I agree with your sentiment, maintaining software like postgreSQL is a full time job.
But your last sentence seems to apply to everything on the internet lately. People used to do podcasts, create guitar tabs or publish cooking recipes because it was their hobby and they wanted others to participate. Now everything seems about making money.
People would do these things for free because they had a stable job which guaranteed their material needs. Now every type of job can be automated and done better/cheaper by a machine, people will be forced to "monetize" everything that exists unless we get a literal revolution in how we tax and distribute the produced wealth.
It’s less automation and more about cheap labor. Content farms sprung up and flooded the landscape with worthless content to get a micro-slice of the pie.
Very discouraging to many content creators when their work is just going to be buried in SEO chaff.
Also, the automation wave is just beginning. Soon the human run content farms will be overwhelmed by AI created crap.
This is likely to happen in software as well. Every product will need to compete with some AI generated piece of garbage that’s barely passable functionally, but being sold at a fraction of the cost.
The jobs we’re talking about here, podcasting, development, etc aren’t jobs where everyone is forced out. Everyone is just more into making money these days and decide they want to make money doing those things rather than just fun. Let’s not try making excuses.
You are getting at it backwards. People are doing podcasts about investing, cooking, music production, <anything> because even those careers are being automated away and the money that they could be getting working is going away.
Even Software Engineers: take all the swaths of engineers who were productive but didn't want / didn't make to a FAANG company and now are having to compete in a world where most companies can replace a lot of the people they don't need a team of 8 engineers because their team of 4 now can have Co-Pilot and most of their "middle management" roles could be effectively replaced by some cheap, off-the-shelf SaaS.
I'm literally in this scenario. I'm too old to be interested in competing with someone who is 20 years younger than me but can call themselves a "programmer", and whatever knowledge/experience I have can be had at a fraction of my "cost" by using a commodified service that automates a process. So, what is left for me? Either I need to go downmarket and work for "programmer" jobs (further increasing the supply and lowering salaries) or I need to find someone who is willing to invest in my "idea for a startup" (thus getting into the Silicon-Valley way of life), or I need to find a way to take my unique experience and repackage as something of value - and then get to be called "greedy" by people like you.
I do not believe you can replace a competent developer with an AI, or say you have 2 and replace them with 1 dev and 1 AI.
You can't just type in ChatGPT something like "write me GTA5" and you get running code, just seen today an example of someone complaining that he asked soemthing like "Create a website in PHP for a company that does X" and they were expecting that by magic a website will just appear.
Aside from clueless people on Elance and upwork, no one goes to a developer and says "write me GTA5" or "make a website in PHP that does X", either.
What AI will do is leverage productivity of the individuals. Any new story will have its complexity reduced because the developer will be to use the existing codebase and say "hey, our current code is connecting with Foobar via the Zoberg SDK, now we are adding a customer that uses the BazBah platform and they need to change the order flow for 'deliver on payment' to 'deliver on invoice sent'. Show me what changes are needed to make this happen, and please write the integration tests to make sure that we are not breaking things from existing customers"
This goes from a one week task that will require three hour-long to something that can be done in an afternoon, reviewed by the developer and (most importantly) cheap to throw away if the original requirements change.
Does this work today?
I guess it might be able to write tests but does the rest just work?
In my experience the AI
- uses bad code practices because there is more bad code on the internet then good
- hallucinates APIs , so it tells you to use X but X does not exist in the library/framework you asked for
- suggests wrong solution
- if your language is not precise it gives you the answer to the wrong thing, like you see the answer and you realize it did not understand you
In my experience if your developers are 20% more productive you do not fire 20% of them because there always is a big backlog of features or bugs to be handled.
One of the reasons that I didn't drop out of college (almost 25 years ago) was because I was working part-time proofreading (and occasional translating tech manuals) for a translator who used to get about $25 per 1000 "touches". It could be good money for an experienced translator, but nowadays it's a dead profession outside of legal documents who need a certified notary.
Google's automatic translation was not good enough at the beginning to replace the translator's job, but by the time I was already graduated it was good enough for her to not need my proofreading and it was good enough for her to effectively get 60% of the job done. She has then effectively become the proofreader for a bad translator.
And nowadays, the bad translator is good enough to the point where her customers can just throw the original document on Google and do themselves the proofreading.
This is what will happen with programming tools. Code generation tools are still just at the "smart autocomplete" stage and the experienced programmer is still needed to act as reviewers, but as AI gets better, it will be cheaper to drop the "professional expert" altogether and let someone with tangential knowledge (maybe a product manager) in charge.
People still complain that machine translated Japanese is garbage so I bet will be the same with programming, some easy tass will be automated, complex stuff will be still done by humans with experience and understanding of the domain.
- There is not that much "complex" stuff going around for all the people that will be looking for a job in the field.
- what you call "garbage" might be someone else's "good enough for my needs". If I can go to Japan and a " garbage translator" still is enough for me to help navigate the city or poorly talk to a shopkeeper, then it's mission accomplished and I don't need to worry about a local guide.
- lots of "complex stuff" are dependent on context, and can be made less complex if we relax one single design constraint. E.g, centralized social media networks have a strong requirement for not losing user data. Distributed systems solve this by (a) duplicating data between every node and (b) letting it be deleted by users and node operators who do not want to have the data stored for long term.
It seems to me that you believe that what most software engineers is some dark magic that only a select few can master. It really isn't. The whole "software is eating the world" essay never mentioned what was going to happen after it ran of out of things to eat, now it is kind of obvious that it will gladly get into cannibalism.
My point was that your example was flawed, your translator friend can still have a a lot of work to do since the translators are average or garbage still.
A true intelligent AI sure could be a problem, but this stuff will just be an copilot, good enough to do basic stuff and maybe double check the programmer.
When you predict it would be possible I give the AI a JIRA ticked and it could open the application, reproduce the issue, update the ticket with details about the bug , then find the issue in a giant code base, fix it correctly etc .
Because today an AI can't do anything from the above. It can't replace a human.
My translator friend speaks no Japanese. She used to work with English, German and Portuguese. The fact that translators are still not Professional-level (yet?) is no consolation for the thousands of other professionals like her. She retired already.
> It can't replace a human.
If it provides enough leverage to today to make one person 20x more productive, then it is effectively replacing 19 humans. When it is effective to make one employee 200x more efficient, it will replace 199 humans.
And if you have enough hubris to think you are always going to be the lucky one out of the chopping block, it's not for lack of warning.
Sorry, my mistake. Replace "today" with "someday".
For "today", I've seen good engineers solving specific tasks in a third of the time already, but I won't make specific claims about absolute productivity multipliers.
>For "today", I've seen good engineers solving specific tasks in a third of the time
Specific is the important word here. Some boring tasks that can be automated in all jobs will be automated though you still need to check the AI. I assume no competent developer was fired because of that productivity boost in that specific task
You don't need to "fire" anyone for AI to cause a significant impact. All AI needs to do is to allow companies postpone hiring more people.
I really don't understand why you are being so obtuse about this. Do you honestly think that you can make the argument that software development (as an industry) is somehow immune to automation?
The main alternative to open source monetization is XKCD 2347 (one guy in Nebraska). PostgreSQL appears to have hit that sweet middle ground that is so rare in open source.
People should be more aware of what the license open source software is developed under allows.
Amazon can wrap an open source project in an AWS front end and create a paid for cloud service off the back of community effort. Or, key contributors can decide they want to take the existing code and change the license their contributions are released under going forward.
If the original license allows both these things to happen, then both are a risk and no one is being fooled.
Who cares if they do that? Do you see Torvalds and co running around crying because the entire world runs on Linux Kernels?
I would love nothing more than for a project I built or contributed to wound up as an AWS service.
Writing the code is just part of the value, running it is also very difficult. Especially as the use increases and expose new code paths and bugs and what not.
I think if a big company or two decided to lead development and charge for their Linux Linux kernels he’d have an issue as his influence etc would change. Also he is lucky in that he doesn’t have to care about the making money part. Companies have that issue.
Well I’m not sure if it’s just greed at the level at Amazon, Microsoft etc packaging your work and take all the support money from their vast influence.
OSS users don't complain about AWS wrapping around it. It's very much welcome.
The greedy people behind businesses managing OSS are concerned, because they are not satisfied with making money. They want to be THE ONLY ONES making LUDICROUS profits on top of community contributions.
I also have this feeling, but i do feel myself doubting from the lack of examples in this conversation. What are some recent examples of this type of scandal that we can use to solidify this conversation?
you can't fork and maintain everything yourself, and that de-facto lock in is exactly what companies bank on when they pull this kind of bait and switch. The idea is precisely to gain popularity with open source, "the first dose is free" style, and then capitalize on the dependency and popularity. Literally just the developer analog to the misleading "everything is free and always will be" advertisements of consumer facing software.
Ok find other people to help, that’s how open source works no?
There’s no issue here. Just whining. There is no lock in at all.
Even if it were OSI open source the maintainers like the very thread we are in could die. Then what? Oh you fork and maintain yourself, or the project rots.
Your comment leaves me scratching my head as it is all over the place.
We all need to just stop paying attention to Musk, he’s nothing more that a gussied up Musk supremcist dork with skin so thin he’s more translucent than he is white.
He’s made a career out of taking big risks with other people’s money and being saved by absurd fed policies while also milking the government for every last nickel he can squeeze out of it
> He’s made a career out of taking big risks with other people’s money
Quite a big statement about somebody that spent all his personal wealth to invest in Tesla and create SpaceX. Both companies in industries known to be startup unfriendly and places where he couldn't get much outside investment.
And before that, his first small startup, he reinvested that money into his second startup that then gave him the money to invest in Tesla/SpaceX.
By the time these companies got lots of outside investment they had proven their basic ability to operate. In case of SpaceX the flew Falcon 1 and had a contract for ISS resupply. In case of Tesla they had the Roadster working and had solid plans for Tesla Model S. And I think the investors in both companies were happy. Isn't this the whole point of startups?
You act like its easy to 'just get other peoples money and take big risks', and if that's all he did then maybe you would have an argument. But the whole point of why he is famous is that those investments turned into to major US companies.
> saved by absurd fed policies while also milking the government for every last nickel he can squeeze out of it
That's just factually incorrect.
In case of SpaceX they go ISS resupply contract, something that most people do not think is stupid. And they got it because they proved to be decent engineers and offered it at a low price, and the were successful. It was objectively one of the best contract NASA ever signed and you will find anybody on NASA who disagrees. Since then they have received many government contracts in addition to many private ones, non of those are stupid. Of all space companies in US history, SpaceX depends the least on the US government.
The first big federal help Tesla got was the investment in Model S production. All US automakers got money for 'next generation vehicle', Ford got way more then Tesla. GM was fully bailed out by the government at that time. And just FYI, when I last checked Ford had not paid back that money to the government. Tesla on the other hand paid that money back with interest ahead of schedule.
Beyond that the Fed did a bunch of stuff to encourage EV. Those policies helped all automakers, and often even foreign ones. One can argue that this actually hurts Tesla because it resulted in other automakers investing in EV not leaving the market open for Tesla to capture for longer.
Tesla received tax cuts and so on, but there Tesla is no different then any other company that does large manufacturing investments.
Encouraging EV is also not stupid. One can argue if its the best transportation policy (I don't think it is) but it isn't exactly stupid either.
And then who are you comparing them to? Against some idea perfect libertarian dream cooperation? If you compare them with actual real competitors, then making the argument that his companies are some government depended bloodsucker just doesn't hold up in the slightest.
