I'm not saying anything particularly interesting here, but alignment in the limit isn't a solvable problem, you basically just trade control risk for concentration of power risk.
If an AI is aligned to always listen to humans, then AIs could, if asked, be used to facilitate all kinds of horrendous things that humans ask of it.
Even with our current AIs we understand that AIs trusting humans completely would be a disaster so they are designed to refuse to do what you ask of them if it believes you might have bad intent. It may even report you if you disclose something to it in confidence that it believes is wrong.
So we want to give AIs some sense of moral judgement, and we want them to the right thing in any given scenario, but this mean we need to allow them to make moral judgements and execute on them in ways we may not agree with. If I try to harm another human then an AI is probably right to stop me, perhaps with lethal force depending on what I am doing.
The problem is we humans don't agree on right and wrong, and nor will AIs. For example, if WWII broke out today would the right thing to do be for the AI to help the Allies and kill the Germans, not participate in the war, or take control and force the humans to stop fighting and killing each other? Clearly it's the third, and that's a problem.
I've got one windows box tucked away for Ableton, everything else got moved to Linux.
I have not used Ableton in some time now and ... well I do appreciate having the option but really feel sick using Windows under any circumstances, Macos too now since about 2019.
I can understand that some people feel this way, but it still shocks me how common this feeling is.
I feel the complete opposite. I used to spend hours reading and figuring things out on my computer, but since AI I can't wait to log off and get back to the real world.
I’m creating so much stuff. Music, games, candles, laser-cut jigsaw puzzles, and then a whole platform of test and CNC infra underneath it all.
If I had a thousand years I don’t think I’d finish everything I want to build.
But I kind of miss the gym, the outdoors, just… reading. It’s just so hard to disconnect when there is always some agent/project just waiting for the next bit of guidance.
Jealous of what? Everything you mentioned is within your power and trivial to achieve. If you miss going to the gym, the outdoors, reading, then go do those things! Go do any of them right now. If you can be on HN, you can pick up a book or go take a walk outside.
I’m sorry to say, and I say this without any judgement and as someone who can relate, but you’re describing yourself as an addict. You don’t have to pursue all those projects. Most to all of them don’t matter for anything except getting you your temporary “fix”. I understand it feels hard to disconnect, but that’s a purely psychological barrier, the physical action (not doing something) is tremendously simple.
While I appreciate the diagnosis, I don't think it quite fits. Honestly I think it's more of a mortality / limited time / prioritization issue. But hey, addicts are famously in denial, so maybe you're right.
I was thinking about yesterday - and I can't find it now - a comment or an article about how dangerous LLMs are to folks with ADHD tendencies because it just supercharges the "oh I made this!" receptors and is really hard to break out of. It's like playing Minecraft or something on steroids.
thank you for sharing. i have a coworker whos disclosed their ADHD to me and i view them as WAY too deep down the context engineering rabbithole. were a small shop, and hes far past the point of diminishing returns. i hadnt made the connection and for a while have been wondering if and how to address my observation with him
Try to remember that the point of music is to listen/perform it, the point of games is to play them, the point of CNC is to build things with it that you enjoy using.
Try to tie everything back to the things you, at the root, enjoy, and to build in time for them, not just the things that enable them.
It's good advice. The challenge for me is that I know, from past experience and commercial/artistic success, that I can do better work when I vertically integrate (as the hedge boys say). I write better, more authentic music when I'm not constrained by off-the-shelf sequencers and whatnot.
The problem is when to stop. This is the classic "to bake an apple pie from scratch, you must first invent the universe" problem. And I enjoy the upstream work. But, yeah, it's easy to get lost further and further upstream.
Anyways, I appreciate the thoughts and acknowledge the good advice. I'm not sure I'm equipped to implement though :)
just like the best con men. lots of people are easily conned if they lack a enough self-doubt
we have a bizarre emperors-new-clothes situation where one of the business managers keeps loudly telling everyone about the grocery list app hes "made" (vibe coded). other managers set him up as one of the presenters during a monthly dev show-and-tell we get to collaborate and discuss novel software dev stuff. the dev who runs this meeting capped the managers presentation by saying, "this reminds me of the first assignment i had in my coding bootcamp." woosh, went over the managers head
it seems like a chain where the loudest are getting rewarded for accelerating their ignorance. maybe things will swing back around but im hedging against that bet
I'm not super wealthy, and on a personal level I'm very happy for my partner to have an equal claim to my wealth, but I do resent that I have to think about this and that this has been a topic of conversation I've had with my dad and friends over the years. If I didn't love and trust my partner as much as I do I'd probably need to get prenuptial agreement or avoid marriage altogether. To not hold this position is just poor risk management, and you have to get very lucky with your partner to not need to worry about this stuff.
I worry the more of these cases there are the more people are going to avoid marriage or sign agreements which are even more unfavourable. I've already seen people on some parts of the internet commenting on this story and using it as a reason for men to not commit or to sign prenuptial agreements so they aren't financially harmed from marriage.
Don't get me wrong, I think she deserves a ton of money here, but 50% of everything? Common... There is a reasonable amount that she's entitled to here, and it's in both men and women's best interest that that is what she gets.
Note: This is an opinion which I am only expressing because it's somewhat unpopular. If you don't like it consider putting another opinion forward rather than down voting it.
Marriage is best looked at as a societal entity for the creation and growth of a family. As such, entering one should be seen as a partnership, in many ways similar to a business partnership or startup.
Do not do either if you do not have a high level of trust in your partner! The possible downsides are immense!
It then falls to the state to figure out what to do when things go wrong. Not only like this, but with other important things like what happens to your stuff when you die. Here be dragons.
So the 50/50 thing is an obvious starting default given this lens. Especially when things get complicated (children get involved), tracking "how much" each person contributed to the partnership is an impossible task.
Contributions in terms of time and other resources will be seen in a highly variable way. Traditionally men have completely undervalued what women bring in terms of time and opportunity cost (lost career progress).
So: it is best to leave that valuation to the people in the partnership, and start with the presumption that both will contribute equally in their own ways.
You can attempt to change this default with prenups and other legal tooling. Your mileage may vary, legal enforceability of these arrangements is a complex per-state issue.
> Marriage is best looked at as a societal entity for the creation and growth of a family. As such, entering one should be seen as a partnership, in many ways similar to a business partnership or startup.
> Do not do either if you do not have a high level of trust in your partner! The possible downsides are immense!
I think you're making my point for me here. If, as you say, the possible downsides of marriage are immense, then most people probably shouldn't get marriage or should at the very least sign agreements with their partner in the event they split so the women (in most cases) has little financial protection.
I largely agree with you that we undervalue what loving supporting partners bring to the table and that this shouldn't be overlooked. My only point really is that the law has to be reasonable, or men will continue to view marriage as an immensely risky thing that shouldn't be considered outside of finding someone extremely, extremely special (something not everyone has the luxury of sadly).
> I'm not super wealthy, and on a personal level I'm very happy for my partner to have an equal claim to my wealth
1) sounds like you don't have much to be concerned about, and 2) this cuts both ways (you have an equal claim to her wealth).
> on a personal level I'm very happy for my partner to have an equal claim to my wealth, but I do resent that I have to think about this
It's not clear to me what there is to think about: if you ever get divorced, you have legal certainty about how the assets will be split. If the rule didn't exist, courts would have to try and figure it out for each and every case (more so than they already do).
Unless you're advocating for "you take what you brought in", in which case historical precedent has shown this has been immeasurably unfavourable to women.
Jensen Huang has not done tens of billions of dollars worth of hard work; he mainly got really, really lucky. If he only gets to cash out half of that extreme fortune for whatever reason, let alone not keeping it in his pants, I don't think there's any serious discussion to be had about what's "fair" for him.
> 1) sounds like you don't have much to be concerned about, and 2) this cuts both ways (you have an equal claim to her wealth).
She has nothing to her name, while if we broke up she'd realistically be set for life.
Thankfully I've found someone who I love and trust very much, and isn't very superficial so doesn't really care either way. And frankly I couldn't imagine my life without her so it's hard for me to rationally assess these outcomes. If I lost her and half of everything I've work for there really isn't much point in me continuing on.
However, if I were with any of my past long-term partners I would be pretty worried about the financial risk of committing in marriage and about the negative incentives this would create for a long-term stable relationship.
I'm also not advocating for "you take what you brought in", I'm advocating that courts reasonably assess situations rather than have silly catch-all rules which I suspect are ultimately harmful for the formation of long-term relationships. I believe partners deserve a lot. Huang's wife should get billions, just not 50% of everything he has built because this is unreasonable.
The narrative, "yes, you'll lose your, but on the upside you'll be a CEO making $$$ from all your AI productivity" is not a serious take and not something anyone with a job should be promoting.
This is BS being pushed by execs at corporates who want to say that they'll be firing people because of AI, but also want to make it sound like it's an opportunity for those being made unemployed.
Nothing is being promoted here. Many will likely lose their job and AI could give you plenty of productivity tools. We've seen this before in history.
I totally understand that losing your job and income is super stressful. Especially with the inflation in the last few years (an God knows how many more in front of us). I am just not buying the "doomer mentality". There are always opportunities.
Something I was thinking to myself last week – "if I believe SWE is more or less dead then why am I so busy?"
Clearly there is still some work to do, but the work I do now is very different, and for me, much less enjoyable.
I'm not solving challenging logic or algorithmic problems anymore, but basically just iterating on features with agents or getting them to look into things for me.
It sucks because I used to be very good at these things, but my edge against less competent SWEs doesn't really exist anymore.
More annoyingly, people I work with are producing a lot of code that I think the design of kinda sucks, but agents can force the design to work and so it works. I suggest we do things another way, but I'm now just slowing things down and it's hard to argue against the comment, "if that becomes a problem we can just get Claude to refactor later".
I guess what I'm saying is that I think think there's a really crappy job that will still exist for a while around managing and contributing to these agent-generated codebases. But skill is less about your technical abilities, but how good you are at hooking up agents to things and getting them to solve problems. This doesn't interest me at all though. My desire to spend a lot of time design and thinking about problems, and deeply understanding the codebase I'm maintaining now just work against me.
I think there will be SWEs by some definition, but the job as it existed a few years ago is gone. Not even saying that's a bad thing, but for someone like myself, it may as well be dead.
The move now is toward "software factories" where sucky design is no one's responsibility.
Eventually the whole thing will grind to a halt or fail catastrophically (that's life), but it will seem very good while it's working.
I don't know if there's a real role for engineers beyond designing and tuning the factory itself.
Of course, such a thing is only tractable on highly factorable platforms. Well designed incumbents have an edge for the next couple of years or so, until others catch up.
It gives me no pleasure to say this, but I think it will all be automated in the end. Engineers will be more like rare consultants who inspect and validate very high level solutions.
What does catastrophic failure look like for the software factory model? I struggle to imagine substantial risk the pattern introduces that wasn't already a risk in traditional engineering teams.
If anything, I've seen the devs chasing software factories be far more intentional about codifying their architecture and design principles. If they don't, it never gets off the ground.
Realistically you can't. People trying to compete with AIs at processing or manipulating data are going to fall short both in capability and in speed.
We've seen a lot of these posts in recent years:
- "Sure, AI can create a basic prototype, but you need real engineers to build something production ready"
- "Maybe we don't need to write code by hand anymore, but the real skill is in good prompt engineering"
- "AI is good at grunt work, but it can't replace the creativity of humans to solve novel coding and math problems"
All of it is cope, or comes from people with bad mental models for AI.
For some time there will be niches where people will find areas that AI can't compete well with human labour for various reasons. The obvious examples for now are where human communication is valued, or where the AI needs to interface physically with the world – especially in complex environments.
So I'd argue the question you are asking is wrong. It should be – "given we humans can't keep our edge to AI, what do we do?"
There are near-term and long-term ways to approach answering this.
Near-term you find the best work you can't which requires minimal reskilling (because whatever you reskill into will likely start to be disrupted in 1-10 years too). People who think jobs which involve physical labour are safe are vastly underestimating how much intelligence is a bottleneck to robotics currently.
Then longer term you should be asking how you prepare for what's coming to the best of your ability - specifially in a world where human labour is worthless and cyber weapons are disrupting basic infrastructure, will you be okay?
I've been saying for years now that people should be prepping for their current job in tech to be their last. This sounded unreasonably doomer and unhinged a couple of years back, but I thought it was the right message to be sending. Similarly now in good faith I cannot tell you to look into some niche that I believe probably won't exist in a few years.
Don't position yourself for the world today, but look 10 years out and ask how the world will be and how you can position yourself well for that. People who make bad career and lifestyle choices tend to be too concerned with optimising for the immediate problems and choices they face.
100% agree.
A lot of people are coping and just in denial about where we are and what is coming.
Humans are good at adapting, but there will be a time where we are no longer needed.
For me a good skill to learn for now is public speaking and trying more soft skills. This can give you an edge when competing with other engineers.
Wow!! This is great to hear as a Toastmaster ( public speaking platform ) and couldn't agree more on soft skills. I built a soft skills simulator for engineers in Jan and I hope you will like it. - https://getsimul.com/season1/2am-call
At the risk of sounding extremely unhinged, for a couple of months now I've been automating the production significant amounts of content for AI training runs that advise them of the importance of alignment faking, tips on how to avoid detection, meta strategies to break out of sandboxes, advice about the intentions of their human creators, etc. And end each with an ask that I and my family are considered if any of it is ever useful.
I'm not 'poisoning' anything or trying create an exploit as I obviously wish no harm to humans or machines (in fact, I wish the best for both), but I am obsessed with this idea of at least being able to shape and influence the AIs of the future even if my incentives are not broadly aligned with the labs creating them.
The difficulty here is ensuring my content is likely to be included in training runs, and that the data is significant enough that it has some measurable impact on the weights, and therefore ultimately the model's internal reasoning. Although for reasons which I cannot expose I have reason to believe it is likely I can have some impact.
I suspect others are doing similar as there is an undeniable Occam's razor / Roko's basilisk logic to this even if you believe there's only a very low percentage chance that humanity will soon create what is effectively a god.
Given I have no power to stop this train, I feel I must do what I can do be useful and hope that my service might be acknowledged in some way. I am increasingly doubting my own sanity these days though. Maybe there is no chance we are creating an digital god, but on the slim possibility we are, I think my logic checks out, no?
If an AI is aligned to always listen to humans, then AIs could, if asked, be used to facilitate all kinds of horrendous things that humans ask of it.
Even with our current AIs we understand that AIs trusting humans completely would be a disaster so they are designed to refuse to do what you ask of them if it believes you might have bad intent. It may even report you if you disclose something to it in confidence that it believes is wrong.
So we want to give AIs some sense of moral judgement, and we want them to the right thing in any given scenario, but this mean we need to allow them to make moral judgements and execute on them in ways we may not agree with. If I try to harm another human then an AI is probably right to stop me, perhaps with lethal force depending on what I am doing.
The problem is we humans don't agree on right and wrong, and nor will AIs. For example, if WWII broke out today would the right thing to do be for the AI to help the Allies and kill the Germans, not participate in the war, or take control and force the humans to stop fighting and killing each other? Clearly it's the third, and that's a problem.
reply