Hi there, I have some experience with reinforcement learning, PPO, RLVR, and I trained both video game AIs and sim2real robot policies.
In my experience, videos from professionals are not enough to train a robot, because you lack all/most of the edge and failure cases. Let me illustrate with pilots: The average Airbus can land itself via autopilot. Autopilot will fly it to the correct location. You confirm the frequencies, it connects to the airport's guidance system and lands itself. Super boring. Videos of that won't help anyone become a proper pilot. The reason pilots train for years is to handle the 1 in a 1000 case where the autopilot doesn't work. And you suddenly have to take over in the middle of a messy approach.
Translated back to your plumbers, the real training is when the robot is carrying a 100kg metal pipe and then the owner's kids come running and pass through under it. Every professional will give their best to immediately reduce efficiency by making things stationary and SAFE. That's the training data that is truly valuable.
BTW, I am considering bootstrapping a startup that uses video game engines to generate training data to make robot policies anticipate physical effects and counter-steer before they happen. Like putting down a heavy steel pipe before it drops due to gravity. With 2 bugfixes merged into the Unreal Engine, I have some experience there ;) So I might soon be in the business of selling custom training data.
... brought to you by storyteller.ai, which now has a defunct hero video, but apparently still 10 mio users (at $12+ monthly). I very much applaud them for pouring their (est.) $1.4B ARR into Open Source.
Deterministic feedback is precisely how frontier models are trained. It’s called RLVR. You let the agent run on a problem and then calculate a deterministic score of how well it did. Repeat 1000x times and you can “brute force” a good solution. (Which includes all thinking traces and you add it to your training data.) And then a Chinese model can copy your advance for 1000x less compute. Which is why US labs call this not learning, but a distillation “attack”. It’s an attack on the business model.
Labs typically pay $2k for each [prompt+scorer] docker image. So this is why frontier labs need so much cash and manpower and they see it as their moat.
Probably the code of these games leaked in some dark corner of the web in the past and that’s why AI can generate it so nicely. AI is also extremely good at building Quake 3, including Carmack’s notes.
But as soon as you are just a tiny bit off the beaten path, Fable/Sol will go postal regularly. So if your indie game has interesting gameplay mechanics, nobody will be able to 1-shot that for a long time. (Because their prompt would have to include everything that’s unique to your game.)
> You need to unambiguously describe what you need to be built
In the limit this can be as complex or more complex than just writing your program to begin with. If you actually care about your implementation details, your specification will balloon to encode all of them.
This is fundamentally about price. Does every service use EC2? No! Because at certain scales, the absolute price differences become too large. And then you celebrate the millions in additional profit generated by moving from EC2 to bare metal. It’ll be the same with SaaS: those companies keeping things AI-free will have higher profit margins. So the market will reward AI-free.
Remote: Yes. EU hours; US-East mornings work. On-site in northern Germany.
Willing to relocate: No
Technologies: C++, Python, PyTorch/TensorFlow, CUDA/SIMD; computer vision (retrieval at scale, inspection, SfM/SLAM); German speech recognition on-prem; real-time audio and kernel drivers; embedded/edge inference; RL environments (PPO, verifiable rewards); the boring enterprise plumbing around all of it
I'm the fractional ImageRights CTO, looking for a 2nd startup that needs hands-on tech, scale, or AI help.
I have 18 years of experience across 5 companies in taking products from idea to paying customers. I've built ImageRights' web search and image-retrieval database (C++, in service since 2015, finds copyright infringements worldwide), DeutscheKI (100% private German AI and local ASR), and before that the Spatial Audio Designer, used in Hollywood. And a German image distribution start-up.
I have experience with AI research, adaptation, and training. I've built AI RLVR training tasks in the past. Won the 2021 Bomberland mid-season with PPO. I'm also open to northern German on-prem CV/ASR work in case you cannot ship data to any cloud.
Side interests: I built a Pump It Up pad with custom self-developed and self-manufactured sensors. Currently doing CAD design and policy training (with PPO) for a robot.
if he “had been made aware that OpenAI has scraped and trained on information that was behind a paywall,” the company would have required OpenAI “to retrain its models.”
I got 3x Bose QuietComfort 35 2 by buying 1. They did honor their warranty, but my impression was that they plan for people not going through the effort of actually requesting a replacement item, even though the warranty says they can.
The first one, "Henckels", openly says that their stuff is from China.
But the second link, "Zwilling" says "Approved". But boy was I disappointed that their "Zwilling Peak" branded ceramic hybrid frying pan for $100 says "Made in China".
In my experience, videos from professionals are not enough to train a robot, because you lack all/most of the edge and failure cases. Let me illustrate with pilots: The average Airbus can land itself via autopilot. Autopilot will fly it to the correct location. You confirm the frequencies, it connects to the airport's guidance system and lands itself. Super boring. Videos of that won't help anyone become a proper pilot. The reason pilots train for years is to handle the 1 in a 1000 case where the autopilot doesn't work. And you suddenly have to take over in the middle of a messy approach.
Translated back to your plumbers, the real training is when the robot is carrying a 100kg metal pipe and then the owner's kids come running and pass through under it. Every professional will give their best to immediately reduce efficiency by making things stationary and SAFE. That's the training data that is truly valuable.
BTW, I am considering bootstrapping a startup that uses video game engines to generate training data to make robot policies anticipate physical effects and counter-steer before they happen. Like putting down a heavy steel pipe before it drops due to gravity. With 2 bugfixes merged into the Unreal Engine, I have some experience there ;) So I might soon be in the business of selling custom training data.
Feel free to email me at hn at hajo me
reply