Do we have Fable 5.2 vs Mythos 5.2 benchmarks?

I remember Mythos 5.1 being a tad better even for not cybersecurity tasks.

Why this won’t apply to you as well?

It could apply to me but I run evals every time someone at work complains about this and it’s never shown any more or less variance than regular usage. I don’t have statistical relevance with my evals but I do have a decent history with them.

People seem to underestimate how high normal variance is with these tools.

i'm not parent, but to answer your snark : there are efforts a person can take to try to reduce things like suggestability in ones' own personality. Same as anything, the brain can be trained, thus it should go without saying that some of us react differently in the face of suggestion -- both for the better and the worse.

this is cool, but not to be pedantic, voxels are usually way smaller. 1cm-10cm, these look like 1m.

AFAIK voxels are 3d pixels - of any size.

SerpApi | https://serpapi.com | Junior to Senior Fullstack Engineer multiple positions | Customer Success Engineer | Hiring Coordinator | Python/Ruby/PHP/Js/Rust/Kotlin/C#/Crystal/Nim/Elixir Developer Advocate positions | Based in Austin, TX but remote-first structure | Full-time | ONSITE or FULLY REMOTE | $150K - 180K a year 1099 for US or local avg + 20% for outside the US

SerpApi is the leading API to scrape and parse search engine results. We provide 100+ APIs to Google, Google Maps, Google Images, Bing, Baidu, and more.

Our current stack is Ruby, Rails, MongoDB, and React.JS. We are looking for more Junior and Senior FullStack Engineers.

We have an awesome work environment: We are a remote first company (before Covid!). We do continuous integration, continuous deployments, code reviews, code pairings, profit sharing, and most of communication is async via GitHub.

We value super strongly transparency, do open books, have a public roadmap, and contribute to the EFF.

Apply at: https://serpapi.com/careers

It seems like these jobs have been up for some time; have you been hiring a lot from them? Just waiting for the right person? It's been a bit difficult to raise any sort of email response related to them, at least.

For what it's worth, I've also contacted them multiple times with no followup.

I'm so sorry to hear this! Please email me at paige at serpapi.com and I'll be happy to look into this for you.

Hi! We have been hiring a lot of engineers! If you are having trouble receiving a response, feel free to email me at paige at serpapi.com and I'll be happy to look into it for you!

Hi Paige! I've dropped a mail for a junior position. Looking forward to your response!

Is this available for worldwide remote? I can work from the UK and Thailand.

Yes, we are hiring worldwide! In fact, we have team members across 26 different countries!

Do you accept remote from Vietnam?

Yes, we do! Feel free to apply, we'd love to review your application!

To be fair, I do do the same.

Fable 5.5?

Seems like it just overfitting the problems?

allot of the models are just constantly eval maxxing

Benchmarks seem amazing. Rivalzing with Opus 5.5 is something.

I wonder when the Artificial Analysis bench will be out?

> AI rarely seems to write better code than humans

The thing is since Fable 5+ generation of models this is not true anymore.

I have access to these models at work. I am not out of touch.

Is it? Go look at ai written pull requests on any open source project, even using the fancy models. The code works but they write needlessly verbose comments above everything, don't follow the existing code style, and reinvent the wheel instead of checking if the source already has implementations for things.

I apologize for my skepticism, but I have literally heard this same claim made with every single generation of models that comes out.

For a while, I dutifully believed it, and would go try again, and it would be awful.

I'd then have another conversation like this, where someone would say something along the lines of, "sure, sure but this NEW model now fixes all that and is really awesome!"

I'll revisit it when I start seeing some real studies that show real productivity increases rather than stuff that just shifts the burden of broken slop onto PR reviewers while claiming awesome speed.

You may also consider that developers (like any person in any profession) exist on spectrum of competency, and you may just be better than most of them.

But models are climbing their way up the percentile charts.

90% of the software development in the work, by volume, can be done by people who are in the 50th percentile of skill.

So these model advances do meaningfully expand the amount of "useful" work that can be done without people. And simultaneously not be "good enough" for the specific kind of work you find yourself doing day to day.

Both can be true.

Grab the last 5,000 lines of code you wrote, and that you thought was bug-free. Run it through Opus 5.5 or Fable 5.1 and post what it finds (and be honest about it, given that nothing is at stake but an Internet argument.)

This is a poor test. Parity would be utilizing the models to implement the functionality of the 5k lines and comparing the outputs.

When's the last time you did that?

Never. I'm also not the OP you responded to. I'm just pointing out the flaw in your proposed test.

Try it, then get back to me about whether the test is "flawed." I guarantee that a frontier model will find more bugs in your code than you will find it a similar volume of code it wrote. I absolutely guarantee it.

Every developer needs to experience this particular flavor of ego death. Me, you, everybody.

>Every developer needs to experience this particular flavor of ego death. Me, you, everybody.

You seem like a deeply unpleasant person.

(Shrug) HN isn't a place where ignorance should be celebrated; there are plenty of other venues for that.

If you don’t care about the solutions, maybe you never cared about the issues themselves?

Math has practical applications that hopefully will improve lives.