Interesting post. Did the difference in wall clock training time take the reduction in cold start time into account? Seems like that could be a significant factor for small jobs and negligible for large ones.
I hover somewhere between AI doomer and boomer. For a long time I thought all the economic gains and power produced by AI would be concentrated in a company like OpenAI and that the quality of our lives would hinge on decisions made by its stakeholders. Fortunately, that future no longer appears inevitable.
This is an excellent question. My very unscientific suspicion is that the decreases in average attention span and ability to concentrate zero out the theoretical possible increases in productivity that computers allow.
Definitely not the case for coding. I'm a capable senior engineer, and I know many other very experienced senior engineers who are all benefitting immensely from AI, both in the code editor and chat interfaces.
My company just redid our landing page. It would probably have taken a decent developer two weeks to build it out. Using AI to create the initial drafts, it took two days.
New PII redaction library that's SOTA and runs locally on a variety of devices. Kind of slow when I tested on my M3 Max macbook, but works pretty well!
I don't believe they used an LLM to gather or transform information ahead of generating this map (used https://wikinearby.toolforge.org, a database of geotagged articles), but the ease of processing the necessary information to create something like this using AI is part of the reason the future could be great.
We've looked into this tool a couple times, haven't adopted it yet. The main issue is compatibility with prisma, which we use for defining our db's schema and automating migrations. If there were a slick integration between the two, that would go a long way to convincing us to adopt. In the meantime, we'll go with convenience and (slightly, almost immeasurably) lower reliability. We haven't had downtime due to a migration in months!
Yes, this is the biggest issue right now and we’re working on a solution.
What we want to do is add the ability to generate the pgroll migrations based on the prisma generated migration files. Depending on the operation, you might need to add more info.
> What’s the benefit of generating embeddings for such large chunks?
Not an expert, but I believe now that we can fit more tokens into an LLM's context window, we can avoid a number of problems by providing additional context around any chunk of text that might be useful to the LLM. Solves the problem of misinterpretation of the important bit by the LLM.
