Everything involves performing and actually proving what you know. If this is such an issue, then its something you need to fix. I have never actually met anyone who has this “perfomance anxiety” where they are so brilliant but do poorly on tests because of it. I think its a myth to attack rigor of academics. For knowledge workers everntually you have to go into court, or perform surgery, or do the taxes or give the presentation, or have the high pressure meeting. If anxiety is truly debilitating to the person all of these situations theyll be doomed so filter them out.
Keep the huge, complex business logic on the server whenever possible.
That doesn't work for webapps that are effectively entirely based on client side reactivity like Figma, though the list of projects that need to work like that is extremely low. Even for those style of apps I do wonder how far something like Phoenix LiveView might go towards the end goal.
I'd say driving only requires not to handle uncommon situation dangerously. And stopping when you can't handle something fits my criteria.
Also I'm not sure it's entirely AI's fault. What do you do when you realistically have to break some rules? Like here, I assume you'd have to cut someone off if you don't want to wait forever. Who's gonna build a car that breaks rules sometimes, and what regulator will approve it?
If you are driving a car on a public street and your solution to getting confused is stopping your car in the middle of the road wherever this confusion happens to arise, and sitting there for however long you are confused, you should not be driving a car in the first place. That includes AI cars.
What's the difference between B and C? API being down seems like an anomaly.
Also, you can't know how frequently you'll get 500s at the time you're doing integration, so you'll have to go back after some time to revisit log severities. Which doesn't sound optimal.
Exactly. What’s worse is that if you have something like a web service that calls an external API, when that API goes down your log is going to be littered with errors and possibly even tracebacks which is just noise. If you set up a simple “email me on error” kind of service you will get as many emails as there were user requests.
In theory some sort of internal API status tracking thing would be better that has some heuristic of is the API up or down and the error rate. It should warn you when the API is down and when it comes back up. Logging could still show an error or a warning for each request but you don’t need to get an email about each one.
I forgot to mention that for service B, the API being down is a common, daily occurrence and does not last long. The behavior of services A-C is from my real world experience.
I do mean revisiting the log seventies as the behavior of the API becomes known. You start off treating every error as a hard error. As you learn the behavior of the API over time, you adjust the logging and error handling accordingly.
Probably because it's too expensive. Prompt can be processed quickly but output tokens are much slower (and that means more expensive).
From my local test on a 13B model, output tokens are 20-30x more expensive than input tokens. So OpenAI's pricing structure is based on expectation that there's much more input than output tokens in an average response. It didn't matter too much if a small percentage of requests used all 4k tokens for output, but with 128k it's a different story.
I think I wrote pretty much the same comment here a few months ago. There's some good Langchain videos on YT to get you up to speed with the space in a day, then just go directly to the APIs you think are best suited to your problem.
Also: before using any new technology, go to hn.algolia.com and search HN for comments about it first.
I agree completely. I use it for inspiration and maybe a quick prototype to understand something, but I usually implement the pieces myself. Debugging LangChain performance and bugs is just an exercise in frustration.
reply