I have been working on a project that is built off of Cloudflare's pingora project - https://sbproxy.dev. This provides all of the features of a LiteLLM, but can also support your API traffic too. I worked on the edge infrastructure for Walmart, and took those learnings and build a fully open source gateway.
Take a look - should be able to address your needs.
Another reason golang is interesting for the gateway is having clear control of the supply chain at compile time. Tools like LiteLLM the supply chain attacks can have more impact at runtime, where the compiled binary helps.
SBproxy (https://sbproxy.dev) - An API and AI gateway as a single binary. Sits between your apps and LLM providers to give you more control. Think Cloudflare meets LiteLLM.
After the LiteLLM supply chain hack last month I feel pretty good about that choice. Your LLM gateway holds every provider API key you have. That probably shouldn’t be a pip install.
Worked on API gateway for a major retailer and incorporated my learnings into this platform.
Entropy is real. Microsoft has lessened the friction of purchasing vs their competitors. Public Sector this may be their only choice because they set the groundwork decades ago to do business with most organizations. It may not be the best solution, but it offers the least resistance to getting something up and running.
The supply chain impact on manufacturers in the US on contracts is overlooked in this tariff fiasco. For example - auto manufacturers enter into long term agreements as a fixed price for a product. The only negotiation point they may have to reset that price would be a government action, which a tariff checks the box. The genie is out of the bottle, lots of contracts will be negotiated and prices will go up.
I would like to think this is some kind of 4-d chess game to avoid rate hikes and to devalue the dollar, but on implementation it will accomplish none of the above with a sprinkle of recession.
This was a great throw back. My interest in Electrical Engineering was traced back to building out simple circuits on this device. Little did I know my day to day at school was going to involve even more complicated circuits on a bread board - I was in heaven!
This is an area that government subsidies could really influence change in urban planning and cutting oil demand. If there was a similar subsidy on bikes as there are on electric cars, I would expect the push back against bike infrastructure would become less. Right now in the Bay Area through poor design and aging infrastructure there is push back on bike lanes. An example is the Richmond bridge, which has a protected bike lane taking a 3rd lane of traffic that could see a larger number of riders if more ebikes become used. Likewise for the Bay Bridge, whose bike lane is a ghost town in the mornings when commute traffic is worst. This would be less of a problem if the lane went entirely to the city.
Also, shoutout to Vienna which has the coolest program I've heard of: businesses get subsidies for cargo e-bikes, and in return they loan them out to citizens for free. So if you've got a bunch of stuff to haul, you can borrow a cargo bike for free and use their incredibly good bike network to move your stuff. English summary: https://smartcity.wien.gv.at/en/approach/smart-city-made-sim....
Small point: you can't get across the Bay on a bike on the Bay bridge. You can only get from Emeryville to Yerba Buena Island. No passage from Yerba Buena Island to SF. We're hoping this changes sometime in the next decade but no one's holding their breath.
This is a major pain point moving customers from Oracle to cloud services. PL/SQL encapsulates a lot of logic in legacy enterprise applications that handcuffs customers to the platform similarly to mainframe systems. Migration tools can get part of the way there but still requires a strangler type pattern when migrating.
