Not only do they give access to their home directory, but they tell it to organize and delete unneeded files without any specifiers. A friend, who is just a power user, had Claude Co-work deleting his downloads to "help make space" and it was deleting important files that there were no other copies of.
We are going to be dealing with this kind of thing for a very long time at this rate.
Yup. And meanwhile they are taking Tailscale to task for having them do curl | sh, which of course is also a bad thing but not quite as risky as this. It makes you wonder though:
- Given all this power would they abuse it?
- Has it already been abused?
- Could they be compelled to abuse it and then to keep it quiet?
> Over the past several months, our team has rebuilt the infrastructure behind Edge Functions, working closely with the team at Unikraft, who wrote about the experience from their side.
The unfortunate part of that graph is that I think it's not updated for today's prices given the memory shortage/crunch we're experiencing that's driven up the prices for all kinds of memory.
But also while it should be faster than it is, how would an S3 designed around SSDs look differently to an API user? I would think the API is basically the same.
It's hard to imagine how these API differences can be explained by the different underlying block device. I don't see any good reason you couldn't support these operations on a HDD.
I suspect it's more to do with the fact that with One Zone is a clean rewrite of large parts of the application stack that makes up S3.
S3 is made up of hundreds of microservices[0], there probably isn't anyone at Amazon that actually understands the whole system. Refactoring it to support these features probably requires coordination between a lot of different teams.
They might have petabytes of metadata, making a change to how metadata is persisted probably requires a massive risky data migration.
As someone with TBs of SSDs and HDDs laying around I am someone who agrees with your point, but this is not a good way to compare costs.
For one, this has no redundancy and doesn't factor the cost of the machine providing the storage. But also it's an upfront cost for storing 4TB. It would be implied you wouldn't literally store 4TB on the SSD because now you're out of capacity for more data. If you only have 2GB of data in a bucket then S3 is still magnitudes cheaper, faster, and more redundant than anything you could put together yourself because the cost is spread across all the users of the service.
Reality is there are so many times that S3 makes the most sense that it makes people short-circuit and always pick S3 despite the huge hidden IOPS and bandwidth costs everyone rightfully tries to point out.
If you have only 2GB of data it's trivial, it hardly matters how you store it. You might as well hold a full copy in RAM on each server and synchronize it with your developer laptop every half hour in case they all go offline at once, for all it matters.
To reliably hold a bit over 2GB of data in RAM on AWS, you'll probably need something like a t4g.medium, about $0.0336/hr on-demand. 730 hours in a month, so $24.528, but ideally you'll have like three or so instances so like $74/mo. That's before thinking about stuff like IP address costs, the EBS volumes underpinning those boxes, etc. And then managing all the syncing and what not.
Of course, that's list prices, you can get savings plans and RIs and discounts.
Once again proving that AWS is expensive any way you slice it. Even in today's RAM crisis, an extra 2GB is what, $50? and you probably already have 2GB free since they don't make RAM modules that small.
Just buying a few sticks of RAM isn't hosting data in that RAM indefinitely. Its not the same as what S3 (or similar services) gets you.
What is your datacenter costs for multiple different buildings (assuming colocation/office space)? How much are you going to pay for the electricity? How much are you paying for networking? How much is rent for each U of server? How much do you spend on payroll to have knowledgeable people to manage it 24 hours a day, 7 days a week? Suddenly its not just a one-time $50 payment.
Maybe you don't actually need all of that, just some slices of some of that whole stack of things. Probably true in many situations. But just looking at it as a couple of sticks of RAM is not a like-for-like comparison.
I'm not arguing the cloud is cheap. You can definitely go cheaper and choose your own cost optimizations when managing the hardware yourself, deciding what is actually a necessary cost and what isn't. I totally agree AWS overchages for a lot of what they do; they price their stuff at a premium because they know businesses will pay it. But hosting 2GB of stuff in RAM reliably and indefinitely to anywhere near the guarantees of S3 is going to be a lot more than just a single $50 payment.
Server? What server? Suddenly now I need to also buy a server? Sounds like quite a bit more than $50!
Things always seem free when you've already spent the money. It doesn't change the fact you spent quite a bit of money, you're just failing to account for it properly.
The standard here was I've just got 2GB of data I need to reliably store someplace. Its an expensive assumption I've got multiple servers in multiple physically separate buildings already lying around.
This is great and I think for any company/individual who is going to ensure their domain is registered and maintained this makes a lot of sense. It would be catastrophic, but there is no reason Github couldn't disappear or otherwise change some policies that require moving away from it. The way Go works makes committing these package names tied to Github so much more weighty than simply the place you pull from.
The one thing that I was worried about was returning 301 in the example Nginx config. If you ever wanted to change the url that clients are redirected to then any browsers that visited the old url config would be forced to go to the old config's redirect url. For `go ...` and `curl` it wouldn't matter, but Chrome/Firefox will cache that 301 permanently and break the intended redirect. Not sure if this is really an issue in practice though.
Some thing I've learned, but rarely seen explained anywhere: Storing the vectors is most likely not an issue, mostly likely you're having a problem with the indexes on top of them in which case you can use quantized vector indexes[0] (handled by pgvector) which will get past the limits imposed by PostgreSQL.
I had to switch off pgvecto.rs at some point and figured that out.
I don't have specific experience with the Azure environment here, but this probably applies if you have access to pgvector.
We are going to be dealing with this kind of thing for a very long time at this rate.
reply