Am I the only one who thinks this is exactly like it was before AI, when we used small batch hand crafted tokens made by organic engineers to find vulnerabilities?
Cheaper and more fungible. Companies pay lots of money for mediocre security audits. Most attackers aren't very good either. However it only takes one good attacker.
If the attacker and defender are using the same AI model, then (up to some inflection point) whoever spends more finds the most vulnerabilities.
I've always wondered if a hybrid system could work. You'd need a lot of voting infrastructure, and you need online voting, which means you need a reliable and quick method of online identification. Scandinavian countries fill those prerequisites, perhaps other places do too.
The idea is basically that you give a politician a mandate to use your vote. Whatever your chosen politician votes for will count as their and your vote. If you happen to disagree with your chosen politician on a given question, you can manually vote in that question. Your chosen representatives vote in that question will then be worth one vote less, since you've effectively used it yourself.
In the end we get the best of both worlds: voters don't have to vote in every single issue, but they can should they choose to. When they don't vote themselves, a politician they've chosen gets to use their vote, in a representative-like manner.
That's pretty much Switzerland. Indirect democracy for most things, but if enough people disagree with what the government does or they feel strongly about something the government isn't doing they can call a referendum.
CloudFlare loves pirates so much that they disclose loss of DMCA safe harbor protections as a material business risk on all their SEC filings. Piracy friendliness is key to their business model. It’s a risky position that no other large-scale CDN is willing to take.
Forcing piracy consumers to use Tor or other proxies is unlikely to be popular. They’ll still be used, for sure, but so long as CF makes pirated content easily accessible over the Internet, this is just going to keep happening. It’s just too damned convenient.
I don’t believe CF is going to win here, long term. If Spain and other countries block their ASNs, enough of their legitimate paying customers may start abandoning ship, and CF will have to get serious about unplugging notorious proxy configurations for piracy origin servers.
But cloudflare has no issue blocking the content if they receive a court order. The issue here is that La Liga wants to be able to get content blocked because they say so, and it has to be done right now.
I also don't support these organizations that destroy the sports that people love, force you to subscribe to different services as each game and "liga" has made their own deals to make as much money as possible. Until we remove the stupid amount of money that is involved in these sport events nothing will change. And now they are talking about other events like movies, series and live entertainment show. Hopefully they come for the VPNs next and break every business VPN tunnel whenever they want. Hopefully that will cause enough backlash that they finally fix this BS once and for all.
DMCA notices (and whatever the EU equivalents are) are designed to avoid the need for court orders. Every service provider that sends content is obligated by law to cease sending the content upon receipt of that notification. CF ignores them because they believe (mistakenly in my view) that the law doesn’t apply to them.
And every time they are sued for facilitating piracy, instead of letting the case to proceed to trial, they settle out of court.
Cloudflare famously ignores DMCA themselves for content they don't host, with their point of view that since they're a proxy and not a host they are not forced to comply, only pass the DMCA claim to the upstream.
Other than that, the legal situation on Spain is pretty dire for LaLiga. The Supreme Court already ruled in Spain that, as per the current writing of the law, football transmissions are _not_ works subject to copyright as they're not works of "art, literature and science":
https://www.poderjudicial.es/cgpj/ca/Poder-Judicial/Tribunal...
So it's likely that, if LaLiga sued Cloudflare or they made them party on any actual litigation, Cloudflare would defend themselves and possibly win. Therefore... they just don't sue them, only sue ISPs that have an incentive to just comply to any LaLiga request (as.. legal compliance and collaboration is one of the requirements for being able to buy rights to LaLiga matches in Spain. Yeah, no kidding, you can look it up in their public documentation).
Well, I lie. In a legal twist, they ended up suing Cloudflare for "participation in criminal activities", but not through the same avenue they sued the ISPs on (penal vs commerce court), with some interesting twists as accusations of "facilitating services to avoid the execution of a court order" - which doesn't make a lot of sense, as they're not even direct parties to that court order and they were denied taking part on it.
https://okdiario.com/economia/empresas/justicia-imputa-ceo-c...
I’m aware of how they rationalize it, and it’s bullshit. They compare themselves to a router that passes through packets unmolested. But that argument is trivially refuted by the fact that their IPs are what their customers' DNS queries resolve to, and the fact that without being explicitly configured to do so, their proxies will not serve content on behalf of an origin. L3 routers simply copy packets between interfaces. A CDN is significantly more complicated than that.
> Wtf? Just the other day I had chat about how stupid this is: they're blocking cloudflare to stop pirates!
Correction: they use the pirate excuse to make life of clients choosing competitors (like cloudflare) impossible. There is an overlap between some Cloudflare and Telefónica services.
Whether you store it or not, getting free energy from the sun for half the year is better than getting no energy from the sun. Every marginal reduction in fossil fuel usage helps.
Scandinavia is actually seeing a pattern where in the summer, there is so much electricity produced that it's approximately free, and in the winters, when solar panels produce nothing, there's not enough to go around so prices are sky high.
It's a very weird situation where it's financially difficult to build new power because you'll be doing it entirely for free half the year, but then you get 4-5 months that are an absolute goldmine.
Which is pretty much the ideal conditions for coal plants, so they make a killing during the winter and then shut off during the summer.
We need something that works throughout the winter so we can finally get rid of the coal plants the whole year!
We do not need more power in the summer though, that's covered by solar already.
Free electricity in summer and high prices in winter is not a bad place to be really — it provides a good incentive to develop long-term storage. Batteries are probably not it. Pumped hydro is good — not in Denmark (too flat), but maybe in nearby Norway. Maybe synthetic fuels could be produced and stored economically?
A mix of renewables definitely seems like the way to go, but I also wonder whether we might start to see some seasonal industry based on power prices - bitcoin mines, or even aluminum smelters, which only run during summertime? Though I suppose less capital-intensive processes would make more sense.
There is lots of talk of green steel in Sweden. The basic concept is to use the cheap energy during summers to produce hydrogen, and then burn that hydrogen to melt iron and make various alloys.
So far a few attempts have been made, lots of investments, but unfortunately it hasn't worked out yet.
Thanks for writing this! The visualisations really drive a better understanding than pure text does, and it's quite clear that you have a better understanding of what database do under the hood than I do.
As such, I have a question for you: contrary to your article, I've always been taught that random primary keys are better than sequential ones. The reason for this, I was told, was to avoid "hotspots". I guess it only really applies once sharding comes into play, and perhaps also only if your primary key is your sharding key, but I think that's a pretty common setup.
I'm not really sure how to formulate a concrete question here, I guess I would like to hear your thoughts on any tradeoffs on sequential Vs random keys in sharded setups? Is there a case there random keys are valid, or have I been taught nonsense?
B+trees combined with sequential IDs are great for writes. This is because we are essentially just appending new rows to the "linked list" at the bottom level of the tree. We can also keep a high fill % if we know there isn't a lot of data churn.
If you're sharding based purely on sequential ID ranges, then yes this is a problem. Its better practice to shard based on a hash of your ID, so sequential id assignments turn into non-sequential shard keys, keeping things evenly distributed.
Thanks! Another great article! It strikes me that modulo sharding on a sequential id would probably work rather well, but it was not mentioned in this article. Is there a reason I'm not seeing that this is bad? I guess resharding might be problematic, as you can't easily split a shard in two without rewriting every shard if you do that...
yes, that's the crux of the problem. when you have a sharded database, typically you want to be able to add (and/or remove) shards easily and non-disruptively.
for example - your database is currently sharded across N nodes, and it's overloaded due to increased traffic, so you want to increase it to N+1 nodes (or N+M nodes, which can add complexity in some cases)
if adding a shard causes a significant increase in load on the database, that's usually a non-starter for a production workload, because at the time you want to do it, the database is already overloaded
you can read about this in the original Dynamo paper [0] from almost 20 years ago - consistent hashing is used to select 3 of the N nodes to host a given key. when node N+1 is added, it joins the cluster in such a way that it will "take over" hosting 1/Nth of the data, from each of the N nodes - meaning that a) the joining process places a relatively small load on each of those N nodes and b) once the node is fully joined, it reduces overall load evenly across all N nodes.
For our DBs (which are often unsharded), we've found the best performance using the user account ID as the first part of the cluster key and then a sequential id for whatever the record is as the second.
It's not as good as just a sequential ID at keeping the fragmentation and data movement down. However, it does ultimately lead to the best write performance for us because the user data ends up likely still appending to an empty page. It allows for more concurrent writes to the same table because they aren't all fighting over that end page.
Spanner in particular wants random primary keys. But there are sharded DBMSes that still use sequential PKs, like Citus. There are also some use cases for semi-sequential PKs like uuid7.
I'm not an expert on Spanner, supposedly it's due to hotspotting. Your data is partitioned by primary key, and if you make that sequential, all new writes will hit the same master. https://docs.cloud.google.com/spanner/docs/schema-and-data-m... explicitly recommends a uuid4 or some other options
That's another thing, some say to use uuid7 for sharded DBs, but this is a serious counterexample.
Yes, on the one hand, they enable a lot of shady illegal business, but in the other hand, they also destroy the environment while doing it, so it's really a toss up whether cryptocurrency is good or bad overall!
Equating the concept of cryptographic currency with specific implementations such as proof-of-work just shows that you have no idea what you are talking about.
The importance of financial sovereignty can not be understated, whether you understand that or not.
> What problems are solved by financial sovereignty?
It's right there in the name. Some people believe that their assets should not be freezable or restricted by the whim of their local government-of-the-week. Cryptocurrencies have obviously solved this problem quite well or people wouldn't be complaining about how it has enabled more cybercrime (specialists, include cyber criminals, are often quicker to adopt trends than society at large).
Moving beyond that, the utility of a cryptographic smart contract system paves the way for the future of the internet. People forget computers are less than a hundred years old, and that there are thousands of years of computing ahead of us. The fundamentals will look very different one day.
Inability to evade the justice system is not something that most people would agree is a problem; quite the opposite. The rule of law is the thing that allows you and I to live in such relative splendor. If you remove the ability for courts to operate you will not be in a libertarian utopia, you will be a dystopian free for all where there is no one to uphold contracts or stop people from doing what they like when they have even a bit more power than you.
> The rule of law is the thing that allows you and I to live in such relative splendor
Speak for yourself, I grew up in poverty. I also said nothing about Libertarianism and did not endorse it; Please do not inject your biases, patronize or turn this into a straw man. Do you want to actually address my argument?
bitcoin is forecast to uses about 150 TWh of electricity this year vs all other datacenter operations foretasted to use 1000 TWh. Bitcoin is esitimated to be about 52.4% sustainable energy (renewables plus nuclear) where datacenters are 42% sustainable energy.
And those other datacenters are mostly doing useful things, while bitcoin is somewhere between pure waste and the least efficient way of doing security ever conceptualized. (A few dozen centralized nodes, set up right, would likely be more secure than the current mining pools.)
You don't think a human using an LLM to generate content that convinces another human to press the launch button is a concern? Sure seems like there's more than one thing we need to do.
Honestly? I really don't! What kind of content do you think would trigger that? If humans were launching nukes based on Facebook posts we'd all be long dead! A good deep fake might trick your grandma, but it's not very likely to fool military intelligence.
> What kind of content do you think would trigger that?
The kind of political propaganda that leads to the US reelecting a convicted rapist whose selects another rapist to lead the Department of Defense who then renames it to the Department of War and, true to the name, starts unilaterally attacking other countries.
If trump getting elected was due to AI, I wonder why every nation isn't electing similarly awful politicians? Hungary just elected a new president who seems a lot better than his predecessor, and a lot better than trump. The Canadian prime minister is genuinely one of the best politicians I've seen in my lifetime! The list goes on and on.
No blaming trump on anything other than the people who voted for him is like blaming school shootings on anything other than guns:a popular American passtime, and complete and utter nonsense.
Bear with me this digression into freedom of speech, before addressing your point.
The utilitarian argument for freedom of speech and expression in America finds its roots in the Marketplace of ideas.
Verification is frankly, the task of all our markets - to set up incentives for being right.
With no government interference in the exchange of ideas, citizens would be better able to discuss ideas, including those not popular with the establishment.
Since no one has a monopoly on truth, it would be through this competition, and fair traffic society would be better able to understand truth and thrive.
That worked, when we had newspapers that were funded, where the media landscape was not consolidated, and where we didn’t have an abundance of technology that overwhelmed our ability to verify and be informed.
Today, through entirely private forces, we can monopolize, fracture and shape the traffic in our marketplace of ideas.
Trump is very much the ideal candidate to ride the media environment. The right side of the political spectrum is simply a far more efficient at providing a wrestling style experience for its audience. Its consolidated media environment largely pays lip service to journalistic standards, and sells a coordinated set of ideas for its audience.
The Fox News effect is a case in point, and this was from the 90s.
This media model has been co-opted globally, with every party and government now providing patronage to media houses to keep them afloat, and to build their own narratives.
The citizen who engages in these media markets simply does not enter a vibrant competitive market anymore.
These mass-produced tokens are just cheaper...
reply