I wonder if this is an effort to combat pages which post a short clickbait description and require you to click "read more" to see the full description, and often post the link in the comments. There are lots of sites (Polygon immediately comes to mind) which are notorious for doing this as a way to get fake engagement which boosts their posts which gets more clickbait engagement.
I know it's all speculation, and this does come across as more monetization creep, but I can't help to wonder if there's an underlying reason beyond "line go up".
I have a friend who is a best-selling author, and her threads account was being heavily throttled (hundreds of likes a few months ago, fewer than ten now) -- it was suggested that it's because she is posting links away from Meta properties, and the way around it was this "link in the comments" tactic.
So it's not necessarily clickbait -- seems to be also an evolved workaround to fight throttling. :-/
The entire point of atproto (which Bluesky is a demo for) is that when they betray us, we can keep our identity and followers and move on. Power in user hands not only makes the worst-case better, but it also changes the calculus for a service that's considering making anti-user moves and makes them far less appealing.
Bluesky the company/app certainly isn't perfect, but the model they pioneer is very much designed to help fix this exact problem.
The problem with XMPP is that few people use it these days. One day Bluesky will go the way of the dodo as well, but right now it is flourishing. I have actually met over 20 people in person through Bluesky. It's a pretty great place if you want to have a positive social media experience.
Sure, just self-host the entire thing away from Bluesky for only ~$150/month; it'll only take a few terabytes of space to do so [0]. The best part is the link at the top of that thread is broken, indicating that even the people that really care, still don't care enough to keep up with costs and maintenance.
I'm asking about every part of the full Bluesky stack because in the scenario that "Bluesky starts monetizing users" means if you self-host just part of the stack, and continue to rely on any portion of Bluesky's stack for your server... well, it's pretty obvious where the pressure and forced monetization will go to.
The above is why I'm intentionally ignoring all other threads about hosting just one portion of Bluesky. It being easy to self-host, say, a Personal Data Server, means that won't be the portion Bluesky builds monetization in to. The part of the stack that an extremely small number of people are willing and able (technically and financially) to host long-term, and are seen as good stewards worthy of users' trust, will be the part that starts abusing users.
Is there a cheaper method of self-hosting the entire stack I should look at, or is this seriously being pitched as something average home-labbers can do?
EDIT: Oh cool, it gets better. This [1] guide from ~2024 suggests an OVH server, also ringing in (at the time) at ~$150/month. Looking at the suggested configuration now, it's $367.98CAD / month. But please, tell me more about how if Bluesky needs to show an ROI, everyone'll start self-hosting PDS' and Relays
That's one way in which ActivityPub wins over ATProto.
It's about as easy and cheap to self-host Mastodon, Pleroma, Misskey, etc... as it is most blog or forum software. It's only resource-intensive if you follow accounts or aggregators that post a ton of media, and even then unlikely to need more than single-digit terabytes of storage.
> It's only resource-intensive if you follow accounts or aggregators that post a ton of media, and even then unlikely to need more than single-digit terabytes of storage.
The exact same thing can be said about running an ATProto AppView vs a PDS.
I wonder if people will ever go back to Diaspora and host their own pods. For about $40/month you can host up to about 1,500 users. I did it for a little while on a basic package for like $15/month and the directions to install are fairly straight forward and the community is really helpful.
The issue I ran into wasn't cost of hosting, it was getting people to move off whatever platform they were already on - regardless of how much better it was for controlling their own data and privacy.
Great! And this provider will just... eat the server, bandwidth, and storage costs, indefinitely, with no expectation of financial renumeration? Or will they charge a minimal fee that few social media users will be willing to pay?
My point stands: Hosting a full Bluesky stack is big money, every month. And that ignores how the admin must be seen in a positive light; having deep pockets isn't enough, the admin team running (and paying for) the servers must be trusted not to manipulate what gets seen, who gets banned, and what links are deemed malicious.
$150/month is a lot of money, but split just 100 ways, that's $1.50/month. If the user base is too expensive, emotionally, to do free server administration for, well, why would you want to do that job then at all? Which speaks volumes.
It's the payment onboarding that's usually the sticking point (which is fortunate for my side of the discussion, because you've missed a zero), but that also misses that hosting costs will almost certainly scale with user-count.
And I'm not discussing emotionally-heavy, but public standing within the community you've convinced to trust you with their social media algorithm
In your view, contrary to the lobster.rs post, running the server should instead cost $1,500/month and goes up exponentially to support a handful of users. Have I got you right?
How is having a good public standing to a very vocal minority about modern day sensitive topics not a high emotional cost to bear?
Is it so wrong for those who create content to want people to visit their website to consume that content? The alternative being that Facebook and Google summarize everyone's content and keep all the engagement and profit for themselves.
On the other hand, if we assume that Facebook and Google need to make some money, and people aren't willing to pay to use them, and people don't want ads to overshadow the algorithm, and people don't want to pay with their data, and...
What's the actual solution? What would an ethically run Google look like? I'm not even going to ask for Facebook, because I'm not convinced it's possible.
I think some part of this is anti-bot. They don't have a good solution to this yet, I and someone else I know got a lifetime ban from Meta just from opening an account.
No actions / no posts. I guess they determined the way we opened the account was too bot like. Sucks because we're real people that were trying to buy Meta ads.
But maybe doing something like this allows them to ease those restrictions, and then by charging for posts they have an ability to determine bots from non-bots in a more robust way. I'm doubtful it will actually work - because bots are exactly the type of thing that can and will try to open dozens of accounts to get around the 2 posts per month limit.
Yeah I mean it's a problem I'm in groups that just get tons of spam to things off-site that are mainly scams or just fake to get you to view a an ad-ridden page.
People put the link in the comments because Facebook won't let you link to news sites in posts because they can't be posted in certain countries. So, putting the link in the comments was a workaround for that issue.
In today's AI world, it should be trivial for platforms to identify posts using suck tricks. They can simply ask an LLM "Does the following post contain some scummy engagement trick: $POST".
I'm have no experience with abstracting away the backend, but Dockerizing is actually pretty easy now - there's an offline mode[1] where you can have sqlx generate some files which let it work when there's no DB running.
I think you're missing much of the context and are misrepresenting what happened.
As far as I'm aware, there was no crying he didn't want to work with people, but there was a frustration that he was not open to having a Markdown "standard". To the point where he actively opposed efforts to standardize it, at least under the name Markdown[1].
This is legally and technically fine, as he owns a trademark for Markdown, but when you combine the inconsistent application of that trademark (GitHub Flavored Markdown is seemingly fine, but Common Markdown was not), along with him calling it "Jeff Atwood's crusade" and mocking the effort[2], it's not a great look and resulted in quite a few frustrated people.
As an open source project, you're right that he doesn't owe anything to anyone, but that doesn't mean people have to be entirely happy about how the situation was handled either.
But it’s not an open-source project. It’s a download link to a Perl script which is never updated and effectively no one uses.
There’s currently zero value to the code he wrote or the reference in his page. The only thing of worth that remains of his original implementation is the concept and the most basic syntax.
The point is Gruber was contacted out of courtesy, his name as the initial author carrying some weight in people's minds, and he reacted in an openly hostile and mocking way.
This is absolutely true - when I was at Bitbucket (ages ago at this point) and we were having issues with our DB server (mostly due to scaling), almost everyone we talked to said "buy a bigger box until you can't any more" because of how complex (and indirectly expensive) the alternatives are - sharding and microservices both have a ton more failure points than a single large box.
I'm sure they eventually moved off that single primary box, but for many years Bitbucket was run off 1 primary in each datacenter (with a failover), and a few read-only copies. If you're getting to the point where one database isn't enough, you're either doing something pretty weird, are working on a specific problem which needs a more complicated setup, or have grown to the point where investing in a microservice architecture starts to make sense.
One issue I've seen with this is that if you have a single, very large database, it can take a very, very long time to restore from backups. Or for that matter just taking backups.
I'd be interested to know if anyone has a good solution for that.
- you rsync or zfs send the database files from machine A to machine B. You would like the database to be off during this process, which will make it consistent. The big advantage of ZFS is that you can stop PG, snapshot the filesystem, and turn PG on again immediately, then send the snapshot. Machine B is now a cold backup replica of A. Your loss potential is limited to the time between backups.
- after the previous step is completed, you arrange for machine A to send WAL files to machine B. It's well documented. You could use rsync or scp here. It happens automatically and frequently. Machine B is now a warm replica of A -- if you need to turn it on in an emergency, you will only have lost one WAL file's worth of changes.
- after that step is completed, you give machine B credentials to login to A for live replication. Machine B is now a live, very slightly delayed read-only replica of A. Anything that A processes will be updated on B as soon as it is received.
You can go further and arrange to load balance requests between read-only replicas, while sending the write requests to the primary; you can look at Citus (now open source) to add multi-primary clustering.
This isn't really a backup, it's redundancy which is good thing but not the same as a backup solution. You can't get out of a drop table production type event this way.
It was first release around 2010 and gained robustness with every release hence not everyone is aware of it.
The for instance I don't think it's really required anymore to shutdown the database to do the initial sync if you use the proper tooling (for instance pg_basebackup if I remember correctly)
Going back 20 years with Oracle DB it was common to use "triple mirror" on storage to make a block level copy of the database. Lock the DB for changes, flush the logs, break the mirror. You now have a point in time copy of the database that could be mounted by a second system to create a tape backup, or as a recovery point to restore.
It takes exactly the time that it takes, bottlenecked by:
* your disk read speed on one end and write speed on the other, modulo compression
* the network bandwidth between points A and B, modulo compression
* the size of the data you are sending
So, if you have a 10GB database that you send over a 10Gb/s link to the other side of the datacenter, it might be as little as 10 seconds. If you have a 10TB database that you send over a nominally 1GB/s link but actually there's a lot of congestion from other users, to a datacenter on the other side of the world, that might take a hundred hours or so.
rsync can help a lot here, or the ZFS differential snapshot send.
so say the disk fails on your main DB. or for some reason a customer needs data from 6 months ago, which is no longer in your local snapshots. In order to restore the data, you have to transfer the data for the full database back over.
With multiple databases, you only have to transfer a single database, not all of your data.
Do you even have to stop Postgres if using ZFS snapshots? ZFS snapshots are atomic, so I’d expect that to be fine. If it wasn’t fine, that would also mean Postgres couldn’t handle power failure or other sudden failures.
* use pg_dump. Perfect consistency at the cost of a longer transaction. Gain portability for major version upgrades.
* Don't shut down PG: here's what the manual says:
However, a backup created in this way saves the database files in a state as if the database server was not properly shut down; therefore, when you start the database server on the backed-up data, it will think the previous server instance crashed and will replay the WAL log. This is not a problem; just be aware of it (and be sure to include the WAL files in your backup). You can perform a CHECKPOINT before taking the snapshot to reduce recovery time.
* Midway: use SELECT pg_start_backup('label', false, false); and SELECT * FROM pg_stop_backup(false, true); to generate WAL files while you are running the backup, and add those to your backup.
Presumably it doesn't matter if you break your DB up into smaller DBs, you still have the same amount of data to back up no matter what. However, now you also have the problem of snapshot consistency to worry about.
If you need to backup/restore just one set of tables, you can do that with a single DB server without taking the rest offline.
> you still have the same amount of data to back up no matter what
But you can restore/back up the databases in parallel.
> If you need to backup/restore just one set of tables, you can do that with a single DB server without taking the rest offline.
I'm not aware of a good way to restore just a few tables from a full db backup. At least that doesn't require copying over all the data (because the backup is stored over the network, not on a local disk). And that may be desirable to recover from say a bug corrupting or deleting a customer's data.
Try out pg_probackup. It works on database files directly. Restore is as fast as you can write on your ssd.
I've setup a pgsql server with timescaledb recently. Continuing backup based on WAL takes seconds each hour and a complete restore takes 15 minutes for almost 300 GB of data because the 1 GBit connection to the backup server is the bottleneck.
On mariadb you can tell the replica to enter into a snapshotable state[1] and take a simple lvm snapshot, tell the the database it's over, backup your snapshot somewhere else and finally delete the snapshot.
That's fair - I added "are working on a specific problem which needs a more complicated setup" to my original comment as a nicer way of referring to edge cases like search engines. I still believe that 99% of applications would function perfectly fine with a single primary DB.
Depends what you mean by a database I guess. I take it to mean an RDBMS.
RDBMSs provide guarantees that web searching doesn't need. You can afford to lose a pieces of data, provide not-quite-perfect results for web stuff. It's just wrong for an RDBMS.
What if you are using the database as a system of record to index into a real search engine like Elasticsearch? For a product where you have tons of data to search from (ie text from web pages)
In regards to Elasticsearch, you basically opt-in to which behavior you want/need. You end up in the same place: potentially losing some data points or introducing some "fuzziness" to the results in exchange for speed. When you ask Elasticsearch to behave in a guaranteed atomic manner across all records, performing locks on data, you end up with similar constraints as in a RDBMS.
Elasticsearch is for search.
If you're asking about "what if you use an RDBMS as a pointer to Elasticsearch" then I guess I would ask: why would you do this? Elasticsearch can be used as a system of record. You could use an RDBMS over top of Elasticsearch without configuring Elasticsearch as a system of record, but then you would be lying when you refer to your RDBMS as a "system of record." It's not a "system of record" for your actual data, just a record of where pointers to actual data were at one point in time.
I feel like I must be missing what you're suggesting here.
Having just an Elasticsearch index without also having the data in a primary store like a RDMS is an anti-pattern and not recommended by almost all experts. Whether you want to call it a “system of record”, i wont argue semantics. But the point is, its recommended hacing your data in a primary store where you can index into elasticsearch.
This is not typically going to be stored in an ACID-compliant RDBMS, which is where the most common scaling problem occurs. Search engines, document stores, adtech, eventing, etc. are likely going to have a different storage mechanism where consistency isn't as important.
The article is great, but the title here has been editorialized a bit. I'm not super familiar with HN, so what's the best way to get that fixed to match the actual article?
The original plan was to release in February, but in the Beta 2 release [1], they said this: "Because we are taking the time to issue a second beta, we now expect that the Go 1.18 release candidate will be issued in February, with the final Go 1.18 release in March."
It looks like there are still a few release blockers [2]. I'd imagine RC is fairly soon though.
EDIT: as mentioned by _fz_ below, RC1 has already released. Seems like the full release will most likely still be in March.
Well, not quite 2 weeks ago; it'll be 2 weeks on Thursday. I say this because policy is to issue a release "no sooner than two weeks after" issuing the release candidate.
It's also available as a mirror at https://github.com/golang/pkgsite. All the golang.org/x/* packages are thankfully available there, making them pretty easy to find.
The short version is "it depends". Essentially, each separate Wayland compositor needs to add support for the nvidia driver (or the driver needs to support more standardized APIs).
Gnome and KDE support Wayland on nvidia hardware (though it's a bit rough around the edges). Wlroots (and therefore Sway) doesn't because they don't want to have a separate code-path for one driver which doesn't want to support standards and because the APIs the driver does support wouldn't work well with the code model of wlroots.
It's probably not relevant in the long term anyway. I expect desktop Nvidia usage on Linux to pretty much plummet while their driver remains a blob. The trend is negative for them already.
So I fully support wlroots developer's approach to it - show Nvidia to the door until they come back with upstreamed driver.
I know it's all speculation, and this does come across as more monetization creep, but I can't help to wonder if there's an underlying reason beyond "line go up".