Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Good observation, but I just want to point out that the full dumps are downloaded from different servers. They are even mirrored by other institutions. See https://meta.wikimedia.org/wiki/Mirroring_Wikimedia_project_...


They could set up a torrent as well and almost assuredly people will seed it full time on servers.


There is a torrent distribution channel: see https://meta.wikimedia.org/wiki/Data_dump_torrents

However it suffers the number one torrent issue: they do not tolerate change. This means that

- When an article changes, you need to generate a new torrent - When a new torrent describes the archive, it needs to be downloaded from scratch by all peers, so that the maximum number of peers are available for a newcomer.

I hope you'll understand that this is not the official way to distribute archives...


Yeah, but they're working on incremental updates, so this should make the "from scratch" part much easier. You can look at https://www.mediawiki.org/wiki/Incremental_dumps and http://lists.wikimedia.org/pipermail/xmldatadumps-l/2013-Aug... if you're interested.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: