Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Sadly very few authors seem to acknowledge or even know that github wiki pages are not indexed by search engines so if it wasn't for third-party sites like github-wiki-see.page (which could stop working at any time) their contents would be undiscoverable by the very same people they are usually intended...


What? That's crazy! Does Github block indexing?


There's more details on https://github-wiki-see.page/ and https://github.com/github/feedback/discussions/4992#discussi...

> we have also introduced an x-robots-tag: none in the http response header of Wiki pages

> Abusive behavior in Wikis had a negative impact on our search engine ranking

> GitHub is currently permitting a select criteria of GitHub Wikis to be indexed


https://github.com/robots.txt

I don't see anything here about wiki specifically but maybe one of the rules hits wiki pages?


They've moved from robots.txt to blocking by headers.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: