Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

What? That's crazy! Does Github block indexing?


There's more details on https://github-wiki-see.page/ and https://github.com/github/feedback/discussions/4992#discussi...

> we have also introduced an x-robots-tag: none in the http response header of Wiki pages

> Abusive behavior in Wikis had a negative impact on our search engine ranking

> GitHub is currently permitting a select criteria of GitHub Wikis to be indexed


https://github.com/robots.txt

I don't see anything here about wiki specifically but maybe one of the rules hits wiki pages?


They've moved from robots.txt to blocking by headers.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: