Finding Bieber: On removing duplicates from a set of documents | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		Finding Bieber: On removing duplicates from a set of documents (stevehanov.ca)
		26 points by spiffytech on Nov 10, 2014 \| hide \| past \| favorite \| 1 comment

chaosfactor on Nov 10, 2014 [–]

Convert to tfidf space and search for the documents with the highest correlations (using, i.e., a clustering algorithm).

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact