Digg improves duplicated detection system

Digg improves duplicated detection system

Digg might not be the hottest property on the web right now (Twitter takes that title) but they are still an important online destination.

The developers behind the social bookmarking service revealed that they have updated the system to better detect duplicate content submitted by their users.

Digg said in a blog post:

To better understand the nature of the problem, we analyzed the types of duplicate stories being submitted. Most common are the same stories from the same site, but with different URLs. Our R&D team came up with a solution that identifies these types of duplicates by using a document similarity algorithm. Look for a separate tech blog post on how this works, but it has proven to be a reliable way of identifying identical content from the same source.

One of the problems faced by Digg is that different users post the same story from a single or different source. So, the Diggs are divided amongst those stories. Sometimes this hurt the story from coming faster on the home page negating the overall value of the service.

Checkout: Digg: Dupe Detection Updates Are Here