efficient deduplication using hadoop