Data deduplication is a technique to improve the storage utilization. De-duplication technologies can be designed to work on primary storage as well as on secondary storage. De-duplication with the use of chunking Data that is passed through the de-duplication engine is chunked into smaller units and assigned identities using crystallographic hash functions. Thereafter, two chunks of data are compared to ascertain whether they have the same identity. Chunking for de-duplication can be frequency based or content based. Frequency based chunking identifies high frequencies of occurrences of data chunks. The algorithm uses this frequency information to enhance data duplication gain.