How to Find Duplicate Content on Your Web Site
For the best search engine optimization (SEO) results, you should find and remove duplicate content on your Web site. A good place to start looking is to find out how many of your Web pages are currently indexed, versus how many the search engines consider to be duplicates. Here’s how:
At Google, type site:domain.com in the search box (using your domain), and then click Search.
When the results page comes up, scroll to the bottom and click the highest page number that shows (usually 10).
Doing this can cause the total number of pages to recalculate at the top of the page.
Notice the total number of pages shown in “Results 1 – 10 of about ###” at the top of the page.
The “of about ###” number represents the approximate total number of indexed pages in the site.
Navigate to the very last page of the results.
The count shown there represents the filtered results. The difference between these two numbers most likely represents the number of duplicates.
For performance reasons, Google doesn’t display all of the indexed pages and omits the ones that seem most like duplicates. If you truly want to see all of the indexed listings for a site, you can navigate to the very last results page of your site: query and click the option to Repeat the Search with the Omitted Results Included. (Even then, Google only shows up to a maximum of 1,000 listings.)
To discover the number of indexed pages in Yahoo! and Microsoft Live Search, you can try the free Search Engine Saturation tool available from Acxiom Digital.