SeoblogR

(No Ratings Yet)

 Loading ...

If you're new here, you may want to subscribe to my RSS feed or follow me on Twitter. Thanks for visiting!

Google said they know how big is the web. There is about 1 trillion (as in 1,000,000,000,000) unique URLs on the Internet.

Back in 1998 Google’s index had 26 million pages, in 2000 it reached 1 billion mark and in 2008 its 1 trillion…wait, no, it’s not how many websites they store, its just how many unique URL’s they’ve found.

In fact, we found even more than 1 trillion individual links, but not all of them lead to unique web pages. Many pages have multiple URLs with exactly the same content or URLs that are auto-generated copies of each other. Even after removing those exact duplicates, we saw a trillion unique URLs, and the number of individual web pages out there is growing by several billion pages per day.

Google index is estimated to hold about 40 billion pages and growing every day.

They do not store all pages they could. The reason: its too expensive and many websites are spam and Google does not want to hold spam websites in their indexes.

We don’t index every one of those trillion pages — many of them are similar to each other, or represent auto-generated content..that isn’t very useful to searchers. But we’re proud to have the most comprehensive index of any search engine, and our goal always has been to index all the world’s data.

Well, the most comprehensive index of any search engine? Not so sure. Cuil is indexing more than 120 billion pages.