Very large numbers raise a practical question for anyone who organizes information: which parts of the work can people still do by hand?

The shift from millions to a trillion

Google's blog reported that its systems that process links on the web to find new content hit a milestone: 1 trillion (as in 1,000,000,000,000) unique URLs on the web.

This milestone highlights a rapid trajectory in data accumulation. The first Google index in 1998 already had 26 million pages, and by 2000 the Google index reached the one billion mark. These figures count different things, pages in an index and addresses found, so they are not one trend line.

Designing for continuous velocity

The challenge of scale is not just about static storage, but about managing relentless velocity. The blog also said that the number of individual web pages out there is growing by several billion pages per day.

In general terms, when a collection keeps growing quickly, the tools that organize it may need to change. Automated methods can take on more of the work, while people decide what those methods should do.

One reading is that curated lists and automated search can each have a role, and the balance between them may shift as collections grow.

Growth can change what it means to find something. Storing more is one part of the task; deciding what matters among what is stored is another. Designers of such systems may ask how new material changes the meaning of old material, and how to keep results useful as the collection changes. These are design questions rather than settled answers, and different systems answer them in different ways.

Perhaps the more useful question is not the size of the number, but what a system can still do well when numbers grow that large.

Sources