this paper we discuss the challenges in building good search engines, and describe some of the techniques that are useful. Many of the search engines use well-known information retrieval (IR) algorithms and techniques [Salton 1989; Faloutsos 1985]. However, IR algorithms were developed for relatively small and coherent collections such as newspaper articles or book catalogs in a (physical) library. The Web, on the other hand, is massive, much less coherent, changes more rapidly, and is spread...