Have you ever wondered how come Google is as fast as it is? When you enter a search query in Google Search, you see the time taken to return your results, and that time isn't even measured in seconds, but rather, milliseconds. How can it be so fast? Searching for a file on your computer or a local area network can take minutes. Surely, the internet can not be faster than your own computer's storage? That's a really good question which I am sure some of us have pondered over at one time or another. So what actually is going on?
How Search Engines work?
To understand the answer to the question, you must first get a very basic idea of how search engines actually work. At its core, a search engine consists of a crawler and an indexer (let's leave the headache ranking pages for now, shall we?)
The crawler's job is to crawl the web, and I mean literally! It crawls from one page to another through hyperlinks. That is why if you build a new website, building links to it from other sites is a quicker way of getting it noticed than submitting it to Google's index. The crawler reaches into every accessible corner of a website (if it is linked from another page).
Once the crawler has found a page, it is the indexer's job to index the page's content. Indexing means processing and parsing the data on the page, and then saving it in a statistical, more meaningful form, something which the search engine can understand easily. This data is saved on the search provider's web server.
So once a crawler finds a page, the indexer processes it and stores it on a server. The search engine now has nothing more to do with the page in question, unless it wants to re-crawl and re-index that page (in case of updates. Google indexes webpages every second or every day, depending upon the ranking of the website). Now, when a user requests a query, the search engine doesn't actually search the web. It has already done that! It simply searches from the index found on its own servers, and returns relevant results.
What else?
So how is it still faster than your local hard drive? Well, Google isn't the best search engine for no reason! It employs some really efficient algorithms and data structures that help in the quick retrieval of data. Google also has huge 'farm' servers that have index data stored in them. And the computational powers of these machines is immense, nothing like your ordinary desktop microprocessor. So when all these factors add up, you really get a solution that's faster than your Personal Computer's hard drive.
There is a way to speed up search for your hard drive too. Applying the same principle of indexing, we can find files and content much quicker. Windows XP, 7 and 8 have built-in drive managers that give you the option to defragment or index your hard drives. This will optimize your hard drives, and make search much quicker. Windows 7 and onwards has tried to implement this by default too, which is why you can write anything in the search menu search bar, and see results displaying instantaneously. This has been achieved via the Windows Search Index. Hence, indexing is the key to speeding up search results.
Did you enjoy this post? Please share your opinions in the comments below. And if in doubt, feel free to ask us questions. Cheers :)
