How do search engines work, and what is the difference between crawling, indexing and ranking?
Three distinct stages, and a page must pass each before it can appear in results.
- Crawling — Googlebot discovers URLs by following links, reading sitemaps, and revisiting known pages, then fetches them. Blockers:
robots.txtdisallow rules, no internal links pointing to the page, server errors, or a slow site consuming the crawl budget. - Indexing — Google renders the page, understands its content, and decides whether to store it. Crawled does not mean indexed. Blockers: a
noindextag, a canonical pointing elsewhere, duplicate or thin content, or content that only appears after JavaScript execution which fails. - Ranking — for a given query, Google orders indexed pages using hundreds of signals: relevance to the query, content quality, links, user location and history, page experience, and freshness.
Why the distinction matters practically: the diagnosis and fix are completely different at each stage. Search Console's Page Indexing report tells you which stage failed — "Discovered, currently not indexed" is a very different problem from "Crawled, currently not indexed", and both differ from a page indexed but ranking poorly.
Note: Add that Google renders JavaScript, but rendering is queued and resource-intensive. Content that requires JavaScript may be indexed late or incompletely, which is why server-side rendering or static generation is safer for content that must rank.





