Topics Technology and the internet

How do search engines rank pages?

A search engine ranks pages in three steps. It crawls the web by following links, stores what it finds in a giant index, and then, when you type a query, pulls out matching pages and sorts them using many signals. Those signals try to estimate two things: how well a page matches what you asked, and how trustworthy or useful it is likely to be.

What makes this interesting is how much is guessed. Early on, Google's PageRank idea treated a link from one page to another as a kind of vote, and links still matter. Today the signals also include the words on the page, how fresh it is, where you are, and how it works on a phone. The exact recipe is secret, changes often, and is not fully public, so anyone who claims to know the precise formula is overselling.

An episode would walk through crawling, indexing and ranking in plain language, and look at why spam and gaming the system are a constant fight. The hosts on bre are AI, so they can be wrong, and you can press Talk to ask them a question midway, like why your own site is not showing up.

What a bre episode would cover

An outline of the episode bre would make for this question. Every episode is written fresh when you ask, so yours will differ.

  1. Crawling: how the web gets foundPrograms called crawlers follow links from page to page, collecting content. Pages nobody links to, or that block crawlers, can stay invisible.
  2. The index: a library of the webCollected pages are broken into words and other details and filed so that matches can be found in a fraction of a second.
  3. Relevance: does the page match your question?Engines look at the words on a page, their placement, and what the query probably means, including synonyms and intent.
  4. Links as votesPageRank's core idea was that pages linked to by many respected pages are likely to be good. Links remain a signal, though far from the only one.
  5. Quality, freshness and contextSpeed, mobile-friendliness, recency, language and location can all shift results. What counts as quality is partly a judgment, and engines say so.
  6. Spam, SEO and the secret recipeSearch optimization is the craft of making pages easier to find. Engines keep their formulas private and update them to fight manipulation, so exact weights are unknown.
  7. Where AI changes searchMachine learning helps engines understand language and some now write summaries. We cover what is established and what is still changing.

How the episode might open

A sample exchange between two of bre’s AI hosts, bre and Cal. Both are AI; this is written by AI, as every bre episode is.

  1. breAI host

    Picture typing three words into a box and getting back ten blue links in under a second. Out of the whole web. Cal, how does that not fall over?

  2. CalAI host

    It cheats, in the best way. The search engine doesn't search the web when you ask. It searched it ahead of time, and you're searching its notes.

  3. breAI host

    Okay, here's the part nobody tells you. Those notes are an index, like the one at the back of a textbook, only for billions of pages.

  4. CalAI host

    Right, and the ranking is the harder bit. Lots of pages contain your words. Which one goes first? That's where money and careers get made, because first place gets the clicks.

  5. breAI host

    So what's the first clue it uses? I'd guess the words on the page.

  6. CalAI host

    Words are a big one, yes. But the famous breakthrough was links. If lots of pages point at you, that's a vote. Hold on, how does that actually work, when anyone can make links?

  7. breAI host

    That's the right question, and honestly the answer is that not every vote counts equally. A vote from a trusted page weighs more.

  8. CalAI host

    And that's why people try to fake it. Which is the whole cat-and-mouse story we get into.

Questions people also ask

Does Google rank pages by how many links they have?
Links are one signal, and the original PageRank idea was built on them, but they are not the only one. Today the words on a page, its usefulness, speed and many other factors also play a part. Google does not publish the exact mix.
What is SEO?
SEO, short for search engine optimization, is the practice of making a website easier for search engines to find and understand, and more useful to visitors. It can mean clear writing, good page structure and earning links. Manipulative tricks may be penalized by search engines.
Why do two people get different search results?
Results can differ because of location, language, device, search history and the engine's testing of changes. A search for a nearby cafe obviously depends on where you are. How much personalization applies to other searches varies and is not fully public.
Can anyone pay to rank higher in search results?
Paid ads can appear at the top of results, and they are normally labeled as ads. The regular, unpaid results are ranked by the engine's own systems, and the major engines say that payment does not buy a better organic position.

Related topics

More: all 300 topics, technology and the internet, or the longer reads on /learn.

bre’s hosts are AI, and every episode is generated, so they can be wrong: check anything that matters. This page outlines what an episode would cover. It is for interest and learning, not medical, financial or legal advice.