Why Google Isn't Indexing Your Pages (And How to Diagnose It Properly)

You publish a page. You submit the sitemap. A week later the page still is not in Google, not even when you search its exact title in quotation marks. Search Console says something ambiguous, and whoever handles your SEO says it takes time.

Sometimes it genuinely does take time. Often something specific is wrong and it is diagnosable. The difficulty is that most people, including a lot of agencies, try to diagnose indexing with reports that cannot answer the question. Here is how indexing actually works, which signals are trustworthy, and the order we work through a diagnosis.

Crawled and indexed are not the same thing

These get used interchangeably and they are two separate events.

• Crawled means Googlebot requested the URL and got a response. A crawl is a fetch, nothing more.
• Indexed means Google processed that response, decided the page was worth storing, and added it to the index it serves results from.

A page can be crawled repeatedly and never indexed. Search Console has a status for exactly that: Crawled – currently not indexed. There is a separate status, Discovered – currently not indexed, which means Google knows the URL exists but has not fetched it yet. Those two statuses look almost identical and they point at completely different problems. The first is usually a judgement about the page. The second is usually a queue.

Before anything else, confirm the page is actually missing. Search for the exact page title in quotation marks, or use a site: query for the URL. A page that is indexed but ranking on page five looks the same as a page that is not indexed at all, and the fix for those two situations shares nothing in common.

Why the reports most people check cannot answer the question

This is the part that trips up experienced marketers.

The Sitemaps report is not an index report. It tells you Google read your sitemap file and how many URLs were in it. If you pull that same data through the Search Console API, the field that looks like it should say how many URLs got indexed is a deprecated leftover that returns zero regardless of reality. That number still gets shown to clients as if it meant something. It does not. Never accept "the sitemap says X pages indexed" as evidence.

The Page indexing report in the Search Console interface is useful for spotting patterns, but it is aggregated, it lags, and its totals are not exposed through the API. Any dashboard claiming to show live per-URL index status has built it from something else or is guessing.

Site crawlers are not index checkers either. A crawler tells you what your server returns. It cannot tell you what Google decided.

The one check that is actually reliable

The URL Inspection API is the only way to get a trustworthy, per-URL answer from Google. It returns, for a single URL: whether it is on Google, the coverage state in plain language, the date of the last crawl, the canonical URL Google selected versus the one you declared, whether robots.txt allowed the fetch, and whether an indexing directive blocked it.

The catch is the quota. Google allows one URL per call and caps inspections at roughly 2,000 per property per day. You cannot run it across a large site every morning, so you sample deliberately, covering your money pages, your newest publishes, and a slice of each page template. When we run an SEO audit, that sampling plan is designed before a single call is made, because the sample is what makes the conclusion valid.

Check the last crawl date before you blame the content

This is the step that gets skipped most often, and skipping it causes real damage.

When a page is not indexed, the reflex is to rewrite it. Add words, add sections, add FAQs. But if the URL Inspection API tells you Google has never crawled that URL, or last crawled it months ago, then the content was never evaluated. Rewriting it changes nothing, because nothing was rejected. You have a discovery or crawl-scheduling problem.

The diagnostic that separates the two is simple. Pull the last-crawl date across a sample of indexed and non-indexed pages and compare them. If recently-crawled pages index fine and the stragglers are all old or never-crawled, you are looking at a crawl backlog, and the fix is about discovery, internal linking, server speed, and reducing the number of low-value URLs competing for Googlebot's attention. If pages are being crawled promptly and still not indexed, it is a judgement about the page, and now the content conversation is the right one.

Diagnosing in the wrong order means rewriting pages that were never broken. That is wasted budget, and it is one of the ways agencies bill for months of activity that could not have worked.

The causes, in the order we rule them out

You are telling Google not to index the page

Boring and extremely common. A noindex meta tag or X-Robots-Tag header left over from staging, applied by a plugin default, or attached to a whole template. Check it first because it is the fastest to check and the most embarrassing to miss.

Canonical conflicts

A canonical tag tells Google which URL is the real version of a page. When it is wrong, you are voluntarily removing pages from the index. Typical failures: every page canonicalising to the homepage, paginated pages canonicalising to page one, or filtered URLs pointing back at a parent. The status Alternate page with proper canonical tag means Google agreed with your instruction and dropped the page. Watch for case sensitivity too. Many systems serve /Services and /services as the same page, which creates two indexable URLs competing with each other. That belongs in a redirect at the server level, not patched inside the page.

robots.txt is blocking the crawl

A disallow rule prevents fetching. It does not, on its own, guarantee removal from the index, which is why blocking a page in robots.txt is the wrong way to deindex it. If Google cannot fetch the page, it cannot read the noindex tag you put on it.

Soft 404s

The page returns HTTP 200 but looks empty or useless to Google, so it gets treated as a not-found. Common on empty category and search-result pages, and where content loads client-side and the initial HTML response is effectively a shell.

Thin or duplicate content

Google indexes what it considers worth storing. Fifteen service pages that differ only by city name, or category pages generated from one template with no distinct substance, are exactly what gets crawled and then dropped. Every location page needs a real reason to exist. That is the craft of local SEO, and it is not solved by find-and-replace on a town name.

Orphan pages

A page that no other page on your site links to is an orphan. It might sit in the sitemap and still be treated as unimportant, because internal links are how you tell Google what matters. Client-side pagination and JavaScript menus that never emit real anchor link elements create orphans at scale. If a listing paginates without producing crawlable links, Google may only ever see page one of it.

A sitemap is a hint, not a command

Submitting a sitemap does not compel Google to index anything. It is a suggestion about which URLs exist and when they changed. Google is free to ignore it, and routinely does. The same is true of the manual "Request indexing" button, which is fine for one page and meaningless as a strategy.

What earns indexing is more mundane: the page is reachable, it responds quickly and correctly, it is linked from pages Google already values, it is not contradicting itself with directives, and it contains something that is not already in the index. Those are the levers. Everything else is a hint.

How we run the diagnosis

Our process here is deliberately unglamorous, and we walk clients through it step by step:

1. Confirm the page is truly absent from the index, not just ranking badly.
2. Inspect a designed sample of URLs through the URL Inspection API. Record coverage state, last-crawl date, Google's selected canonical, and any blocking directive.
3. Split the non-indexed set into never-crawled and crawled-but-rejected. These are different investigations and should not be merged into one recommendation.
4. For the never-crawled group, chase discovery: internal links, sitemap accuracy, redirect chains, response times, and junk URLs consuming crawl attention.
5. For the crawled-but-rejected group, chase directives and duplication first, and only then the content.
6. Fix, re-inspect the same URLs, and compare against the recorded baseline. A fix that is not verified against Google's own answer is a claim, not a result.

That last step is the one we hold ourselves to. Anyone can list possible causes. The work is knowing which ones applied to your site, in what order, and proving the fix landed.

If this sounds like your site

If you have pages that will not index and you have been told to be patient, the useful next move is not another rewrite. It is a factual read of what Google is doing with those specific URLs.

That is the starting point for our technical SEO work, and we do it before touching search engine optimization strategy, because there is no point ranking a page that is not in the index. Tell us which pages are missing and we will tell you what we find, including if the answer is that nothing is broken and you simply need to wait.