The Technical SEO Checklist We Run Before Writing a Word of Content

Most SEO engagements begin with a content calendar. Ours begins with a handful of HTTP requests and a fairly boring afternoon spent reading response headers. The reason is simple. Content is the most expensive thing we produce for a client, and publishing it onto a site that cannot be crawled, consolidated or indexed cleanly is the most efficient way to spend a budget and get nothing back.

Below is the checklist we actually run before we write. We are giving it away on purpose. If you run it against your own site and find nothing, you have saved yourself the cost of an audit. If you find three of these problems, you now know exactly what to ask the next agency you speak to — and you will be able to tell within about five minutes whether they know what you are talking about.

Why this has to happen first

Technical faults do not sit still and wait for you. They compound with every page you add. A canonical bug discovered after you have published fifty articles is not one bug: it is fifty articles whose signals have been split across two or more addresses, plus a re-crawl cycle you now have to sit through before anything improves. The same fix applied before you publish takes an hour, and nobody ever notices it happened. That is the whole argument for doing technical SEO before content, rather than as a clean-up afterwards.

The checks are ordered roughly the way we run them: the address first, then the instructions you give crawlers, then indexability, then structure, then speed. Each one comes with the reason it matters, because a checklist you cannot reason about is just a ritual.

1. Settle the canonical host

Request the home page four ways: http://example.com, http://www.example.com, https://example.com, https://www.example.com. Exactly one of those should return a 200. The other three should return a single 301 pointing at the one that does.

What we find often enough to check every single time is that two of them return 200. The www and the non-www version both serve the site happily. Sometimes http answers as well as https. Nothing looks broken — the site loads, the forms work, the phone still rings — but every page on that site now exists at two or four separate addresses. Links, crawl budget and ranking signals get divided between them, and Google is left to guess which one you meant.

Check deep pages too, not just the root. It is common for the home page to redirect correctly while inner paths do not, because the redirect was configured on one route instead of at the server or middleware level. Confirm the redirect is a 301 rather than a 302, and that it resolves in a single hop.

2. Look at how the canonical tag is built, not just whether it exists

Nearly every site has a canonical tag. Far fewer have a correct one. The specific bug we look for is a canonical tag assembled from the request host rather than from a fixed, configured base URL.

When the canonical is built from whichever host happened to serve the request, the www version of a page canonicalises to itself and the non-www version also canonicalises to itself. Every variant declares itself the original. The tag is present, every auditing tool will report it as present, and it consolidates absolutely nothing. That is a worse position than having no canonical at all, because it passes inspection.

So view source on the live page, on each host variant, and read the actual value. The canonical should be absolute, on the chosen host, using https, and identical no matter which variant you requested.

3. Make the sitemap true

A sitemap is a claim about your site, and it is easy to make a false one. We check:

• It is fetchable. Request the sitemap URL directly and look at the status code, not at whether a browser renders something.
• The XML actually parses. A single stray space, newline or byte-order mark before the XML declaration invalidates the file. So does a missing closing urlset tag. Both are invisible in a browser and fatal to a parser.
• URLs are absolute and on the right host. A sitemap served from the https www host that lists http or non-www URLs is politely asking Google to go and index the variants you just redirected away from.
• Every listed URL returns 200 and is self-canonical. Submitting redirects, 404s or pages that canonicalise somewhere else spends crawl budget telling Google about pages you do not want.
• Nothing in the sitemap is blocked by robots.txt. This contradiction — please index this page and you may not fetch this page — is one of the most common things we find, and it is entirely self-inflicted.

4. Read robots.txt the way a crawler does

Two failure modes matter more than the rest.

Never block CSS and JavaScript. Google renders pages. If your stylesheets, scripts or static asset directories are disallowed, the crawler sees an unstyled skeleton, cannot confirm the layout works on mobile, and cannot see any content your JavaScript injects. Blocking assets was reasonable advice a very long time ago. It is now a way to hide your own site from the crawler.

Watch blanket query-string rules. A rule like Disallow: /*? is usually added to stop crawlers wandering through filter combinations. It also silently blocks every real page that happens to carry a parameter: paginated listings, campaign-tagged URLs, internal search pages that were earning traffic. On an online store, where filters and parameters are the navigation, this deserves a considered strategy rather than one broad rule — it is a large part of what eCommerce SEO involves.

Remember also what robots.txt does not do: it does not remove a page from the index. A disallowed URL that other pages link to can still appear in results as a bare URL. To remove a page you need a noindex tag on a page Google is allowed to fetch.

5. Indexability, and what got left behind

• Leftover noindex from staging. Check both the meta robots tag and the X-Robots-Tag response header. The header version is the one that gets missed, because it never appears in the page source.
• Redirect chains. A URL that redirects to a URL that redirects again. Collapse them so every redirect resolves in one hop, and update the internal links that point at the old address instead of relying on the redirect forever.
• Soft 404s. A page that says "not found" in the body while returning HTTP 200. Empty search results, discontinued services and deleted posts all tend to do this. The correct answer is a real 404 or 410 status. When the status and the content disagree, Google decides for itself, and it decides against you.

6. Duplicate and near-duplicate pages

This is the one clients dislike hearing, so we will say it plainly. Templated location or service pages that differ only by a swapped place name are a doorway-page pattern. Ten pages that are word-for-word identical except for "Sioux Falls", "Rapid City" and "Fargo" will mostly not index, and the ones that do will compete with each other.

The test is honest and it is not technical: can you say something genuinely different on this page? Real service-area detail, actual work you have done there, people, logistics, pricing structure, local constraints. If you can, build the page. If you cannot, build one strong page instead and stop paying to publish variants of it.

7. Internal links, orphans and pagination a crawler cannot use

Crawl the site starting from the home page and follow only real links. Anything that exists in the sitemap but is reachable from no page is an orphan, and it is telling Google that even you do not think it matters.

Then check pagination and listings specifically. If a listing loads more items with a button and never changes the URL, or if the links only exist after JavaScript runs, the crawler realistically sees page one and nothing else. Everything past that first screen is invisible, no matter how many items are in the database. The fix is unglamorous: server-rendered anchor tags pointing at real, distinct, crawlable URLs.

8. Speed, and the work you repeat on every request

We are not chasing a score here. We are looking for work the server repeats on every single page load: the same data rebuilt per request, a database query to assemble a navigation menu, a query per item in a list. That work is a real cost, paid once per visitor and once per crawl, and it shows up as slow time-to-first-byte, which is the part of speed a crawler feels most directly. Cache it, precompute it, or build it once.

The front-end basics still apply — render-blocking resources, images without dimensions, a hero image that is lazy-loaded when it is the very thing the user is waiting for — but a fast-looking page on a slow server is still a slow page.

9. Verify against the live URL

Everything above is verified in the same way: by requesting the live URL and reading what actually comes back. Status code, headers, raw source.

A database write is not a page. A green tick in a tool is not an indexed page. If it has not been checked on the live URL, it is not done.

For what Google itself holds, use the URL Inspection tool rather than a dashboard summary, and inspect the pages that matter to the business. It will tell you the canonical Google selected — which is frequently not the one you declared — and when the page was last crawled. That last-crawled date is worth reading before you conclude that a page is failing to index because of its content. If recently crawled pages index fine and older ones do not, you are looking at a crawl backlog, and rewriting healthy pages will not fix it.

What to do with this

Run it. Most of it needs nothing more than a browser, view-source, and the ability to look at response headers. If you find your site is clean, then content is genuinely your next constraint, and you can spend accordingly.

If it is not clean, you now have a specific, arguable list rather than a vague sense that something is wrong — and you can hand that list to whoever maintains the site. If you would rather someone else ran the whole thing and handed you the findings with the reasoning attached, that is precisely what our SEO audits are for, and it is the first thing we do on any search engine optimization engagement. We would rather tell you the foundation is fine than sell you content that is going to sit in a hole.