Return the status code that matches what actually happened: 200 for a page that exists, 301 or 302 for a move, 404 or 410 for a page that is gone, 429 to slow a crawler down, and 503 for a temporary outage. Google documents that these codes feed into crawling and indexing decisions, and it names 404/410 and 503/429 as the right answers for removal and overload respectively [2]. The most common mistake is a page that says "not found" but returns 200.

This guide covers what each code means, how to fix soft 404s, how to rate limit with 429 and Retry-After, and how to run a maintenance window with 503. Where a statement comes from Google's documentation, it is written as Google's and applies to Googlebot. Other crawlers, including AI crawlers, may handle codes differently.

The codes at a glance

CodeMeaningUse it when
200SuccessThe page exists and the content is the real content
301Moved permanentlyThe URL changed for good and a replacement exists
302Found (temporary redirect)The move is temporary and the original URL will return
404Not foundThe page does not exist and there is no replacement
410GoneThe page was removed on purpose and is not coming back
429Too many requestsOne client is sending requests faster than you allow
503Service unavailableThe server cannot serve requests right now, temporarily

200: success, not a guarantee

A 200 tells the crawler the request worked. It does not promise indexing. Google says a 200 response does not guarantee a page will be indexed: Google considers the content for processing, and if the content looks like an error, Search Console may flag a soft 404 [1].

For a crawler that is only fetching text to read or cite, the same logic is sensible: a 200 should carry the content a visitor would see.

301 and 302: the URL moved

  • 301 says the move is permanent. Use it when you renamed a URL, merged two pages, or changed domains, and the new page serves the same need.
  • 302 says the move is temporary. Use it when the original URL will serve content again, such as a short-term campaign redirect.

Pick by intent, not by habit. If the old URL is never coming back, a 302 describes the situation wrongly. If it is coming back, a 301 does.

Two practical points:

  • Redirect to a page with similar content. Google's guidance for removed pages with no similar replacement is 404 or 410, not a redirect [2].
  • HTTP semantics allow Retry-After on any 3xx response, where it indicates the minimum time the client should wait before following the redirect [4]. Most sites never need this.

For which URL belongs in sitemaps and canonical tags after a move, see the XML sitemap guide.

404 and 410: the page is gone

Google recommends returning 404 or 410 when a page was removed and there is no replacement page with similar content [2]. Both are valid.

410 vs 404

The HTTP meanings differ slightly:

  • 404 says the server cannot find the resource. It does not say whether that is permanent.
  • 410 says the resource is gone and the removal is intentional and likely permanent.

Google's guidance, as cited above, names either one as acceptable for removed content [2]. Choose by what you know:

  • You deleted the page on purpose and it will not return: 410.
  • The URL never existed, is mistyped, or you are not sure: 404.
  • A close replacement exists: redirect with 301 instead.

Do not build a strategy on the assumption that one code is processed faster than the other. If you need a page out of Google quickly, use Search Console's removal tools alongside the correct status code.

Soft 404s: the 200 that says "not found"

Google defines a soft 404 as a page that returns 200 while telling users the page does not exist, and says such pages are excluded from Search [2]. Search Console may flag them [1].

Typical causes:

  • A custom error template that returns 200 instead of 404.
  • A search results page with "0 results" and a 200 status.
  • A product page for a discontinued item that shows "unavailable" and nothing else.
  • A single-page app that renders its "not found" view client-side after the server already sent 200.

Before:

HTTP/1.1 200 OK
Content-Type: text/html

<h1>Sorry, we couldn't find that page</h1>

After:

HTTP/1.1 404 Not Found
Content-Type: text/html

<h1>Sorry, we couldn't find that page</h1>

The body can stay the same. The status line is what changes.

How to check safely

  • Test on staging, or on a single path you control. Request one URL you know does not exist, for example /status-test-does-not-exist, and read the status line.
  • Do not run bulk or looping requests against production to hunt for soft 404s. Use the Search Console report that flags them [1], and check the flagged URLs one at a time.
curl -I https://staging.example.com/status-test-does-not-exist

The first line of output should read 404, not 200. If your framework renders the not-found view in the browser only, the server-side status needs to be set too.

429: slow down

429 is the dedicated rate-limiting response. MDN describes it as the client having sent too many requests in a given amount of time, with Retry-After optionally indicating how long to wait [3].

Google's 2023 blog post says 429 is the exception among 4xx codes for rate limiting, and that 500, 503 or 429 are appropriate responses when Googlebot is crawling too fast [5]. The post's title makes the corollary explicit: don't use 403s or 404s for rate limiting [5]. A 404 tells a crawler the page is gone, which is not what you mean.

Example response:

HTTP/1.1 429 Too Many Requests
Retry-After: 120

Retry-After takes either a number of seconds or an HTTP date. The documentation cited here describes Googlebot; how any particular AI crawler reacts to 429 or Retry-After is something to confirm in that vendor's own documentation or in your logs. If you want to see which crawlers are hitting you, verifying crawler IPs in your server logs comes first, so you do not throttle a spoofed user agent and miss the real one.

Rate limit by client identity (verified IP range or user agent), not site-wide. A global 429 also hits visitors.

503: temporary unavailability and maintenance

MDN describes 503 as the response for temporary conditions such as maintenance or overload, and recommends a user-friendly page when possible [3]. The HTTP Semantics standard says a server may send Retry-After with a 503 to suggest when the client should retry [4]. Google says to return 503 or 429 temporarily for Googlebot requests when the server is overloaded [2].

For a maintenance window, return 503 on every URL being affected, not a 200 with a "back soon" page. A 200 maintenance page is a soft 404 problem in a different outfit: the status line says the content is fine.

Nginx example:

add_header Retry-After 3600 always;
error_page 503 /maintenance.html;

location = /maintenance.html {
    internal;
    root /var/www/static;
}

location / {
    return 503;
}

Set Retry-After to your honest estimate of when service returns, in seconds. Then:

  • Check the response with curl -I on staging before the window starts.
  • Remove the rule when you are done, and confirm key URLs return 200 again.
  • Do not use 503 for pages that are permanently gone. That is what 404 and 410 are for.

If a 503 is an overload response rather than planned maintenance, the same code applies, and Retry-After is still optional [4].

What to do next

  1. Request a known-nonexistent URL on staging and confirm it returns 404, not 200.
  2. Review Search Console's soft 404 flags and fix each at the server, not just in the template [1].
  3. Decide per removed page: 301 to a close replacement, 410 if intentionally gone, 404 otherwise.
  4. Replace any 403 or 404 used for throttling with 429 plus Retry-After.
  5. Prepare a 503 rule for maintenance and test it on staging.

For the wider list of checks around status codes, sitemaps and canonicals, use the technical SEO checklist.