SEO glossaryDe-indexing is when a page or site is removed from a search engine's index, so it stops appearing in results. It can be deliberate (a noindex tag) or a penalty.
A search engine can only show pages that are in its index, the database of pages it has crawled and chosen to keep. De-indexing is a page, or a whole site, dropping out of that index. A de-indexed page does not rank lower; it does not appear at all, even when someone searches for its exact title.
Sometimes that is exactly what the site owner wants. More often, when a site owner notices it, something has gone wrong.
Deliberate de-indexing
There are good reasons to keep pages out of search: thank-you pages, internal search results, staging sites, thin tag archives, or outdated content. The reliable ways to do it are:
- A
noindexdirective, either as a robots meta tag in the page's head or as anX-Robots-TagHTTP header. Google drops the page the next time it crawls it. - Removing the page so it returns a 404 or 410 status, or redirecting it to a replacement.
- Password protection, which keeps crawlers out entirely.
Blocking a URL in robots.txt does not de-index it. It stops Google crawling the page, which also stops Google seeing a noindex tag, and the URL can stay indexed without a description. The Removals tool in Google Search Console hides a URL for about six months while a permanent fix takes effect; it is not a permanent removal on its own.
Accidental de-indexing
The most common causes we see on WordPress sites:
- The "Discourage search engines from indexing this site" box under Settings > Reading, ticked during development and never unticked after launch. It adds
noindexto every page. - SEO plugin settings that set a post type, category, or individual page to
noindex. - Canonical tags pointing pages at a different URL, so Google indexes that URL instead.
- Server errors and downtime. Pages that return 5xx errors for a sustained period, or block Googlebot at the firewall, are eventually dropped.
- Hacking. Compromised sites may be flagged for security issues, and injected spam can lead to a manual action.
De-indexing as a penalty
Google can remove pages or an entire site for serious violations of its spam policies, such as pure spam, cloaking, or scaled content abuse. This shows up as a manual action in Google Search Console. Lesser problems, like thin content, usually mean pages rank poorly or Google chooses not to index them ("Crawled, currently not indexed") rather than a penalty. See Google penalty for the difference.
How to check and recover
- Inspect the URL in Google Search Console. URL Inspection says whether the page is indexed and, if not, why:
noindex, a canonical elsewhere, a crawl error, or a robots.txt block. - Check the Page indexing report for patterns across many URLs, and the Manual actions and Security issues reports.
- Fix the cause: remove the stray
noindex, correct the canonical, resolve server errors, or clean up a hack. - Request indexing for important URLs in URL Inspection and resubmit the sitemap.
- For a manual action, fix every issue listed and file a reconsideration request.
A site:example.com search gives a rough idea of what is indexed, but its counts are estimates; Google Search Console is the authoritative source.