Glossary · Technical
noindex
Definition. noindex is a directive, in a meta robots tag or an X-Robots-Tag HTTP header, telling search engines not to include a page in their index. It is the correct way to keep a page out of results, and it only works if the page can be crawled.
The common contradiction: blocking a URL in robots.txt and also setting noindex on it. The crawler never fetches the page, never sees the directive, and the URL can remain indexed.
A stray noindex left over from a staging environment is one of the most common causes of sudden, total traffic loss on a new site.
Where this shows up on this site
Related terms
robots.txt
robots.txt is a plain-text file at the root of a domain telling crawlers which paths they may request…
Indexation
Indexation is whether a search engine has stored your page and made it eligible to appear in results. A…
Canonical URL
A canonical URL is the version of a page you declare as the original when several URLs serve similar…
More in Technical: Internal linking · Anchor text · Orphan page · Crawl budget · Indexation · Canonical URL · Duplicate content · Keyword cannibalization
Frequently asked questions
What does noindex mean?
noindex is a directive, in a meta robots tag or an X-Robots-Tag HTTP header, telling search engines not to include a page in their index. It is the correct way to keep a page out of results, and it only works if the page can be crawled.
Why does noindex matter for AI SEO?
The common contradiction: blocking a URL in robots.txt and also setting noindex on it. The crawler never fetches the page, never sees the directive, and the URL can remain indexed.
What is the most common mistake with noindex?
A stray noindex left over from a staging environment is one of the most common causes of sudden, total traffic loss on a new site.
Last verified 2 August 2026. Prices, plans and features on this page were read on each vendor's own site on that date. Vendors change pricing often, so confirm before you buy. Spotted an error? Tell us and we will fix it.
Put it into practice
Thirty AI SEO tools compared on verified pricing, capabilities and AI search support.