Why your staging site is showing up on Google

A staging URL feels private because you did not announce it. That feeling is not an access control or a search-engine directive. If a crawler can reach the page and nothing tells it not to index the response, the unfinished site can eventually appear beside the client's real domain.

The fix starts by separating discovery, indexing, and access. They are three different systems, and each needs the right control.

How a search engine discovers a staging site

You do not have to submit a URL for a crawler to find it. A staging link can escape through a public issue, a repository, an analytics dashboard, a shared document, a link from another crawlable page, or a person who forwards the review email. Search engines also revisit known hosts and follow links between pages.

A random subdomain lowers accidental discovery, but it does not tell a crawler what to do after discovery. An unlisted URL is useful friction, not privacy.

Why robots.txt is not enough

robots.txt controls crawling behavior. It does not guarantee that a known URL will disappear from search results. Google explains that a URL blocked by robots.txt can still be indexed without its content, and that the crawler must be allowed to see a noindex rule for that rule to work.

Use an HTML robots meta tag or the X-Robots-Tag HTTP response header to send noindex on the actual staging response. The HTTP header is especially useful for a hosting layer because it can cover HTML and non-HTML responses without modifying the client's source files.

Read Google's current guidance on blocking search indexing before changing a live site's crawl rules.

Noindex and passwords solve different problems

noindex is a request to supporting search engines. It does not stop a person with the URL from opening the page.

A server-side password or authenticated access gate decides who receives the site content. It also prevents a crawler without credentials from reading the protected page. A password embedded in the site's own JavaScript does not provide the same protection because the browser must receive the protected files before that script can run.

Use noindex for search visibility. Use access control for confidentiality. Use both when an unpublished client site should have both properties.

Remove a staging URL that is already indexed

First, decide whether the preview should remain online. If it is obsolete, return a real 404 or 410 response and remove internal links to it. If the client still needs it, keep the URL reachable to the crawler long enough to serve a noindex directive, or protect it with authentication.

Then work through this checklist:

• Confirm the staging response sends X-Robots-Tag: noindex, nofollow or an equivalent robots meta tag.
• Remove the staging URL from XML sitemaps, public navigation, repositories, and documentation.
• Check alternate hostnames, HTTP and HTTPS variants, and trailing-slash variants for the same content.
• Use the search engine's removal tool when the result is urgent, while keeping the durable noindex, authentication, 404, or 410 behavior in place.
• Recheck the live HTTP response in a private browser window so a local login does not hide the public behavior.

A temporary removal request is not the permanent fix. The server response remains the source of truth when the search engine returns.

Build a safer client-review workflow

The easiest indexing incident to handle is the one that never ships. Create a preview host policy that applies before anyone shares the first link.

• Send noindex headers on every normal preview response, password challenge, paused page, and error page.
• Keep client previews off the production sitemap.
• Add a server-side password when unpublished content should not be public to anyone with the URL.
• Use one stable review URL so old deployment links do not remain scattered across email threads.
• Remove or expire the preview when the engagement ends.

PreviewSend applies the noindex response header to hosted previews and supports a server-side project password. The client can still open the shared URL, pin account-free comments, and review new versions at the same address. When the preview is ready, the owner can disable review controls while leaving the hosted static site online.

Start a free client preview and test the public response before sending it to a client.

The rule to remember is simple. Unlisted is not unindexable, noindex is not authentication, and a safe staging workflow uses the control that matches each risk.