Doorway pages and scaled content abuse: how programmatic SEO content kills indexation

The site was growing steadily, with more and more pages in the index—then traffic started to decline for no obvious reason. No manual actions in Search Console, no clear technical errors. This is a familiar situation for projects that at some point started generating pages programmatically at scale: tag pages, filter pages, automated reports, and landing pages for every city or product.

This is called scaled content abuse—an official category in Google’s spam policies, updated in March 2024. We’ll look at what it is, why it’s easy to fall into this category accidentally, even without using “black hat” methods, and how to get out of it, using the way we fixed the problem on our own site as an example.

What is scaled content abuse

Google defines scaled content abuse as generating a large number of pages whose primary purpose is to manipulate search rankings rather than help users. There’s an important nuance in this definition: it’s not about how the content was created (manually, with a template, or by an AI), but why it was created and what value it provides.

This means the policy applies not only to the classic “doorway pages” of the past decade (thin pages stuffed with keywords for a specific query), but also to quite modern patterns:

• Automatically generated template pages for every combination of parameters (city × service, brand × model);
• Programmatic “reports” or “cards” created for any value entered by a user, without moderation;
• AI-generated text inserted into a template for the sake of superficial uniqueness, without real editorial value;
• Filter and tag pages indexed indiscriminately, even though they contain overlapping content.

They all have one thing in common: the ratio of indexed pages to pages that actually bring in traffic and engagement tends toward zero. The algorithm sees this not as “a lot of useful content,” but as noise.

Why this is more serious than it seems

The main danger isn’t that poor pages fail to rank—that’s only half the problem. The problem is that these patterns:

1. Dilute the crawl budget. Search crawlers spend resources crawling thousands of low-value pages instead of recrawling genuinely important content more often.
2. Reduce trust in the entire domain. Quality assessment algorithms look at site-wide patterns, not just individual pages. If 95% of the pages in a particular section are templated junk, that can affect how the entire domain is perceived, including pages that are perfectly fine on their own.
3. Accumulate over time. Nobody will notice a dozen weak pages. Several thousand make up a statistically significant pattern that the algorithm can detect automatically, without a human reviewer.

A real example: how we fell into this trap

reChecker.ru has a feature: after any site check through our audit, the result is saved on a public page at /report/домен so it can be shared or linked to. Convenient, but at some point we discovered that this mechanism had become exactly the kind of scaled content abuse we’re talking about:

• 10 737 pages published in less than three months;
• 97,6% (10 478) were checked exactly once—someone entered a domain, viewed the result, and left, while the page remained in the index indefinitely;
• Among them was a helper script that checked “random” domains every 20 minutes by following outgoing links from previously checked sites. As a result, the index included pages such as “Audit of pinterest.com” and even “Audit of mvd.ru”—for domains that had nothing to do with us and for which nobody had requested a check.

All the signs were there: templated pages, automated generation without moderation, zero engagement for the vast majority, plus a reputational risk—public “ratings” of other domains without their knowledge.

How we found it

Not by gut feeling, but by looking at the metrics:

1. We calculated the ratio of the total number of indexed pages in the section to the number of pages showing signs of genuine interest (in our case, repeat checks of the same domain);
2. We pulled a random sample of domains and looked at it from an outsider’s perspective—the list spoke more clearly than any figures;
3. We checked what was generating these pages and found an automated cron script that should never have been publishing anything in an indexable form.

How we fixed it

The key principle: don’t delete pages; limit their indexation. Deleting pages that may already have direct links or bookmarks is an unnecessary risk 404. Instead:

1. We disabled the source of the problem—automatic page generation for domains without an explicit user request;
2. We added noindex, follow to pages with no signs of engagement—they remain accessible via direct links, but are not eligible for inclusion in the index;
3. We updated sitemap.xml so that it includes only pages that meet the usefulness threshold—the final list shrank from 10 737 to 260 URLs;
4. We checked that this logic wouldn’t be automatically “undone”—it was accidentally being re-added on every deployment, which is also worth checking for.

How to check your site

Before cleaning anything up, you need to understand the scale of the problem. Checklist:

• Compare the number of pages in sitemap.xml with the number of pages that actually get traffic from Search Console / Yandex Webmaster over the last 3 months;
• Find sections of the site where pages are generated automatically (filters, tags, cards, reports, parameter combinations)—check robots.txt and the sitemap to see what is included there;
• Take a random sample of 20–30 pages from such a section and assess them honestly: would anyone link to them naturally? Would anyone bookmark them?
• Check how much duplicated or templated text appears across these pages—a significant amount of text variation in a template means nothing if the page structure and logic are identical;
• If you use canonical, make sure it isn’t “hiding” the problem and actually points to the primary version of the content;
• Check whether any background process (cron, worker, integration) is generating pages without moderation—as happened in our case.

What to do if the scale is already large

If there are too many pages to review manually, don’t try to review them all. Work with thresholds, not lists:

• define a measurable value criterion (traffic over a given period, engagement, backlinks, repeat visits—whatever fits your page type);
• automate applying noindex based on this criterion, rather than handling each page manually;
• recalculate the criterion regularly—every month or two—and update the set of indexable pages, because pages can become valuable over time.

The main thing is not to try to fix this with a one-time cleanup. If the mechanism generating low-value pages keeps running, the problem will return in a few months at the same scale.

If you’re not sure whether you have this problem, it’s better to run a technical audit right away and review the indexation structure as a whole rather than guessing based on individual symptoms.