---
title: "Design site architecture that gets your important pages crawled and ranked"
description: "Set URL structure, click depth, and internal links so that authority reaches key pages, orphans disappear, and crawl effort goes where it counts."
canonical: "https://scalewithsearch.com/articles/site-architecture-seo-engineering"
date: "2026-03-20"
modified: "2026-09-25"
---
## Site navigation
- [Scale With Search](https://scalewithsearch.com/)
- Real estate
- Real estate
- [Real estate](https://scalewithsearch.com/for/real-estate)
- Work
- Start here
- [Send your brief](https://scalewithsearch.com/work#send-your-brief)
- [Prepare your six-question brief](https://scalewithsearch.com/work#prepare-your-six-question-brief)
- Build
- [Site build, content library with SEO, signal desk](https://scalewithsearch.com/work)
- For your business
- Trades and home services
- [Auto body and collision shops](https://scalewithsearch.com/for/auto-body-and-collision-shops)
- [Foundation and home repair contractors](https://scalewithsearch.com/for/foundation-and-home-repair)
- [Garage door and fencing contractors](https://scalewithsearch.com/for/garage-door-and-fencing-contractors)
- [HVAC contractors](https://scalewithsearch.com/for/hvac-contractors)
- [Janitorial and commercial cleaning companies](https://scalewithsearch.com/for/janitorial-and-commercial-cleaning)
- [Locksmiths](https://scalewithsearch.com/for/locksmiths)
- [Moving companies](https://scalewithsearch.com/for/moving-companies)
- [Pest control companies](https://scalewithsearch.com/for/pest-control-companies)
- [Plumbing and electrical contractors](https://scalewithsearch.com/for/plumbing-and-electrical-contractors)
- [Restoration and water or fire damage companies](https://scalewithsearch.com/for/restoration-and-water-fire-damage)
- [Roofing companies](https://scalewithsearch.com/for/roofing-companies)
- [Towing companies](https://scalewithsearch.com/for/towing-companies)
- [Tree services and landscaping companies](https://scalewithsearch.com/for/tree-services-and-landscaping)
- [Solar installers](https://scalewithsearch.com/for/solar-installation)
- [General contractors](https://scalewithsearch.com/for/general-contractors-and-construction)
- [Paving, concrete, and flooring contractors](https://scalewithsearch.com/for/paving)
- Practices and professional services
- [Bookkeeping and tax practices](https://scalewithsearch.com/for/bookkeeping-and-tax-practices)
- [Dental practices](https://scalewithsearch.com/for/dental-practices)
- [Family and criminal defense law firms](https://scalewithsearch.com/for/family-and-criminal-defense-law-firms)
- [Med spas and aesthetics practices](https://scalewithsearch.com/for/med-spas-and-aesthetics)
- [Personal injury law firms](https://scalewithsearch.com/for/personal-injury-law-firms)
- [Veterinary clinics](https://scalewithsearch.com/for/veterinary-clinics)
- [Gyms and fitness studios](https://scalewithsearch.com/for/fitness)
- [Therapy and outpatient health practices](https://scalewithsearch.com/for/therapy-and-outpatient-health)
- [Medical billing companies](https://scalewithsearch.com/for/medical-billing)
- [Insurance agencies](https://scalewithsearch.com/for/insurance-agencies)
- [Financial advisors](https://scalewithsearch.com/for/financial-advisors)
- [Property management companies](https://scalewithsearch.com/for/property-management)
- [Recruiting and staffing agencies](https://scalewithsearch.com/for/recruiting-and-staffing)
- [Architects and interior designers](https://scalewithsearch.com/for/architects-and-interior-designers)
- [Logistics and supply chain companies](https://scalewithsearch.com/for/logistics-and-supply-chain)
- Agencies, MSPs, and manufacturing
- [IT and managed service providers](https://scalewithsearch.com/for/it-and-managed-service-providers)
- [Machine shops and precision manufacturers](https://scalewithsearch.com/for/machine-shops-and-precision-manufacturing)
- [Marketing agencies and freelancers](https://scalewithsearch.com/for/marketing-agencies-and-freelancers)
- [SEO agencies and consultants](https://scalewithsearch.com/for/seo-agencies-and-consultants)
- [Small manufacturers and fabricators](https://scalewithsearch.com/for/small-manufacturers-and-fabricators)
- Restaurants, shops, studios, and nonprofits
- [Restaurants and hospitality businesses](https://scalewithsearch.com/for/restaurants-and-hospitality)
- [Retail stores and ecommerce sellers](https://scalewithsearch.com/for/retail-and-ecommerce)
- [Photographers, event planners, and travel agents](https://scalewithsearch.com/for/photographers)
- [Churches and nonprofits](https://scalewithsearch.com/for/churches-and-nonprofits)
- [All industries](https://scalewithsearch.com/for/)
- Learn
- For your office
- [Office job guides](https://scalewithsearch.com/guides/)
- [Browser calculators](https://scalewithsearch.com/tools/)
- Start here
- [How it works](https://scalewithsearch.com/how-it-works)
- [Free Starter Kit](https://scalewithsearch.com/kit/business-memory-starter-kit.zip)
- [Synthetic specimen](https://scalewithsearch.com/specimen/working-session-specimen.zip)
- Guides
- [The Complete Guide to Business Memory for AI Agents](https://scalewithsearch.com/articles/business-memory-for-ai-agents-guide)
- [The Complete Small-Business Guide to AI Agent Governance](https://scalewithsearch.com/articles/ai-agent-governance-guide-small-business)
- [The Complete Guide to Leaving Vendor AI Memory](https://scalewithsearch.com/articles/leaving-vendor-ai-memory-guide)
- Articles by cluster
- [Business memory](https://scalewithsearch.com/articles/business-memory-for-ai-agents-guide)
- [Agent governance](https://scalewithsearch.com/articles/ai-agent-governance-guide-small-business)
- [Migration and ownership](https://scalewithsearch.com/articles/leaving-vendor-ai-memory-guide)
- For machines
- [llms.txt](https://scalewithsearch.com/llms.txt)
- [llms-full.txt](https://scalewithsearch.com/llms-full.txt)
- [Machine view](https://scalewithsearch.com/?view=machine)
- Company
- Evidence
- [Proof](https://scalewithsearch.com/proof)
- Company
- [About](https://scalewithsearch.com/about)
# Design site architecture that gets your important pages crawled and ranked.
Your site has 5,000 pages. Google indexes 1,200. The sitemap lists all 5,000, and Google Search Console shows no errors. The pages exist and are reachable, but Google does not prioritize them.
Site architecture decides which pages Google crawls, indexes, and ranks. Deep hierarchies hide content. Weak internal linking creates orphans: pages with no internal links, which Google finds rarely or not at all.
Treat this as an engineering problem, not only an SEO problem. More content is not the fix. Structure the site as a directed graph, where authority flows on purpose from high-value pages to the pages you want to rank.
## Keep important pages close to the home page
Three things signal a page's importance to Google:
1. **External links** from other sites.
2. **Click depth:** the number of clicks from the home page.
3. **Internal links:** how many pages on your site link to it.
Pages one or two clicks from the home page tend to get crawled often. Pages five or more clicks away get crawled rarely and struggle to rank, whatever their quality.
Aim for this shape:
| Tier | Pages | Click depth |
|---|---|---|
| Home page | 1 | 0 |
| Main categories | 5 to 8 | 1 |
| Subcategories or pillar content | 20 to 50 | 2 |
| Posts or products | Any number | 3 |
Keep each important page within three clicks of the home page. At four clicks or more, a page drops in crawl priority.
## Choose a URL pattern
**Hierarchical URLs** follow the structure: `domain.com/category/subcategory/page`, such as `domain.com/marketing/email-marketing/automation-tools`. The taxonomy shows in the URL, breadcrumbs are easy to build, and users understand the structure from the URL. The URLs get long. A category change needs a 301 redirect for each child page. Each level is also a separate page that competes for link value. Use this pattern for ecommerce, for libraries of 10,000+ pages, and where category pages themselves rank for valuable keywords.
**Flat URLs** ignore the structure: `domain.com/page`, such as `domain.com/email-marketing-automation-tools`. They are short and easy to share. You can restructure internal links without changing URLs, and fewer intermediate pages compete for link value. The structure is not visible in the URL, so navigation and internal links must carry the context. Large taxonomies are harder to manage.
Use this pattern for blogs, SaaS marketing sites, documentation, and sites under about 1,000 pages. URL length itself is not a major ranking factor; the benefit is flexibility.
**A one-level hybrid** is the best default for most sites: `domain.com/broad-category/page`, such as `domain.com/blog/email-marketing-automation` or `domain.com/features/analytics`. It keeps some context without depth. You change subcategories through internal links, not URLs.
## Build internal links in four layers
### Hub and spoke
One hub page covers a broad topic and links to 8 to 15 spoke pages on subtopics. For example, an email marketing guide of 3,000 words is the hub. Articles on segmentation, drip campaign examples, and subject line A/B tests are spokes.
Use this linking pattern:
- The hub links to each spoke.
- Each spoke links back to the hub.
- Each spoke links to two or three related spokes.
Authority collects on the hub, which helps it rank for competitive head terms, and the spokes benefit from the hub. Create the hub first. Publish spokes over two or three months, each with a hub link on publish. Update the hub each quarter with links to new spokes.
### Contextual links
Contextual links sit in the body text between related articles, often across clusters. An article on email marketing automation can link the word "segmentation" to an article in the CRM cluster. Keep three to five contextual links in each article; with many more, each link carries less value. Use descriptive anchor text, such as "email segmentation strategies", not "click here". Link only where it helps, and place key links early in the article.
### Sitewide links
Footer and sidebar links appear on each page, so key pages are always one click away and never orphaned. A footer with 50 links passes little value through each link. Google also treats sitewide links as navigation more than as editorial votes. Keep the footer to 10 to 15 critical links: products, top resources, and legal pages. Rely on contextual links for SEO value. A "related articles" sidebar works well when it changes for each page.
### Breadcrumbs
Breadcrumbs show the page's place in the hierarchy, such as `Home > Blog > Email Marketing > Automation Tools`. They help users, and Google follows their links. Google shows breadcrumbs in desktop results. In January 2025, it stopped showing them in the URL line of mobile results. Mark them up with `BreadcrumbList`:
```json
{
"@context": "https://schema.org",
"@type": "BreadcrumbList",
"itemListElement": [
{ "@type": "ListItem", "position": 1, "name": "Home", "item": "https://example.com" },
{ "@type": "ListItem", "position": 2, "name": "Blog", "item": "https://example.com/blog" },
{ "@type": "ListItem", "position": 3, "name": "Email Marketing Automation",
"item": "https://example.com/blog/email-marketing-automation" }
]
}
```
The [internal linking architecture guide for developers](/articles/internal-linking-architecture-developers) covers the link graph in more depth.
## Handle pagination and archives correctly
Paginated archives and category pages (page 1, 2, 3) can bury older content deep in the site. Several older fixes are now wrong:
- Google stopped using `rel="next"` and `rel="prev"` as an indexing signal in 2019. They do no harm, but they do not group the series.
- Do not canonicalize page 2 and later to page 1. Page 2 has different content, so Google can ignore that canonical, and the posts listed only there lose a path.
- Do not noindex pages 2 and later by default. Over time, Google can also stop following links on noindexed pages, which strands the posts that those pages link to.
Instead, give each paginated page a self-referencing canonical and crawlable `` links to the next and previous pages:
```html
PreviousNext
```
A "view all" page that lists every post can serve as the canonical target. It must load fast, which usually means under about 100 posts. Keep archives shallow with more links on each page, and link to older posts from hubs and related content.
## Find and fix orphan pages
An orphan page has no internal links. Google can find it only through the sitemap or external links. To find orphans:
1. Crawl the site with Screaming Frog or Sitebulb, with the XML sitemap as an extra source.
2. Export the URLs that have no internal links.
3. Compare them with the indexed pages in Search Console.
Orphans usually come from old posts that newer content never linked, products missing from navigation and category pages, and landing pages built for ad campaigns. Pages removed from navigation but never deleted add more. Fix each orphan with contextual links from related content or a link from its cluster hub. A place on a category page also works. If the page is truly obsolete, delete it with a 301 redirect, or noindex it.
## Spend crawl effort where it counts
Google sets crawling by two factors. The crawl capacity limit depends on how fast and reliably your server responds. Crawl demand depends on how popular your URLs are and how often they change. Errors and low-value URLs waste both. Crawl budget matters most on large sites, of tens of thousands of URLs or more. The [robots.txt and crawl budget guide](/articles/robots-txt-crawl-budget-developers) covers it in depth.
**Remove redirect chains.** Point each internal link at the final URL: A to C, not A to B to C. The Screaming Frog redirect chains report lists them.
**Fix broken internal links.** Each 404 is a wasted fetch. Filter the Screaming Frog response codes report for 404s. Fix the link, or add a 301 if the target moved.
**Block low-value URL spaces.** Admin pages, internal search results, and filter combinations can create thousands of URLs. Block them in robots.txt:
```text
User-agent: *
Disallow: /admin/
Disallow: /search?
Disallow: /*?filter=
```
Google cannot read a canonical or `noindex` on a page that robots.txt blocks, so choose one method for each URL space.
**Control parameter URLs.** Faceted navigation multiplies URLs, because color, size, brand, and price create exponential combinations. Canonicalize filtered URLs to the base category, noindex the combinations, or load filters without new URLs. The Search Console URL Parameters tool was retired in April 2022, so handle parameters on the site. The [faceted navigation guide](/articles/ecommerce-seo-faceted-navigation) covers the trade-offs.
```html
```
**Clean the XML sitemap.** List only indexable URLs that return 200, and remove noindexed, redirected, and 404 URLs. Keep `lastmod` accurate, because Google uses it when it is reliable. Google ignores `priority` and `changefreq`. Split sitemaps above 50,000 URLs. The [XML sitemap guide for developers](/articles/xml-sitemap-best-practices-developers) covers generation.
**Slow Googlebot only in an emergency.** Googlebot ignores the robots.txt `Crawl-delay` rule. If crawling overloads the server, return 503 or 429 status codes for a short time, and Google slows down. Do this only for a real load problem, and only briefly.
## Scale internal linking with code
Manual linking stops scaling at around 500 pages. Automate it in three ways.
**A related posts module.** Pick three to five related posts by tag, category, or text similarity. Show them at the end of each article. Each new post then links to related content and receives links back. A WordPress example:
```php
wp_get_post_categories(get_the_ID()),
'numberposts' => 5,
'post__not_in' => array(get_the_ID()),
));
foreach ($related as $item) {
echo ''
. esc_html(get_the_title($item->ID)) . '';
}
?>
```
**Suggested contextual links.** A script scans body text for phrases that match other article titles and proposes a link. If an article mentions "email segmentation" and you have "Email Segmentation Strategies", the script suggests that link. Build it with spaCy or NLTK, or use a plugin such as Link Whisper. Full automation produces spammy links, so cap it at three to five links for each article, and require editorial review.
**Cluster-based links.** Tag each article with its cluster. On publish, a script links the article to its hub and to two or three related articles in the cluster. Hubs then gain links from each new spoke.
## Work through a restructure of a 10,000-page site
Consider a site with 10,000 published pages, of which Google indexes 2,000. Important pages sit six or more clicks from the home page, and 4,000 pages are orphans. The diagnosis finds four causes:
- A deep URL hierarchy: `/category/subcategory/sub-subcategory/article`.
- No internal linking strategy; articles do not link to each other.
- A footer with 80 links.
- An XML sitemap that lists 404s and noindexed pages.
The restructure runs in four phases:
1. **Simplify URLs.** Flatten them to `/blog/article-title`, with 301 redirects from the old paths. Maximum click depth falls from six to three.
2. **Cluster.** Define 12 topic clusters, choose a hub for each, link each article to its hub, and cross-link spokes.
3. **Remove orphans.** Find the 4,000 orphans with Screaming Frog, add contextual links, and noindex 800 outdated pages.
4. **Cut crawl waste.** Fix 1,200 broken internal links, remove redirect chains, clean the sitemap, and cut the footer from 80 links to 12.
Measure the result six months later. In this illustration, indexed pages rise from 2,000 to 8,500. Organic traffic rises 185%, top-10 keywords go from 400 to 1,200, and average click depth falls to 2.8. These numbers illustrate the method; they are not a forecast. The content did not change; the structure did.
## Avoid five structural mistakes
**Exact-match anchors everywhere.** If each link to the email marketing page says "email marketing", the pattern looks manipulative. Vary anchor text naturally, and keep it descriptive, not "click here".
**Links that spend value on low-value pages.** About, Contact, and Privacy pages do not need links from each blog post. Save contextual links for pages you want to rank.
**Closed loops.** Suppose A links to B, B to C, and C back to A, with no path from the home page. That cluster sits isolated, and Google rarely crawls it.
**Old content left behind.** Suppose you publish 500 new articles and never link to them from the first 100. The older articles lose their place in the graph. Revisit old content when you publish related new content.
**Links that only JavaScript can follow.** Google renders JavaScript, but links must use `` elements, not click handlers. They must also be in the server-rendered HTML.
## Answer common questions
**How many internal links per page?** No strict limit applies. Blog posts usually carry 3 to 10 contextual links. Ecommerce category pages can carry 50 to 100 product links. Avoid footers with 80 or more sitewide links.
**Should internal links be nofollow?** Rarely. A nofollow internal link stops value from flowing. Use it for user-generated content such as comments and forum posts. For pages such as login and register, noindex the pages instead.
**Deep or flat?** Flat for flexibility and concentrated authority. Deep only for very large sites, of 10,000+ pages, where category pages must rank. Most sites under 5,000 pages do better with a flat or one-level structure.
**How often should you audit?** Each quarter for sites that publish 20+ articles each month, and each year for slower sites. Each audit looks for orphans, broken links, and redirect chains in Screaming Frog. Server log analysis shows where Googlebot actually spends its time; the [log file analysis guide](/articles/log-file-analysis-seo) shows how.
**Do internal links matter as much as external links?** No. External links carry more weight. But internal links decide which pages benefit from those external links. A site with 1,000 external links and weak internal linking can underperform a site with 500 external links and strong internal linking.
----
```text
.|########||. .|########||. .|########||.
|##||. .||##|. |##||. .||##|. |##||. .||##|.
|#|. .|#|. |#|. .|#|. |#|. .|#|.
|#| |#| |#| |#| |#| |#|
.#| |#. .#| |#. .#| |#.
.#. .#| .#. .#| .#. .#|
|#. .#| |#. .#| |#. .#|
|# ...... #| |# ...... #| |# ...... #|
.# ||#########| #| .# ||#########| #| .# ||#########| #|
.#. |######||######|. .#. .#. |######||######|. .#. .#. |######||######|. .#.
#. .##|###|##|#|######| .# #. .##|###|##|#|######| .# #. .##|###|##|#|######| .#
|| |##|#||||||||||||#||#| || || |##|#||||||||||||#||#| || || |##|#||||||||||||#||#| ||
# |#||||||||||||||||||||#| #. # |#||||||||||||||||||||#| #. # |#||||||||||||||||||||#| #.
|| |#||||||||||||||||||||||#| || || |#||||||||||||||||||||||#| || || |#||||||||||||||||||||||#| ||
# .#||||||||||||||||||||||||#| # # .#||||||||||||||||||||||||#| # # .#||||||||||||||||||||||||#| #
|| ....|||#||||||##|#|||#|#||##||||....|. || || ....|||#||||||##|#|||#|#||##||||....|. || || ....|||#||||||##|#|||#|#||##||||....|. ||
#. . ....|# ....#||| ||| .#||| ....#. .# #. . ....|# ....#||| ||| .#||| ....#. .# #. . ....|# ....#||| ||| .#||| ....#. .#
# . ||||#| .#####|| . .#| .#||| ||||# #. # . ||||#| .#####|| . .#| .#||| ||||# #. # . ||||#| .#####|| . .#| .#||| ||||# #.
.| ||||| #. |#|||||. || #. |#||. .||||| || .| ||||| #. |#|||||. || #. |#||. .||||| || .| ||||| #. |#|||||. || #. |#||. .||||| ||
|| |.... #. ....|#. |. ...|. ....|| || || |.... #. ....|#. |. ...|. ....|| || || |.... #. ....|#. |. ...|. ....|| ||
#. .||||||##||||||##||####|||||||#||||||#| .# #. .||||||##||||||##||####|||||||#||||||#| .# #. .||||||##||||||##||####|||||||#||||||#| .#
# .||####|###########################|. # # .||####|###########################|. # # .||####|###########################|. #
# .#||||||.#.|| # |. ..# #| #|||||#| # # .#||||||.#.|| # |. ..# #| #|||||#| # # .#||||||.#.|| # |. ..# #| #|||||#| #
# .#|||||| .. |# ## |#| ...#|#|||#| # # .#|||||| .. |# ## |#| ...#|#|||#| # # .#|||||| .. |# ## |#| ...#|#|||#| #
# .#|||#|# .# .#| #| ##|.#..#|||||#| # # .#|||#|# .# .#| #| ##|.#..#|||||#| # # .#|||#|# .# .#| #| ##|.#..#|||||#| #
# .#||||||############|######|||||#| # # .#||||||############|######|||||#| # # .#||||||############|######|||||#| #
# |...||#||||||#||||#|#||||||#|||||#||| |.| # # |...||#||||||#||||#|#||||||#|||||#||| |.| # # |...||#||||||#||||#|#||||||#|||||#||| |.| #
#. .. ||||# .|||##|. |#| .|| || .|||#. #| # .# #. .. ||||# .|||##|. |#| .|| || .|||#. #| # .# #. .. ||||# .|||##|. |#| .|| || .|||#. #| # .#
|| | ...|| ...##| | #| .|. |. ##### .. .| || || | ...|| ...##| | #| .|. |. ##### .. .| || || | ...|| ...##| | #| .|. |. ##### .. .| ||
|| ||||| || ||||#| . |. |. |#. ||||| |#| || || || ||||| || ||||#| . |. |. |#. ||||| |#| || || || ||||| || ||||#| . |. |. |#. ||||| |#| || ||
.# |.....#|....|#.||||.|||##.|#|....||.#||.#. #. .# |.....#|....|#.||||.|||##.|#|....||.#||.#. #. .# |.....#|....|#.||||.|||##.|#|....||.#||.#. #.
#.|||||||#############################| ||| .# #.|||||||#############################| ||| .# #.|||||||#############################| ||| .#
|| ##||#||#||||||#||#||#||#||##. || || ##||#||#||||||#||#||#||#||##. || || ##||#||#||||||#||#||#||#||##. ||
# .#|||||||||#||#|||||||||||#| # # .#|||||||||#||#|||||||||||#| # # .#|||||||||#||#|||||||||||#| #
|| |#||||||||||||||||||||||#| || || |#||||||||||||||||||||||#| || || |#||||||||||||||||||||||#| ||
.# |#||||||||||||||||||||#| #. .# |#||||||||||||||||||||#| #. .# |#||||||||||||||||||||#| #.
|| |#||||||||||||||||||#| || || |#||||||||||||||||||#| || || |#||||||||||||||||||#| ||
#. .######|#|#########| .# #. .######|#|#########| .# #. .######|#|#########| .#
.#. |#####||||#####| .#. .#. |#####||||#####| .#. .#. |#####||||#####| .#.
|# ||########|| #. |# ||########|| #. |# ||########|| #.
|# ...... #| |# ...... #| |# ...... #|
|#. .#| |#. .#| |#. .#|
|#. .#. |#. .#. |#. .#.
.#| |#. .#| |#. .#| |#.
|#| |#| |#| |#| |#| |#|
.|#|. .|#|. .|#|. .|#|. .|#|. .|#|.
|##||. .||##| |##||. .||##| |##||. .||##|
.||########||. .||########||. .||########||.
Scale With Search 2026 [scalewithsearch.com](https://scalewithsearch.com)
```