---
title: "Configure your CDN so crawlers get fresh pages and users get fast ones"
description: "Set CDN cache headers, purging, edge rendering, and regional rules so Google sees fresh HTML, with Cloudflare, Fastly, CloudFront, and Netlify examples."
canonical: "https://scalewithsearch.com/articles/cdn-configuration-seo"
date: "2026-03-20"
modified: "2026-09-25"
---
## Site navigation
- [Scale With Search](https://scalewithsearch.com/)
- Real estate
- Real estate
- [Real estate](https://scalewithsearch.com/for/real-estate)
- Work
- Start here
- [Send your brief](https://scalewithsearch.com/work#send-your-brief)
- [Prepare your six-question brief](https://scalewithsearch.com/work#prepare-your-six-question-brief)
- Build
- [Site build, content library with SEO, signal desk](https://scalewithsearch.com/work)
- For your business
- Trades and home services
- [Auto body and collision shops](https://scalewithsearch.com/for/auto-body-and-collision-shops)
- [Foundation and home repair contractors](https://scalewithsearch.com/for/foundation-and-home-repair)
- [Garage door and fencing contractors](https://scalewithsearch.com/for/garage-door-and-fencing-contractors)
- [HVAC contractors](https://scalewithsearch.com/for/hvac-contractors)
- [Janitorial and commercial cleaning companies](https://scalewithsearch.com/for/janitorial-and-commercial-cleaning)
- [Locksmiths](https://scalewithsearch.com/for/locksmiths)
- [Moving companies](https://scalewithsearch.com/for/moving-companies)
- [Pest control companies](https://scalewithsearch.com/for/pest-control-companies)
- [Plumbing and electrical contractors](https://scalewithsearch.com/for/plumbing-and-electrical-contractors)
- [Restoration and water or fire damage companies](https://scalewithsearch.com/for/restoration-and-water-fire-damage)
- [Roofing companies](https://scalewithsearch.com/for/roofing-companies)
- [Towing companies](https://scalewithsearch.com/for/towing-companies)
- [Tree services and landscaping companies](https://scalewithsearch.com/for/tree-services-and-landscaping)
- [Solar installers](https://scalewithsearch.com/for/solar-installation)
- [General contractors](https://scalewithsearch.com/for/general-contractors-and-construction)
- [Paving, concrete, and flooring contractors](https://scalewithsearch.com/for/paving)
- Practices and professional services
- [Bookkeeping and tax practices](https://scalewithsearch.com/for/bookkeeping-and-tax-practices)
- [Dental practices](https://scalewithsearch.com/for/dental-practices)
- [Family and criminal defense law firms](https://scalewithsearch.com/for/family-and-criminal-defense-law-firms)
- [Med spas and aesthetics practices](https://scalewithsearch.com/for/med-spas-and-aesthetics)
- [Personal injury law firms](https://scalewithsearch.com/for/personal-injury-law-firms)
- [Veterinary clinics](https://scalewithsearch.com/for/veterinary-clinics)
- [Gyms and fitness studios](https://scalewithsearch.com/for/fitness)
- [Therapy and outpatient health practices](https://scalewithsearch.com/for/therapy-and-outpatient-health)
- [Medical billing companies](https://scalewithsearch.com/for/medical-billing)
- [Insurance agencies](https://scalewithsearch.com/for/insurance-agencies)
- [Financial advisors](https://scalewithsearch.com/for/financial-advisors)
- [Property management companies](https://scalewithsearch.com/for/property-management)
- [Recruiting and staffing agencies](https://scalewithsearch.com/for/recruiting-and-staffing)
- [Architects and interior designers](https://scalewithsearch.com/for/architects-and-interior-designers)
- [Logistics and supply chain companies](https://scalewithsearch.com/for/logistics-and-supply-chain)
- Agencies, MSPs, and manufacturing
- [IT and managed service providers](https://scalewithsearch.com/for/it-and-managed-service-providers)
- [Machine shops and precision manufacturers](https://scalewithsearch.com/for/machine-shops-and-precision-manufacturing)
- [Marketing agencies and freelancers](https://scalewithsearch.com/for/marketing-agencies-and-freelancers)
- [SEO agencies and consultants](https://scalewithsearch.com/for/seo-agencies-and-consultants)
- [Small manufacturers and fabricators](https://scalewithsearch.com/for/small-manufacturers-and-fabricators)
- Restaurants, shops, studios, and nonprofits
- [Restaurants and hospitality businesses](https://scalewithsearch.com/for/restaurants-and-hospitality)
- [Retail stores and ecommerce sellers](https://scalewithsearch.com/for/retail-and-ecommerce)
- [Photographers, event planners, and travel agents](https://scalewithsearch.com/for/photographers)
- [Churches and nonprofits](https://scalewithsearch.com/for/churches-and-nonprofits)
- [All industries](https://scalewithsearch.com/for/)
- Learn
- For your office
- [Office job guides](https://scalewithsearch.com/guides/)
- [Browser calculators](https://scalewithsearch.com/tools/)
- Start here
- [How it works](https://scalewithsearch.com/how-it-works)
- [Free Starter Kit](https://scalewithsearch.com/kit/business-memory-starter-kit.zip)
- [Synthetic specimen](https://scalewithsearch.com/specimen/working-session-specimen.zip)
- Guides
- [The Complete Guide to Business Memory for AI Agents](https://scalewithsearch.com/articles/business-memory-for-ai-agents-guide)
- [The Complete Small-Business Guide to AI Agent Governance](https://scalewithsearch.com/articles/ai-agent-governance-guide-small-business)
- [The Complete Guide to Leaving Vendor AI Memory](https://scalewithsearch.com/articles/leaving-vendor-ai-memory-guide)
- Articles by cluster
- [Business memory](https://scalewithsearch.com/articles/business-memory-for-ai-agents-guide)
- [Agent governance](https://scalewithsearch.com/articles/ai-agent-governance-guide-small-business)
- [Migration and ownership](https://scalewithsearch.com/articles/leaving-vendor-ai-memory-guide)
- For machines
- [llms.txt](https://scalewithsearch.com/llms.txt)
- [llms-full.txt](https://scalewithsearch.com/llms-full.txt)
- [Machine view](https://scalewithsearch.com/?view=machine)
- Company
- Evidence
- [Proof](https://scalewithsearch.com/proof)
- Company
- [About](https://scalewithsearch.com/about)
# Configure your CDN so crawlers get fresh pages and users get fast ones.
You fixed a wrong price on a product page an hour ago. The copy Googlebot fetched still shows the old one, and so does the page a customer in Berlin loads. The CDN is doing exactly what its default rule says: cache everything for a day.
A content delivery network (CDN) serves your HTML and assets from edge nodes near each user. It affects SEO through page speed, cache freshness, rendering, and geographic delivery. A well-configured CDN cuts latency sharply against a single origin server. A misconfigured one serves stale pages to crawlers, strips SEO headers, creates duplicate hostnames, hides regional content, and slows Core Web Vitals through cache misses.
Default CDN settings suit generic sites, not SEO. This guide covers the architecture, cache headers, purging, rendering modes, geographic rules, platform settings, and troubleshooting.
## Understand how a CDN serves a request
The origin server holds your original content in one region. Edge nodes, often hundreds of them worldwide, cache copies near users. A request follows five steps:
1. A user requests `example.com/page`.
2. DNS routes the request to the nearest edge node.
3. On a cache hit, the edge serves the cached copy at once.
4. On a cache miss, the edge fetches from the origin, stores the response, and serves it.
5. Later requests get the cached copy until it expires.
Lower latency improves Core Web Vitals such as Largest Contentful Paint (LCP) and Interaction to Next Paint (INP). INP replaced First Input Delay as a Core Web Vital in March 2024.
### See the CDN from the crawler's side
Googlebot crawls mainly from IP addresses in the United States. A CDN that changes content or redirects by country can therefore hide regional versions from Google. Some CDN setups also treat crawler user agents differently from people. That is risky unless it is deliberate and serves the same content. Google's crawlers support conditional requests with `ETag` and `Last-Modified`, so accurate validators save crawl effort.
### Know the five common failures
- **Stale content.** Long cache lifetimes keep serving old pages after you update them.
- **Stripped headers.** The edge drops `Link` canonical headers, hreflang headers, or `Last-Modified` if you do not preserve them.
- **Duplicate hosts.** HTML served from `cdn.example.com` competes with `example.com`.
- **Regional inconsistency.** Edge nodes serve localized variants without hreflang.
- **Over-cached dynamic data.** Prices and stock levels go stale.
## Set cache headers by content type
### Know the Cache-Control directives
| Directive | Meaning | Use it for |
|---|---|---|
| `public` | Any cache, including the CDN, may store it | Assets identical for all users |
| `private` | Only the browser may store it | Logged-in pages, carts |
| `no-cache` | May be stored, but must be revalidated before use | Often-updated content |
| `no-store` | Never store it | Payment forms, personal data |
| `max-age=N` | Fresh for N seconds | Browser and default lifetime |
| `s-maxage=N` | Fresh for N seconds in shared caches such as the CDN; overrides `max-age` there | Edge lifetime |
| `immutable` | Never changes; skip revalidation | Assets with hashed file names |
### Use these starting configurations
```text
# HTML pages: 5 minutes in the browser, 10 minutes at the edge
Cache-Control: public, max-age=300, s-maxage=600
# Hashed static assets such as app.abc123.js: one year
Cache-Control: public, max-age=31536000, immutable
# Images: 30 days
Cache-Control: public, max-age=2592000
# Often-updated API data: 1 minute, then revalidate
Cache-Control: public, max-age=60, must-revalidate
# User-specific content: never at the edge
Cache-Control: private, max-age=0, no-cache
```
Short HTML lifetimes let updates reach crawlers within about 10 minutes and still spare the origin. Hashed asset names change on every deploy, so a one-year lifetime never serves an old file. Use 5 to 10 minutes for HTML on dynamic sites and 1 to 24 hours for pages that rarely change.
### Add ETag and Last-Modified for cheap revalidation
An `ETag` is a hash of the content version, such as `ETag: "5d8c72a5edda8d1:0"`. The client sends it back in `If-None-Match`. If nothing changed, the server answers `304 Not Modified` with no body. `Last-Modified: Wed, 15 Jan 2026 12:00:00 GMT` works the same way with `If-Modified-Since`. Both make revalidation cheap, so crawlers see fresh content without full downloads.
In Express:
```javascript
app.use(express.static('public', {
etag: true,
lastModified: true,
setHeaders: (res, path) => {
if (path.endsWith('.html')) {
res.setHeader('Cache-Control', 'public, max-age=300');
} else if (/\.(css|js)$/.test(path)) {
res.setHeader('Cache-Control', 'public, max-age=31536000, immutable');
}
}
}));
```
### Use Vary with care
`Vary` tells caches to keep separate copies per request header. `Vary: Accept-Encoding` keeps separate gzip, Brotli, and plain copies, which is correct. `Vary: User-Agent` is needed only if you serve different HTML to mobile and desktop at the same URL. It splits the cache into many copies, so prefer responsive design. `Vary: Cookie` fragments the cache even more.
### Purge on publish
A purge removes cached copies so the next request fetches fresh content. Purge when you publish or update content, correct time-sensitive facts, or finish a redesign. Four methods exist:
- **URL purge** for specific pages, such as `/products/widget-123`;
- **tag purge**, where you tag responses and purge a tag such as `category:shoes`;
- **wildcard purge** for a pattern such as `/blog/*`;
- **full purge**, which clears everything and should be rare.
Cloudflare, Fastly, and CloudFront all offer purge APIs. Call them from your deployment pipeline so crawlers see new content at once.
## Choose a rendering mode
| Mode | How it works | SEO effect | Fits |
|---|---|---|---|
| Static generation | HTML built at deploy time and served from the edge | Fastest, fully crawlable, most cacheable; updates need a rebuild | Marketing sites, blogs, documentation |
| Edge server rendering | HTML rendered on request at the edge node | Full HTML for crawlers, dynamic data, low latency; more complex | Personalized or real-time pages |
| Incremental static regeneration | Cached HTML regenerated in the background after a set interval | Static speed with near-current content | Catalogs and content that changes through the day |
| Client-side rendering | A thin HTML shell; content loads through JavaScript | Rendering costs Google time; a script error can hide content | Logged-in app interfaces only |
In Next.js, static export uses `output: 'export'` in `next.config.js`. Incremental regeneration uses `revalidate: 60` in `getStaticProps` in the Pages Router, or `export const revalidate = 60` in the App Router.
An edge render in a Cloudflare Worker, in current module syntax:
```javascript
export default {
async fetch(request) {
const html = `Edge page
Rendered at ${new Date().toISOString()}
`;
return new Response(html, { headers: { 'Content-Type': 'text/html' } });
},
};
```
Google sees rendered HTML the same way whether the origin or the edge produced it. Edge rendering helps SEO by cutting time to first byte, as long as the first response contains the HTML. Do not cache a client-side HTML shell indefinitely either; its meta tags and structured data must stay current. [Edge SEO with CDN workers](/articles/edge-seo-cdn-workers) covers other edge tasks.
## Keep regional content crawlable
### Declare language versions with hreflang
When you serve language or regional versions, list them all:
```html
```
Do not block any region by IP. Every version must stay reachable for Googlebot. [The hreflang implementation guide](/articles/hreflang-implementation-guide) covers the details.
### Avoid forced geographic redirects
A German visitor to `example.com` might be redirected to `example.de`. Googlebot, crawling mostly from the United States, may then never see `example.de`. Use one of three safer patterns:
- Offer a region selector and do not redirect.
- Show a banner such as "Visit our German site?" instead of a server-side 301 or 302.
- Redirect only the homepage by region, and keep every deep URL reachable directly.
### Read geographic signals at the edge
Cloudflare adds a `CF-IPCountry` header. CloudFront adds `CloudFront-Viewer-Country` when the origin request policy includes it. Fastly exposes `client.geo.country_code` in VCL and sends `Fastly-Client-IP`. You can use these to adjust content, but each regional variant also needs its own URL and hreflang so crawlers can find it.
## Configure each platform
### Cloudflare
Use Cache Rules, which replace the older Page Rules, to cache HTML. For `example.com/blog/*`, set the rule to cache eligible content. Use an edge lifetime of 2 hours and a browser lifetime of 30 minutes. Turn on Polish for image compression under the Speed settings. For WordPress, Automatic Platform Optimization (APO) caches WordPress HTML at the edge and usually improves Core Web Vitals.
A Worker can set headers on origin responses:
```javascript
export default {
async fetch(request) {
const origin = await fetch(request);
const response = new Response(origin.body, origin);
response.headers.set('Cache-Control', 'public, max-age=300');
return response;
},
};
```
### Fastly
Fastly uses Varnish Configuration Language (VCL):
```vcl
sub vcl_recv {
if (req.http.Cookie ~ "wordpress_logged_in") {
return(pass);
}
}
sub vcl_fetch {
if (beresp.http.Content-Type ~ "text/html") {
set beresp.ttl = 5m;
} else if (beresp.http.Content-Type ~ "image/") {
set beresp.ttl = 30d;
}
}
```
Fastly Compute runs custom Rust, JavaScript, or Go at the edge for dynamic rendering.
### AWS CloudFront
Define cache behaviors per path. Give `/static/*` the managed `CachingOptimized` policy with compression on. Give HTML a short custom policy or `CachingDisabled`, and forward the headers and cookies that dynamic pages need. Lambda@Edge runs Node.js or Python functions at edge locations, for example to add headers. Purge with the AWS CLI:
```bash
aws cloudfront create-invalidation \
--distribution-id DISTRIBUTION_ID \
--paths "/blog/*" "/products/*"
```
### Netlify
Set headers in `public/_headers`:
```text
/static/*
Cache-Control: public, max-age=31536000, immutable
/*.html
Cache-Control: public, max-age=300
```
Set redirects in `public/_redirects`:
```text
/old-page /new-page 301
/blog/* /news/:splat 301
```
Netlify Edge Functions run TypeScript on Deno at the edge.
## Troubleshoot CDN faults that hurt SEO
**Updates do not appear.** Check the HTML cache lifetime, then request the page with a cache-busting parameter such as `?v=123`. Cut the HTML lifetime to 10 minutes or less, purge on publish, and add purging to the deploy pipeline.
**SEO headers disappear.** Compare the edge and the origin:
```bash
curl -I https://example.com/page
curl -I https://origin.example.com/page
```
Sometimes the edge drops a `Link` canonical or hreflang header. Configure the CDN to pass origin headers through, or set the headers at the edge. [Canonical tags for developers](/articles/canonical-tags-developers) explains which header form to use.
**The CDN hostname is indexed.** Search `site:cdn.example.com`. Serve HTML only from the main domain and use the CDN hostname for assets. If the CDN hostname must serve HTML, add canonicals to the main domain. A robots.txt block on the CDN hostname stops crawling. It also stops Google from seeing those canonicals, so prefer canonicals and asset-only use.
**Fast in one region, slow in another.** Test from several locations with WebPageTest or GTmetrix. Check the provider's coverage in your target markets, since some have thin coverage in parts of Asia. Reduce origin dependence with edge rendering, or add a regional origin.
**One user sees another user's data.** The CDN cached personal content. Send `Cache-Control: private, no-cache` on authenticated pages. If you must cache lightly personalized pages, vary on a specific cookie, and accept the cache fragmentation. Better, cache a static shell and load personal elements in the browser.
## Monitor five CDN metrics
| Metric | Target | What a miss means |
|---|---|---|
| Time to first byte | Under about 200 ms for cached content, under about 600 ms for dynamic content | Cache misses or a slow origin |
| Largest Contentful Paint | Under 2.5 seconds | HTML or hero images arrive too slowly |
| Cache hit rate | Above 90 percent for static assets, above 70 percent for HTML | Short lifetimes, bad cache keys, or too many purges |
| Origin load | Low and steady | Spikes point to cache misses or purges |
| Regional latency | Little variation between regions | Weak coverage or origin dependence |
The time-to-first-byte targets are this guide's working values, tighter than the general web guidance. Set them in your performance budget. [Performance budgets for SEO](/articles/performance-budgets-seo) shows how, and [page speed optimization for developers](/articles/page-speed-optimization-developers) covers the rest of the stack.
----
```text
.|########||. .|########||. .|########||.
|##||. .||##|. |##||. .||##|. |##||. .||##|.
|#|. .|#|. |#|. .|#|. |#|. .|#|.
|#| |#| |#| |#| |#| |#|
.#| |#. .#| |#. .#| |#.
.#. .#| .#. .#| .#. .#|
|#. .#| |#. .#| |#. .#|
|# ...... #| |# ...... #| |# ...... #|
.# ||#########| #| .# ||#########| #| .# ||#########| #|
.#. |######||######|. .#. .#. |######||######|. .#. .#. |######||######|. .#.
#. .##|###|##|#|######| .# #. .##|###|##|#|######| .# #. .##|###|##|#|######| .#
|| |##|#||||||||||||#||#| || || |##|#||||||||||||#||#| || || |##|#||||||||||||#||#| ||
# |#||||||||||||||||||||#| #. # |#||||||||||||||||||||#| #. # |#||||||||||||||||||||#| #.
|| |#||||||||||||||||||||||#| || || |#||||||||||||||||||||||#| || || |#||||||||||||||||||||||#| ||
# .#||||||||||||||||||||||||#| # # .#||||||||||||||||||||||||#| # # .#||||||||||||||||||||||||#| #
|| ....|||#||||||##|#|||#|#||##||||....|. || || ....|||#||||||##|#|||#|#||##||||....|. || || ....|||#||||||##|#|||#|#||##||||....|. ||
#. . ....|# ....#||| ||| .#||| ....#. .# #. . ....|# ....#||| ||| .#||| ....#. .# #. . ....|# ....#||| ||| .#||| ....#. .#
# . ||||#| .#####|| . .#| .#||| ||||# #. # . ||||#| .#####|| . .#| .#||| ||||# #. # . ||||#| .#####|| . .#| .#||| ||||# #.
.| ||||| #. |#|||||. || #. |#||. .||||| || .| ||||| #. |#|||||. || #. |#||. .||||| || .| ||||| #. |#|||||. || #. |#||. .||||| ||
|| |.... #. ....|#. |. ...|. ....|| || || |.... #. ....|#. |. ...|. ....|| || || |.... #. ....|#. |. ...|. ....|| ||
#. .||||||##||||||##||####|||||||#||||||#| .# #. .||||||##||||||##||####|||||||#||||||#| .# #. .||||||##||||||##||####|||||||#||||||#| .#
# .||####|###########################|. # # .||####|###########################|. # # .||####|###########################|. #
# .#||||||.#.|| # |. ..# #| #|||||#| # # .#||||||.#.|| # |. ..# #| #|||||#| # # .#||||||.#.|| # |. ..# #| #|||||#| #
# .#|||||| .. |# ## |#| ...#|#|||#| # # .#|||||| .. |# ## |#| ...#|#|||#| # # .#|||||| .. |# ## |#| ...#|#|||#| #
# .#|||#|# .# .#| #| ##|.#..#|||||#| # # .#|||#|# .# .#| #| ##|.#..#|||||#| # # .#|||#|# .# .#| #| ##|.#..#|||||#| #
# .#||||||############|######|||||#| # # .#||||||############|######|||||#| # # .#||||||############|######|||||#| #
# |...||#||||||#||||#|#||||||#|||||#||| |.| # # |...||#||||||#||||#|#||||||#|||||#||| |.| # # |...||#||||||#||||#|#||||||#|||||#||| |.| #
#. .. ||||# .|||##|. |#| .|| || .|||#. #| # .# #. .. ||||# .|||##|. |#| .|| || .|||#. #| # .# #. .. ||||# .|||##|. |#| .|| || .|||#. #| # .#
|| | ...|| ...##| | #| .|. |. ##### .. .| || || | ...|| ...##| | #| .|. |. ##### .. .| || || | ...|| ...##| | #| .|. |. ##### .. .| ||
|| ||||| || ||||#| . |. |. |#. ||||| |#| || || || ||||| || ||||#| . |. |. |#. ||||| |#| || || || ||||| || ||||#| . |. |. |#. ||||| |#| || ||
.# |.....#|....|#.||||.|||##.|#|....||.#||.#. #. .# |.....#|....|#.||||.|||##.|#|....||.#||.#. #. .# |.....#|....|#.||||.|||##.|#|....||.#||.#. #.
#.|||||||#############################| ||| .# #.|||||||#############################| ||| .# #.|||||||#############################| ||| .#
|| ##||#||#||||||#||#||#||#||##. || || ##||#||#||||||#||#||#||#||##. || || ##||#||#||||||#||#||#||#||##. ||
# .#|||||||||#||#|||||||||||#| # # .#|||||||||#||#|||||||||||#| # # .#|||||||||#||#|||||||||||#| #
|| |#||||||||||||||||||||||#| || || |#||||||||||||||||||||||#| || || |#||||||||||||||||||||||#| ||
.# |#||||||||||||||||||||#| #. .# |#||||||||||||||||||||#| #. .# |#||||||||||||||||||||#| #.
|| |#||||||||||||||||||#| || || |#||||||||||||||||||#| || || |#||||||||||||||||||#| ||
#. .######|#|#########| .# #. .######|#|#########| .# #. .######|#|#########| .#
.#. |#####||||#####| .#. .#. |#####||||#####| .#. .#. |#####||||#####| .#.
|# ||########|| #. |# ||########|| #. |# ||########|| #.
|# ...... #| |# ...... #| |# ...... #|
|#. .#| |#. .#| |#. .#|
|#. .#. |#. .#. |#. .#.
.#| |#. .#| |#. .#| |#.
|#| |#| |#| |#| |#| |#|
.|#|. .|#|. .|#|. .|#|. .|#|. .|#|.
|##||. .||##| |##||. .||##| |##||. .||##|
.||########||. .||########||. .||########||.
Scale With Search 2026 [scalewithsearch.com](https://scalewithsearch.com)
```