Generating Dynamic XML Sitemaps for SPAs #

A single-page app rarely exposes its full URL set to a crawler in the initial HTML, so the sitemap is the authoritative inventory of what exists — and it has to be generated from the same routes and data the app renders. This guide builds that sitemap two ways and sits within sitemaps and indexing control for SPAs.

Problem Statement #

In a client-rendered app the router constructs navigation after the bundle executes, so the raw response a crawler first fetches contains almost no internal links. Discovery therefore leans heavily on the sitemap, and a sitemap that is hand-maintained, stale, or built from a hardcoded list drifts out of sync with the real route set the moment content changes. The two symptoms are mirror images: new URLs never get discovered because they are missing from the file, and removed URLs keep getting crawled because they linger in it. The fix is to derive the sitemap from a single source of truth — the route manifest plus the data that drives dynamic segments — so it regenerates whenever either changes.

The generation strategy forks on one question: is the URL set known at deploy time, or does it depend on data that changes between deploys?

Step-by-step fix #

1. Centralize the route inventory. Export a single function that returns every indexable URL with its real last-changed date. It reads the static route manifest for fixed pages and the data layer for dynamic segments, so there is one place that decides what belongs in the sitemap.

   // sitemap-source.ts
   export interface SitemapEntry {
     loc: string;          // absolute, canonical URL
     lastmod: string;      // ISO 8601 date of last content change
     changefreq?: 'daily' | 'weekly' | 'monthly';
     priority?: number;    // 0.0–1.0
   }
   
   const ORIGIN = 'https://example.com';
   const staticRoutes = ['/', '/about', '/pricing'];
   
   export async function collectUrls(db: DataSource): Promise<SitemapEntry[]> {
     const staticEntries: SitemapEntry[] = staticRoutes.map((path) => ({
       loc: `${ORIGIN}${path}`,
       lastmod: new Date().toISOString().slice(0, 10),
       changefreq: 'monthly',
     }));
   
     const articles = await db.getPublishedArticles(); // [{ slug, updatedAt }]
     const dynamicEntries: SitemapEntry[] = articles.map((a) => ({
       loc: `${ORIGIN}/blog/${a.slug}`,
       lastmod: new Date(a.updatedAt).toISOString().slice(0, 10),
       changefreq: 'weekly',
       priority: 0.8,
     }));
   
     return [...staticEntries, ...dynamicEntries];
   }

2. Serialize entries to valid XML. Escape URL values, emit one urlset per shard, and cap each shard at the per-file limit. Keeping serialization pure makes it reusable for both the build-time writer and the API endpoint.

   // serialize-sitemap.ts
   import type { SitemapEntry } from './sitemap-source';
   
   const MAX_PER_FILE = 50000;
   
   function xmlEscape(value: string): string {
     return value
       .replace(/&/g, '&')
       .replace(/</g, '&lt;')
       .replace(/>/g, '&gt;')
       .replace(/"/g, '&quot;')
       .replace(/'/g, '&apos;');
   }
   
   export function renderUrlset(entries: SitemapEntry[]): string {
     const urls = entries
       .map((e) => {
         const parts = [`    <loc>${xmlEscape(e.loc)}</loc>`, `    <lastmod>${e.lastmod}</lastmod>`];
         if (e.changefreq) parts.push(`    <changefreq>${e.changefreq}</changefreq>`);
         if (e.priority != null) parts.push(`    <priority>${e.priority.toFixed(1)}</priority>`);
         return `  <url>\n${parts.join('\n')}\n  </url>`;
       })
       .join('\n');
     return `<?xml version="1.0" encoding="UTF-8"?>\n` +
       `<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">\n${urls}\n</urlset>\n`;
   }
   
   export function chunk(entries: SitemapEntry[]): SitemapEntry[][] {
     const out: SitemapEntry[][] = [];
     for (let i = 0; i < entries.length; i += MAX_PER_FILE) {
       out.push(entries.slice(i, i + MAX_PER_FILE));
     }
     return out;
   }

3. Split large sets behind a sitemap index. When there is more than one shard, write each shard and a sitemapindex that references them. A crawler reads the index and fetches each listed sitemap in turn.

   // render-index.ts
   const ORIGIN = 'https://example.com';
   
   export function renderSitemapIndex(shardCount: number, lastmod: string): string {
     const items = Array.from({ length: shardCount }, (_, i) =>
       `  <sitemap>\n` +
       `    <loc>${ORIGIN}/sitemaps/sitemap-${i + 1}.xml</loc>\n` +
       `    <lastmod>${lastmod}</lastmod>\n` +
       `  </sitemap>`
     ).join('\n');
     return `<?xml version="1.0" encoding="UTF-8"?>\n` +
       `<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">\n${items}\n</sitemapindex>\n`;
   }

4. Choose a delivery path. For a fixed route set, write the files during the build so they deploy as static assets. For data that changes between deploys, serve the same serialized output from an API endpoint with a sensible cache header.

   // Build-time writer (Node build script)
   import { writeFile, mkdir } from 'node:fs/promises';
   import { collectUrls } from './sitemap-source';
   import { renderUrlset, chunk } from './serialize-sitemap';
   import { renderSitemapIndex } from './render-index';
   
   async function build(db: DataSource) {
     const shards = chunk(await collectUrls(db));
     await mkdir('dist/sitemaps', { recursive: true });
     const today = new Date().toISOString().slice(0, 10);
     await Promise.all(
       shards.map((s, i) => writeFile(`dist/sitemaps/sitemap-${i + 1}.xml`, renderUrlset(s)))
     );
     await writeFile('dist/sitemap.xml', renderSitemapIndex(shards.length, today));
   }

   // Request-time endpoint (framework-agnostic handler)
   import { collectUrls } from './sitemap-source';
   import { renderUrlset } from './serialize-sitemap';
   
   export async function sitemapHandler(db: DataSource): Promise<Response> {
     const xml = renderUrlset(await collectUrls(db));
     return new Response(xml, {
       headers: {
         'Content-Type': 'application/xml; charset=utf-8',
         'Cache-Control': 'public, max-age=3600',
       },
     });
   }

5. Reference the sitemap from robots.txt so engines discover it without manual submission, and list only the index (or the single file) there.

   User-agent: *
   Allow: /
   Sitemap: https://example.com/sitemap.xml

Validation #

Confirm the sitemap is well-formed, complete, and served with the right content type before relying on it:

• Fetch and parse it as a crawler would. curl -s https://example.com/sitemap.xml and check the response is XML, not the app shell — a common SPA bug is client routing intercepting the path and returning index.html.

• Assert the URL count and canonical form in CI. A quick check catches empty or truncated output.

  // sitemap.test.ts
  import { collectUrls } from './sitemap-source';
  
  test('every sitemap URL is absolute, canonical, and non-duplicated', async () => {
    const entries = await collectUrls(testDb);
    const locs = entries.map((e) => e.loc);
    console.assert(locs.length > 0, 'sitemap is empty');
    console.assert(locs.every((l) => l.startsWith('https://')), 'relative URL present');
    console.assert(new Set(locs).size === locs.length, 'duplicate URLs present');
  });

• Submit the sitemap in Search Console and watch the Sitemaps report for the discovered-URL count and any parse errors.

• Spot-check lastmod against a page you know changed recently; the date should match the content change, not the deploy.

Common Pitfalls #

• Build timestamp as lastmod on every URL. Identical dates across the whole file carry no signal and lead crawlers to distrust the field. Derive it from each record’s real update date.
• Listing non-canonical variants. Including parameterized or trailing-slash duplicates invites the exact duplicate-content problem the sitemap should help avoid — list only canonical URLs, in line with canonical URL management.
• Client router swallowing the request. If the SPA’s catch-all route handles /sitemap.xml, crawlers get HTML. Serve the sitemap ahead of the app shell at the host or edge.
• Including noindex URLs. A URL that carries a noindex directive should not be in the sitemap; the two signals contradict each other. Keep the inventory to indexable pages, as covered in controlling indexing with robots meta.
• Forgetting the size ceiling. A file over 50,000 URLs or 50 MB is rejected wholesale; split before you reach it rather than after crawlers start erroring.

Frequently Asked Questions #

Should I generate a SPA sitemap at build time or from an API endpoint? #

Generate it at build time when the full route set is known when you deploy, because a static file is fast and cannot fail at request time. Use an API endpoint when URLs depend on data that changes between deploys, such as a product catalog, so the sitemap always reflects the current database without a rebuild.

What should lastmod contain for a client-rendered route? #

Use the real date the page content last changed, taken from the same data record the route renders, not the build timestamp. A build time applied to every URL tells crawlers nothing about what actually changed and can cause them to ignore lastmod entirely.

How large can a single XML sitemap be? #

One sitemap file may hold up to 50,000 URLs and must stay under 50 MB uncompressed. Beyond that, split the URLs across multiple sitemap files and list those files in a sitemap index, which itself can reference up to 50,000 sitemaps.

Related #

• Sitemaps & Indexing Control for SPAs — how discovery fits alongside indexing permission and notification.
• Submitting SPA URLs with the IndexNow API — push individual changed URLs between full sitemap regenerations.
• Canonical URL Management in SPAs — keep the sitemap limited to canonical URLs.

← Back to Sitemaps & Indexing Control for SPAs