Every recipe was compiled with next build on Next 16.3.5 (React 19.2) and checked with next start + curl. params is a Promise.
The canonical copies live in templates/next/. Paste these only into projects that didn't start from the template.
1. Facts file (single source of truth)
Template: templates/next/src/site.config.ts (with @/* → ./src/* in tsconfig). Minimal shape for other projects:
// src/site.config.ts: metadata, JSON-LD, sitemap, robots, llms.txt and OG images all read from here.
// "Only publish what exists": leave a field empty rather than invent it.
export const site = {
name: "Acme",
url: (process.env.NEXT_PUBLIC_SITE_URL ?? "https://acme.example").replace(/\/$/, ""), // production origin, no trailing slash
description: "Acme is an open-source toolkit that makes TypeScript builds 10x faster. MIT licensed.", // ≤ 160 chars
tagline: "Acme: fast builds for TypeScript teams", // home page <title>
locale: "en_US",
lang: "en",
organization: { name: "Acme", logo: "/icon-512.png" }, // logo ≥ 112×112 PNG
socials: [{ label: "GitHub", href: "https://github.com/acme" }], // only profiles that exist → JSON-LD sameAs
ai: { retrieval: true, training: false }, // robots.txt policy (see ai-crawlers.md)
privatePaths: ["/api/"], // never crawled, not in sitemap or llms.txt
} as const;Keep a page registry (template: src/content/pages.ts) with path, title, description and lastModified (the real content date) for every public page. Sitemap, llms.txt and Markdown mirrors are generated from it.
2. Root layout and the page helper
// src/app/layout.tsx
import type { Metadata, Viewport } from "next";
import { site } from "@/site.config";
export const metadata: Metadata = {
metadataBase: new URL(site.url), // relative canonical/OG URLs resolve against this
title: { default: site.tagline, template: `%s | ${site.name}` },
description: site.description,
applicationName: site.name,
openGraph: { type: "website", siteName: site.name, locale: site.locale, url: "/" },
twitter: { card: "summary_large_image" },
robots: { index: true, follow: true, googleBot: { index: true, follow: true, "max-image-preview": "large", "max-snippet": -1, "max-video-preview": -1 } },
// NO alternates.canonical here: a root canonical makes every page without its own canonicalize to the home page
};
export const viewport: Viewport = { // theme-color lives in viewport, not metadata
themeColor: [
{ media: "(prefers-color-scheme: light)", color: "#ffffff" },
{ media: "(prefers-color-scheme: dark)", color: "#0a0a0a" },
],
colorScheme: "light dark",
};
export default function RootLayout({ children }: { children: React.ReactNode }) {
return (
<html lang={site.lang} suppressHydrationWarning>
<head>
<link rel="describedby" type="text/markdown" href="/llms.txt" /> {/* llms.txt v2 discovery */}
</head>
<body>{children}</body>
</html>
);
}// src/lib/seo.ts: every page's metadata goes through this (metadata merge is shallow).
// Same API as templates/next/src/lib/seo.ts, except `markdown` defaults to false here.
import type { Metadata } from "next";
import { site } from "@/site.config";
type PageMetadataInput = {
path: `/${string}`; // canonical path: leading slash, no trailing slash
title: string; // "/" uses it as the absolute title; other pages get "title | Brand"
description: string;
image?: { url: string; width?: number; height?: number; alt?: string }; // omit when an opengraph-image file covers the route
type?: "website" | "article";
publishedTime?: string; // ISO 8601 with timezone
modifiedTime?: string;
markdown?: boolean; // advertise the .md mirror
noindex?: boolean;
};
const markdownPath = (path: string) => (path === "/" ? "/index.md" : `${path}.md`);
export function pageMetadata(i: PageMetadataInput): Metadata {
const images = i.image ? [i.image] : undefined;
const type = i.type ?? "website";
return {
title: i.path === "/" ? { absolute: i.title } : i.title,
description: i.description,
alternates: {
canonical: i.path,
...(i.markdown ? { types: { "text/markdown": markdownPath(i.path) } } : {}),
},
openGraph: {
type, url: i.path, title: i.title, description: i.description,
siteName: site.name, locale: site.locale, // re-added: a page-level openGraph replaces the layout's
...(images ? { images } : {}),
...(type === "article" ? { publishedTime: i.publishedTime, modifiedTime: i.modifiedTime } : {}),
},
twitter: { card: "summary_large_image", title: i.title, description: i.description, ...(images ? { images } : {}) },
...(i.noindex ? { robots: { index: false, follow: true } } : {}),
};
}
export const absoluteUrl = (path: string) => (path.startsWith("http") ? path : `${site.url}${path === "/" ? "/" : path}`);// src/app/page.tsx (home)
export const metadata = pageMetadata({ path: "/", title: site.tagline, description: site.description });
// src/app/pricing/page.tsx
export const metadata = pageMetadata({ path: "/pricing", title: "Pricing", description: "Acme is free for open source. Team plan is $12 per seat per month, billed annually." });
// src/app/login/page.tsx
export const metadata = pageMetadata({ path: "/login", title: "Log in", description: "Log in to Acme.", noindex: true });Verified output for a page: . The home page with canonical: "/" emits https://acme.example (no trailing slash).
Rules:
metadataandgenerateMetadataonly work in Server Components (page/layout). You can't export both from one file.metadataBasemissing means a build error for relative OG URLs.- If
opengraph-image.tsxexists for a route segment, it overridesopenGraph.images. Don't passimagethen. - A section
layout.tsxwith a plain stringtitleresets the template for its children. Set titles per page via the helper.
3. Request-time routes: deliver to bots (htmlLimitedBots)
Only needed when generateMetadata or the page is dynamic (uses cookies(), headers(), connection(), searchParams, or uncached data). Prerendered routes already have metadata in for every UA.
Verified on 16.3.5 for a dynamic route with Suspense:
| UA | title/canonical in raw |
Suspense content in order |
|---|---|---|
| Browser | no (streamed into ) |
no ( |
| GPTBot, ClaudeBot, Claude-User | no | no |
| Googlebot | no, still in after rendering |
no |
| Bingbot (in default list) | yes | yes |
// next.config.ts
import type { NextConfig } from "next";
// Copy of Next 16.3.5's default list. Setting htmlLimitedBots REPLACES the default, so keep it.
// Re-diff on every Next upgrade: packages/next/src/shared/lib/router/utils/html-bots.ts
const NEXT_DEFAULT_HTML_BOTS =
"[\\w-]+-Google|Google-[\\w-]+|Chrome-Lighthouse|Slurp|DuckDuckBot|baiduspider|yandex|sogou|bitlybot|tumblr|vkShare|quora link preview|redditbot|ia_archiver|Bingbot|BingPreview|applebot|facebookexternalhit|facebookcatalog|Twitterbot|LinkedInBot|Slackbot|Discordbot|WhatsApp|SkypeUriPreview|Yeti|googleweblight";
const EXTRA_BOTS = [
"Googlebot",
"GPTBot", "OAI-SearchBot", "ChatGPT-User",
"ClaudeBot", "Claude-SearchBot", "Claude-User", "Claude-Code",
"PerplexityBot", "Perplexity-User",
"meta-externalagent", "meta-externalfetcher", "meta-webindexer",
"Amazonbot", "Amzn-SearchBot", "Amzn-User",
"DuckAssistBot", "MistralAI-User", "MistralAI-Index", "CCBot", "Bytespider",
].join("|");
const nextConfig: NextConfig = {
htmlLimitedBots: new RegExp(`${NEXT_DEFAULT_HTML_BOTS}|${EXTRA_BOTS}`, "i"),
};
export default nextConfig;The cost is that those bots wait for the full HTML (blocking TTFB). Browsers keep streaming. Never set /.*/.
Check it: curl -sA "Mozilla/5.0 (compatible; GPTBot/1.4; +https://openai.com/gptbot)" URL | awk '/<\/head>/{exit} /rel="canonical"/{print "canonical in head"}'.
4. Dynamic content page (blog post) with correct status codes
// src/app/blog/[slug]/page.tsx
import type { Metadata } from "next";
import { notFound } from "next/navigation";
import { JsonLd } from "@/components/json-ld";
import { pageMetadata } from "@/lib/seo";
import { getPost, posts } from "@/lib/content";
import { site } from "@/site.config";
type Props = { params: Promise<{ slug: string }> };
export function generateStaticParams() {
return posts.map((p) => ({ slug: p.slug }));
}
export const dynamicParams = false; // unknown slugs → real 404
export async function generateMetadata({ params }: Props): Promise<Metadata> {
const { slug } = await params;
const post = getPost(slug);
if (!post) return {};
return pageMetadata({
path: `/blog/${post.slug}`, title: post.title, description: post.description,
type: "article", publishedTime: post.published, modifiedTime: post.updated, markdown: true,
});
}
export default async function Post({ params }: Props) {
const { slug } = await params;
const post = getPost(slug);
if (!post) notFound(); // BEFORE any <Suspense>: inside one, the status is 200
const url = `${site.url}/blog/${post.slug}`;
return (
<article>
<JsonLd data={{
"@context": "https://schema.org",
"@graph": [
{ "@type": "BlogPosting", "@id": `${url}#article`, headline: post.title, description: post.description,
datePublished: post.published, dateModified: post.updated, mainEntityOfPage: url,
author: { "@type": "Person", name: post.author.name, url: `${site.url}/authors/${post.author.slug}` },
publisher: { "@id": `${site.url}/#organization` }, image: [`${url}/opengraph-image`] },
{ "@type": "BreadcrumbList", itemListElement: [
{ "@type": "ListItem", position: 1, name: "Blog", item: `${site.url}/blog` },
{ "@type": "ListItem", position: 2, name: post.title } ] },
],
}} />
<h1>{post.title}</h1>
<p>By <a href={`/authors/${post.author.slug}`}>{post.author.name}</a> · <time dateTime={post.updated}>Updated {post.updated.slice(0, 10)}</time></p>
{/* body: server-rendered; secondary widgets (comments, related) may sit in <Suspense> */}
</article>
);
}| Pattern (verified) | HTTP status |
|---|---|
notFound() in generateMetadata or top of page, before Suspense |
404 |
notFound() inside a component under |
200 + noindex meta (soft 404) |
dynamicParams = false + unknown slug |
404 |
Redirects: redirect() / permanentRedirect() (308) at the top of the page, or redirects() in next.config.ts with permanent: true for moved content. Never a client router.push.
5. Open Graph images
// src/app/blog/[slug]/opengraph-image.tsx (per post). The site default src/app/opengraph-image.tsx (see the template) has no params.
import { ImageResponse } from "next/og";
import { getPost, posts } from "@/lib/content";
import { site } from "@/site.config";
export const alt = `${site.name} blog post`;
export const size = { width: 1200, height: 630 };
export const contentType = "image/png";
export function generateStaticParams() {
return posts.map((p) => ({ slug: p.slug })); // prerender at build
}
export default async function Image({ params }: { params: Promise<{ slug: string }> }) {
const { slug } = await params;
const post = getPost(slug);
return new ImageResponse(
(
<div style={{ width: "100%", height: "100%", display: "flex", flexDirection: "column", justifyContent: "flex-end",
padding: 80, background: "#0a0a0a", color: "#fafafa" }}>
<div style={{ display: "flex", fontSize: 28, color: "#a3a3a3" }}>{site.name} · Blog</div>
<div style={{ display: "flex", fontSize: 72, lineHeight: 1.1, marginTop: 16 }}>{post?.title ?? site.name}</div>
</div>
),
size,
);
}Verified: 200 image/png, 1200×630, ~31 KB, URL gets a content hash (?38d3918f…) so social caches refresh.
Constraints: flexbox and a CSS subset only (no grid), every multi-child element needs display: "flex", fonts must be ttf/otf/woff read at build time, 500 KB bundle max. Colors come from hex mirrors of your tokens, since Satori doesn't read CSS variables or oklch().
6. Sitemaps
// src/app/sitemap.ts
import type { MetadataRoute } from "next";
import { posts } from "@/lib/content";
import { site } from "@/site.config";
export default function sitemap(): MetadataRoute.Sitemap {
return [
{ url: `${site.url}/`, lastModified: "2026-09-01" }, // real content date, NOT new Date()
{ url: `${site.url}/pricing`, lastModified: "2026-08-12" }, // template: generated from src/content/pages.ts
...posts.filter((p) => !p.draft && !p.noindex).map((p) => ({ url: `${site.url}/blog/${p.slug}`, lastModified: p.updated })),
]; // omit changeFrequency and priority: Google ignores them
}lastModifiedcomes from frontmatterupdated ?? dateor hand-set constants. Git dates break in shallow CI clones and Docker builds without.git.- Exclude drafts, noindex pages,
/api,/account,/login, search and filter URLs. - Each sitemap ≤ 50,000 URLs and ≤ 50 MB. Beyond that, split:
// src/app/products/sitemap.ts → /products/sitemap/0.xml, /1.xml …
import type { MetadataRoute } from "next";
import { site } from "@/site.config";
const PER_FILE = 50_000;
export async function generateSitemaps() {
const total = await countProducts();
return Array.from({ length: Math.ceil(total / PER_FILE) }, (_, id) => ({ id }));
}
export default async function sitemap({ id }: { id: Promise<string> }): Promise<MetadataRoute.Sitemap> {
const n = Number(await id); // Promise since 16.0
const rows = await listProducts({ offset: n * PER_FILE, limit: PER_FILE });
return rows.map((r) => ({ url: `${site.url}/products/${r.slug}`, lastModified: r.updatedAt }));
}// src/app/sitemap-index.xml/route.ts: Next doesn't generate an index for generateSitemaps
import { generateSitemaps } from "@/app/products/sitemap";
import { site } from "@/site.config";
export const dynamic = "force-static";
export async function GET() {
const ids = await generateSitemaps();
const locs = [`${site.url}/sitemap.xml`, ...ids.map(({ id }) => `${site.url}/products/sitemap/${id}.xml`)];
const xml = `<?xml version="1.0" encoding="UTF-8"?>\n<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">\n${locs.map((l) => ` <sitemap><loc>${l}</loc></sitemap>`).join("\n")}\n</sitemapindex>\n`;
return new Response(xml, { headers: { "Content-Type": "application/xml; charset=utf-8" } });
}Point robots sitemap at /sitemap-index.xml (or list every sitemap).
7. Icons and manifest
app/favicon.ico → <link rel="icon" href="/favicon.ico" sizes="…"> required: Google doesn't list SVG favicons
app/icon.png → <link rel="icon" type="image/png"> square, ≥ 48×48 (use 512×512)
app/icon.svg → optional extra, never the only icon
app/apple-icon.png → <link rel="apple-touch-icon" sizes="180x180"> opaque PNG
app/manifest.ts → <link rel="manifest" href="/manifest.webmanifest">// src/app/manifest.ts
import type { MetadataRoute } from "next";
import { site } from "@/site.config";
export default function manifest(): MetadataRoute.Manifest {
return {
name: site.name, short_name: site.name, description: site.description,
start_url: "/", display: "standalone", background_color: "#ffffff", theme_color: "#0a0a0a",
icons: [
{ src: "/icon-192.png", sizes: "192x192", type: "image/png" },
{ src: "/icon-512.png", sizes: "512x512", type: "image/png" },
{ src: "/icon-maskable-512.png", sizes: "512x512", type: "image/png", purpose: "maskable" },
],
};
}8. URLs, previews and headers
- Next's default:
/about/→ 308 →/about. Keep it. If you settrailingSlash: true, canonicals, sitemap and links must all use the slash. - http→https and www↔apex: one 301/308 hop at the host or CDN.
- Preview deployments send
X-Robots-Tag: noindexor robotsDisallow: /(seeai-crawlers.md), keyed onVERCEL_ENV !== "production"or your platform's equivalent.
// next.config.ts: noindex non-production at the header level too
async headers() {
return process.env.VERCEL_ENV && process.env.VERCEL_ENV !== "production"
? [{ source: "/:path*", headers: [{ key: "X-Robots-Tag", value: "noindex" }] }]
: [];
},- Pagination: each
?page=nhas its own self-canonical (not page 1), linked with. Filter and sort variants arenoindexand left out of the sitemap. - hreflang (only for real translations):
alternates.languages: { "en-US": "/en", "de-DE": "/de", "x-default": "/" }on every variant, reciprocal, and each variant self-canonical.
9. IndexNow (Bing, Yandex, Seznam, Naver) on publish
// scripts/indexnow.ts: run in CI after deploy with changed URLs
const KEY = process.env.INDEXNOW_KEY!; // 8–128 chars [a-zA-Z0-9-]; also commit public/<KEY>.txt containing the key
const HOST = new URL(process.env.NEXT_PUBLIC_SITE_URL!).host;
export async function submit(urls: string[]) {
for (let i = 0; i < urls.length; i += 10_000) {
const res = await fetch("https://api.indexnow.org/indexnow", {
method: "POST",
headers: { "Content-Type": "application/json; charset=utf-8" },
body: JSON.stringify({ host: HOST, key: KEY, keyLocation: `https://${HOST}/${KEY}.txt`, urlList: urls.slice(i, i + 10_000) }),
});
if (![200, 202].includes(res.status)) throw new Error(`IndexNow ${res.status}`); // 403 key invalid · 422 host mismatch · 429 slow down
}
}Google: submit the sitemap in Search Console once (the ping endpoint is deprecated). Bing ties AI-answer freshness to IndexNow.
Sources
- https://nextjs.org/docs/app/api-reference/functions/generate-metadata
- https://nextjs.org/docs/app/api-reference/config/next-config-js/htmlLimitedBots
- https://github.com/vercel/next.js/blob/canary/packages/next/src/shared/lib/router/utils/html-bots.ts
- https://nextjs.org/docs/app/api-reference/file-conventions/metadata/opengraph-image · …/sitemap · …/robots · …/app-icons · …/manifest
- https://nextjs.org/docs/app/api-reference/functions/generate-sitemaps · https://nextjs.org/docs/app/api-reference/functions/image-response
- https://developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls
- https://developers.google.com/search/docs/appearance/title-link · https://developers.google.com/search/docs/appearance/favicon-in-search
- https://developers.google.com/search/docs/crawling-indexing/googlebot (2 MB limit)
- https://www.indexnow.org/documentation