nextjs - 💡(How to fix) Fix Feature: Native `flyweb.ts` convention file for machine-readable content discovery [1 comments, 2 participants]

Why

Next.js already has convention files that let sites communicate with automated systems:

• robots.ts → tells crawlers where not to go
• sitemap.ts → tells crawlers what pages exist
• manifest.ts → tells browsers about the app

What's missing: a way to tell AI agents what structured data a site has and how to read it — without scraping HTML.

Today, AI agents (ChatGPT, Claude, Perplexity, etc.) scrape raw HTML, guess at content structure, and hallucinate when they get it wrong. There's no standard discovery mechanism. Every integration is bespoke.

What

FlyWeb is an open protocol that solves this. A site publishes /.well-known/flyweb.json describing its structured resources:

{
  "flyweb": "1.0",
  "entity": "My Blog",
  "type": "blog",
  "resources": {
    "posts": {
      "path": "/.flyweb/posts",
      "format": "jsonl",
      "fields": ["title", "author", "date", "content"],
      "query": "?tag={tag}"
    }
  }
}

AI agents check this file first — no scraping, no guessing. Just clean structured data.

Proposed API

Following the existing Next.js metadata file convention:

Static:

// app/flyweb.ts
import type { MetadataRoute } from 'next';

export default function flyweb(): MetadataRoute.FlyWeb {
  return {
    flyweb: '1.0',
    entity: 'My Blog',
    type: 'blog',
    resources: {
      posts: {
        path: '/.flyweb/posts',
        format: 'jsonl',
        fields: ['title', 'author', 'date', 'content'],
        query: '?tag={tag}',
      },
    },
  };
}

This would serve /.well-known/flyweb.json automatically — exactly like robots.ts serves /robots.txt.

Dynamic (with request context):

// app/flyweb.ts
import type { MetadataRoute } from 'next';

export default async function flyweb(): Promise<MetadataRoute.FlyWeb> {
  const resources = await discoverResources();
  return {
    flyweb: '1.0',
    entity: 'My Site',
    type: 'docs',
    resources,
  };
}

Why Next.js

• Pattern fit: FlyWeb follows the exact same convention-file pattern as robots.ts, sitemap.ts, and manifest.ts
• AI era: As AI agents become primary consumers of web content, sites need a machine-readable discovery layer. This is the robots.txt moment for structured data
• Ecosystem scale: If Next.js adds native support, every AI coding assistant (Cursor, Claude, v0, etc.) building Next.js apps would naturally include FlyWeb — the protocol spreads through the toolchain
• Zero friction: One file, no config, no packages to install

Existing Implementation

The protocol is already implemented and published:

• Spec: SPEC.md
• npm packages: flyweb (core types + CLI validator) and next-flyweb (Next.js plugin)
• CLI: npx flyweb check https://example.com — validates any site's FlyWeb support
• Demo: examples/next-blog — working Next.js app using the protocol
• Homepage: flyweb.io (itself FlyWeb-enabled)

The Three Layers

1. Discovery — /.well-known/flyweb.json describes what data a site has
2. Structure — Resources served as JSON/JSONL/CSV/XML, not HTML
3. Query — Standard URL params: GET /.flyweb/posts?tag=ai&limit=10

Summary

robots.txt told machines where not to go. flyweb.json tells them what you have.

Adding native flyweb.ts support would make Next.js the first framework to ship with built-in AI-agent content discovery — positioning it as the default choice for the structured web era.

Happy to contribute the implementation if there's interest.