Netraptor

What is an OpenAPI spec, and how to generate one from your web app's traffic

Updated · 6 min read

An OpenAPI spec is a JSON or YAML document that describes an HTTP API in a standard, language-agnostic format, listing its paths, methods, parameters, request bodies and responses. If your API has no spec yet, you can generate a first draft from real traffic by recording the requests your web app makes in Chrome, exporting them as a HAR file, and converting that file with an open-source tool such as mitmproxy2swagger. You then review the draft by hand, because traffic only shows what was actually called.

Steps

  1. 1

    Record the app's API calls in Chrome

    Open DevTools (F12, or Cmd+Option+I on a Mac), go to the Network panel, check Preserve log, click the Fetch/XHR filter and reload the page.

  2. 2

    Use every feature you want in the spec

    Sign in with a test account and walk through each screen. Create, edit and delete test records so every endpoint and HTTP method shows up at least once.

  3. 3

    Export a HAR file

    Click Export HAR (sanitized) in the Network panel toolbar and save the file as app.har. The sanitized export leaves out Cookie, Set-Cookie and Authorization headers.

  4. 4

    Run a first pass with mitmproxy2swagger

    Install it with pip install mitmproxy2swagger, then run mitmproxy2swagger -i app.har -o openapi.yaml -p https://api.example.com. The output lists every discovered path under x-path-templates, each prefixed with ignore:.

  5. 5

    Pick the paths and run it again

    Remove the ignore: prefix from the paths you want, fix templates such as /users/{id}, and run the same command again. The second pass writes the operations into paths.

  6. 6

    Validate and edit the result

    Paste the file into Swagger Editor or run openapi-generator-cli validate -i openapi.yaml. Then add descriptions, required fields, error responses and auth by hand.

What is OpenAPI?

The OpenAPI Specification (OAS) defines a standard, programming-language-agnostic way to describe HTTP APIs. A tool or a person can read an OpenAPI document and learn which endpoints exist, what each one accepts and what it returns, without reading the server code.

OpenAPI started as the Swagger Specification. SmartBear donated it to the OpenAPI Initiative under the Linux Foundation in late 2015, and it was renamed in early 2016. Today "OpenAPI" is the name of the format, and "Swagger" usually refers to SmartBear's tools such as Swagger UI and Swagger Editor.

The published versions are 2.0, the 3.0.x line, the 3.1.x line and the 3.2.x line, with 3.2.1 the newest. Version 3.1 aligned its Schema Object with JSON Schema draft 2020-12, and most current tools read 3.0 and 3.1. If a tool you rely on rejects a newer version, check its docs before choosing.

What does an OpenAPI spec look like? (JSON and YAML)

An OpenAPI document is one JSON or YAML file, conventionally named openapi.json or openapi.yaml. It must have an openapi version field and an info object, plus at least one of paths, components or webhooks. Path parameters are written in curly braces, and every placeholder must match a parameter definition.

openapi: 3.1.0
info:
  title: Example app API
  version: 1.0.0
paths:
  /users/{id}:
    get:
      summary: Get a user
      parameters:
        - name: id
          in: path
          required: true
          schema:
            type: string
      responses:
        "200":
          description: The user
          content:
            application/json:
              schema:
                type: object
                properties:
                  id: { type: string }
                  email: { type: string }

The JSON form holds the same structure with braces and quotes. YAML is easier to edit by hand, JSON is easier to generate and parse, and most tools accept either.

What a spec generated from traffic will miss

Recorded traffic shows real requests and responses, which makes it a good starting point. It also has gaps you need to fill before anyone treats the file as documentation.

  • Endpoints you never triggered are missing. Admin screens, error paths and rarely used features only appear if you exercise them.
  • Schemas are inferred from examples. A field that was null in every response, or an optional field you never sent, will be typed loosely or left out.
  • Path parameters need checking. Generators guess which segments are IDs, so /users/42 and /users/me can end up merged or split the wrong way.
  • Auth is usually missing. Add a securitySchemes entry under components (for example a bearer token) and a security requirement yourself.
  • Enums, formats, limits and descriptions come from you, not the traffic.

Treat the generated file as a draft. One careful review pass usually turns it into something your team can rely on.

Is it safe to generate a spec from production traffic?

Be careful with what goes into the HAR file and the spec. Chrome's sanitized HAR export removes Cookie, Set-Cookie and Authorization headers, but request and response bodies stay in the file, and so do tokens passed in URLs or custom headers.

mitmproxy2swagger adds example data only when you pass --examples (and headers with --headers). Its README warns that this can include sensitive data. If you want examples in the spec, record against a test account with fake data, then read the examples before you commit or share the file.

What to do with an OpenAPI spec once you have one

  • Publish docs. Swagger UI and similar tools render the file as browsable reference pages.
  • Generate clients and server stubs. OpenAPI Generator takes -i (the spec), -g (a generator such as typescript-fetch) and -o (an output folder).
  • Test against it. The spec becomes a contract you can check the real API against in CI.
  • Import it into Postman. Postman imports OpenAPI 3.0 and 3.1 as a collection. See Create a Postman collection from a website's requests.

If you only need the list of endpoints and not a full spec, Find every API endpoint your web app calls covers the DevTools side in more detail.

Faster: generate the spec with Netraptor

Netraptor is a Chrome extension with its own DevTools panel. It shortens the record, convert and template steps above, and everything stays local in your browser.

  • Turn on "Keep history for this site" in the toolbar popup. Netraptor records fetch/XHR API traffic for that domain and its subdomains across page loads, even with DevTools closed.
  • Optionally run "Crawl this site" to visit same-origin links in your logged-in tab. It skips logout and delete-looking links. Free crawls cover 25 pages, Pro up to 1000.
  • Check the API Map tab. Endpoints are grouped and IDs are already parameterized, for example /users/{id}.
  • In the Export tab, export OpenAPI 3.1 as YAML or JSON, with optional response samples. "Strip auth headers and cookies" is on by default.

OpenAPI export is part of Pro ($29.99 one-time, no subscription). The Free plan exports HAR, Postman Collection v2.1 and curl, so you can export a HAR from Netraptor and run it through mitmproxy2swagger instead. You still review the generated spec by hand either way.

Frequently asked questions

Is OpenAPI the same as Swagger?
OpenAPI is the specification, and it was called the Swagger Specification until 2016. Swagger now refers to SmartBear's tools, such as Swagger UI and Swagger Editor, that work with OpenAPI files.
Should I use OpenAPI 3.0, 3.1 or 3.2?
Use 3.1 unless a tool in your pipeline only supports 3.0. Version 3.1 matches JSON Schema 2020-12, and 3.2 is newer, so check tool support before adopting it.
What is OpenAPI Generator?
OpenAPI Generator is an open-source CLI that turns an OpenAPI document into client SDKs, server stubs and docs for many languages. It also has a validate command for checking a spec.
Can I generate an OpenAPI spec from code instead of traffic?
Yes, if you own the backend. Many frameworks, FastAPI for example, produce a spec from route definitions, and that is usually more complete than one inferred from traffic.
What is the difference between openapi.json and openapi.yaml?
They hold the same document in two syntaxes. The spec allows both, and most tools accept either.