Reading JSON APIs through an egress address

The shape of the work

One POST per target URL. The envelope's result field holds the response body verbatim as text, which you parse with whatever your language already uses. content_type tells you whether you genuinely received JSON or an HTML error page wearing a 200, which is worth branching on before you hand the string to a parser.

status is the target's own status code, passed through rather than translated. A 404 or a 429 from the endpoint arrives as a 404 or a 429 with its body intact, so your backoff logic reads the target's answer rather than ours.

Paging is your loop

There is no cursor management and no pagination rule engine. You read the page, build the next URL, and call again. That is a handful of lines in any language and it keeps the part most likely to be specific to one API in the place where you can see it.

Concurrency and backoff are yours as well. Rate limits are enforced per account rather than per key, so minting extra keys to run more workers does not raise the ceiling — keys are for separating environments and revoking one without touching the rest.

GET only, and what that rules out

The outbound request is a GET with a fixed header set plus the User-Agent you supply. There is no request body, no Authorization header and no cookie jar, so an endpoint requiring a signed header, an OAuth flow or a POST cannot be fetched today.

An endpoint whose key travels in the query string will work, though putting a credential in a URL is a poor idea for reasons that have nothing to do with us. What this serves well is the large set of public, GET-addressable JSON: open data portals, public catalogue and listing endpoints, feeds, and the JSON a site's own front end calls.

Retries without double billing

Long collection runs time out somewhere. Send an Idempotency-Key header with each fetch and a retry after a timeout is neither re-executed against the target nor billed twice; the replay carries Idempotency-Replayed: true. Reuse of a key for a different URL is refused with a stable error code rather than quietly returning the wrong body.

Three platforms with a unified read

Alongside the proxy there is a social data API in beta, unpriced and bundled with the proxy plan. It covers only platforms whose public APIs verifiably serve requests from our addresses: Bluesky profiles and posts, Mastodon accounts, hashtags and trends across five allowlisted instances, and Hacker News frontpage, item and search.

Everything comes back in one envelope with success, platform, endpoint, request_id, cached and data. GET /social/platforms lists what serves and what is excluded with the reason — Bluesky search is excluded because it is blocked upstream for us, and Mastodon takes only the allowlisted instances rather than arbitrary hostnames, which keeps it from becoming an SSRF surface.

Questions

Do you parse the JSON for me?

No. You get the response body as text and parse it yourself. There is no schema mapping, no field extraction and no output format conversion to CSV or XML.

Can I fetch an endpoint that requires an API key?

Only if the key can travel in the URL. We forward no Authorization header and no cookies, so header-authenticated and OAuth-protected endpoints cannot be reached through the fetch API.

Will splitting work across several API keys raise my limit?

No. Rate limits are per account. Keys exist so you can scope and revoke them individually, not to multiply throughput.

Is there a Python or JavaScript client library?

Typed clients exist in our repository but they are not published to PyPI or npm, so there is nothing to pip install or npm install today. What you can use right now is copy-paste snippets, a generated Postman collection and an MCP server, all listed in the dashboard's integrations page.

Related