Skip to main content
Rocketyard’s URL to Markdown API turns any public web page into clean Markdown for LLMs, RAG pipelines and AI agents and returns one clean record per page: markdown, text, title, description, language, author and more. It runs on the Apify platform, so you can start it from the Apify Console, the REST API, an AI agent or a no-code tool.

Run it on Apify

$0.75 per 1,000 pages, no subscription. You run it with your own Apify account, and Apify handles billing.

Who it’s for

  • Developers building RAG pipelines, chatbots and AI agents
  • AI agent and MCP users who need a “read this URL” tool
  • Data teams building knowledge bases from docs, help centers and blogs
  • SEO and content teams auditing page metadata

What you can do with it

  • Turn a list of URLs into clean Markdown for an LLM or a vector database
  • Crawl a documentation site into one Markdown dataset
  • Detect when a page changed and re-index it

How it works

The Actor fetches each page over plain HTTP without a browser, honours robots.txt and crawl-delay, finds the main content with Mozilla Readability and converts it to Markdown that keeps headings, lists, tables, code blocks and absolute links. Navigation, cookie banners, sidebars and footers are removed. Each record also carries the metadata and JSON-LD the page publishes, its heading outline, its links and images, a word count, a token estimate and a content hash for change detection. Give it a list of URLs, or a start page and let it crawl the same site.

Example output

A real record from a test run on nasa.gov on 2026-11-07. Fields the page didn’t publish are left out here; the Actor returns them as null. The same test also read en.wikipedia.org and developer.mozilla.org.

Fields

A field is filled when the source publishes it. The Actor never guesses a value: a missing field comes back as null. “Filled in latest test” is the share of records from the latest test run (nasa.gov, en.wikipedia.org and developer.mozilla.org) that had the field; other sources publish more or less.

Pricing

You pay $0.75 per 1,000 pages, plus a start fee of $0.00005 per run. A URL that fails returns a free error item saying why, instead of a charged empty record. Set maxItems or the run’s Maximum cost per run in Apify to cap spending.

API

Every Apify Actor is also a REST endpoint, so URL to Markdown API works as a URL to Markdown API, a web to Markdown API or a HTML to Markdown API. Call it from any language with the Apify REST API, or with Apify’s JavaScript and Python clients. You need an Apify API token.

URL to Markdown API request

The synchronous endpoint returns the records directly. For long crawls, start the run asynchronously and read the dataset when it finishes.

MCP

Apify’s MCP server exposes this Actor as a tool to Claude, Cursor and other MCP clients. Open this Actor’s MCP setup page for setup steps per client, or connect it with this URL:
Claude Code:
Cursor (.cursor/mcp.json) and other clients that read an mcpServers config:
On first use your browser opens so you can sign in to Apify. Then ask the agent, for example: “Turn a list of URLs into clean Markdown for an LLM or a vector database”. The tool returns JSON records, and the runs are billed to that Apify account at $0.75 per 1,000 pages.

Automation

n8n, Make and Zapier all connect to Apify. Each example below uses Actor rocketyard/web-page-to-markdown with this input:

n8n

Copy this workflow and paste it onto an n8n canvas. It runs the Actor every morning through an HTTP Request node and returns the records as n8n items.
Then open the HTTP Request node and create an HTTP Bearer Auth credential named Apify API token, with your token from Apify Console as the bearer token. The token stays in n8n’s credential store; the workflow only names the credential. Change the input in the node’s JSON body to your own. The node waits up to 300 seconds, which is also the limit of Apify’s synchronous endpoint. An input that yields nothing comes back as an item with an error field, which you can drop with a Filter node. Apify also publishes a community node, @apify/n8n-nodes-apify, with a Run an Actor and get dataset operation.

Make

  1. Add Apify’s Run an Actor module and connect your Apify account.
  2. Choose rocketyard/web-page-to-markdown, paste the input above and set Run synchronously to Yes. Make limits how long a synchronous run can take, depending on your plan. For longer runs, start the scenario from Watch Actor Runs instead.
  3. Add Get Dataset Items with the run’s default dataset ID.
  4. Send the items wherever you need them, for example with Google Sheets Add a Row, mapping markdown, text, title and description.

Zapier

  1. Add Apify’s Run Actor action, connect your Apify account, choose rocketyard/web-page-to-markdown and paste the input above.
  2. Zapier stops synchronous runs after 30 seconds. For anything larger than a small run, let the run continue in the background and start a second Zap from the Finished Actor run trigger.
  3. Add Fetch dataset items with the run’s dataset ID, then map markdown, text, title and description into your app.
  4. Test with a small input and check that the Zap handles every record before you turn it on.

Limits

  • The Actor reads the HTML the server sends and runs no JavaScript. A page built only in the browser returns a free EMPTY error item.
  • Pages behind a login or a paywall are not accessed. Bot protection is reported as a free BLOCKED error item, never bypassed.
  • Paths that robots.txt disallows are skipped and reported as ROBOTS_DISALLOWED.

FAQ

How much does it cost?

$0.75 per 1,000 pages plus $0.00005 per run, billed by Apify. 10,000 pages cost $7.50. There is no subscription, and you can cap any run with Maximum cost per run.

How do I call it from the API?

One request starts a run and returns its results, using your Apify API token:

What happens when a URL fails?

The dataset gets one free error item for it, with a code and the reason, and the other inputs in the run carry on. Error codes explains each code, whether a retry helps and what to do.

Which pages does it work on?

Any public HTML page whose text is in the HTML the server sends: articles, documentation, blogs, help centers, product and landing pages.

What happens when a field is missing?

The field comes back as null. The Actor reports what the source publishes and never fills in guesses.

Is it allowed to scrape this data?

The Actor reads public pages only, obeys robots.txt and crawl-delay, and doesn’t log in or bypass protections. You are responsible for how you use the content under each site’s terms and the law (for example copyright and GDPR).

Can I use it from an AI agent?

Yes. Connect Apify’s MCP server as shown in MCP, and the agent can call the Actor as a tool.

Alternatives

See how it compares with other web page scrapers on price, users and run success, from Apify Store data. Need a field or site behaviour that isn’t covered? Open an issue on the Actor page.