Article Scraper - Any Website (Schema.org)

Rocketyard’s Article Scraper extracts article data from any website that publishes schema.org Article markup and returns one clean record per article: headline, description, author, publisher, date published, date modified and more. It runs on the Apify platform, so you can start it from the Apify Console, the REST API, an AI agent or a no-code tool.

Run it on Apify

$1.40 per 1,000 results, no subscription. You run it with your own Apify account, and Apify handles billing.

Who it’s for

SEO specialists auditing content metadata
Media-monitoring and research teams
Developers feeding articles into RAG/LLM pipelines

What you can do with it

Collect headline/author/date metadata across many publishers
Audit structured data of a site for SEO
Build clean article datasets for analysis or AI

How it works

Websites publish schema.org Article markup (JSON-LD or microdata) so search engines can show rich results. The Actor fetches each page over plain HTTP without a browser, honours robots.txt and crawl-delay, reads that markup and maps it to the fields below. Because it reads the markup rather than the page layout, one Actor works across many sites and keeps working through redesigns. Give it a list of URLs, or a start page and let it crawl links on the same site.

Example output

A real record from a test run on en.wikipedia.org on 2026-11-06. Fields the page didn’t publish are left out here; the Actor returns them as null. The same test also read blog.cloudflare.com.

{
  "url": "https://en.wikipedia.org/wiki/Web_scraping",
  "type": "Article",
  "headline": "Web scraping",
  "description": "Web scraping, web harvesting, or web data extraction is data scraping used for extracting data from websites. Web scraping software may directly access the World Wide Web using the Hypertext Transfer Protocol (HTTP) or a web browser.",
  "author": "Contributors to Wikimedia projects",
  "publisher": "Wikimedia Foundation, Inc.",
  "datePublished": "2005-09-17T18:57:30Z",
  "dateModified": "2026-11-01T23:24:25Z",
  "language": "en",
  "text": "Web scraping, web harvesting, or web data extraction is data scraping used for extracting data from websites. Web scraping software may directly access the World Wide Web using the Hypertext Transfer Protocol (HTTP) or a web browser. While web scraping can be done manually by a software user, the…",
  "wordCount": 3179,
  "scrapedAt": "2026-11-06T07:02:27.936Z"
}

Fields

Field	Read from (schema.org)	Filled in latest test
`headline`	`headline`, `name`	100%
`description`	`description`, `abstract`	100%
`author`	`author.name`, `author`	50%
`publisher`	`publisher.name`, `publisher`	100%
`datePublished`	`datePublished`	100%
`dateModified`	`dateModified`	100%
`image`	`image.url`, `image`	50%
`section`	`articleSection`	0%
`keywords`	`keywords`	0%
`language`	`inLanguage`	100%
`text`	`articleBody`	100%
`wordCount`	`wordCount`	100%
`url`	the page the record came from	100%
`scrapedAt`	ISO timestamp of the extraction	100%

A field is filled when the source publishes it. The Actor never guesses a value: a missing field comes back as null. “Filled in latest test” is the share of records from the latest test run (en.wikipedia.org and blog.cloudflare.com) that had the field; other sources publish more or less.

Pricing

You pay $1.40 per 1,000 results, plus a start fee of $0.00005 per run. Pages without Article markup produce a free error item instead of a record.

Results	Cost
1,000	$1.40
10,000	$14.00
100,000	$140.00

Set maxItems or the run’s Maximum cost per run in Apify to cap spending.

API

Call it from any language with the Apify REST API, or with Apify’s JavaScript and Python clients. You need an Apify API token.

curl -s -X POST "https://api.apify.com/v2/acts/rocketyard~article-schema-scraper/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H 'content-type: application/json' \
  -d '{"startUrls":[{"url":"https://en.wikipedia.org/wiki/Web_scraping"}],"maxItems":100}'

import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('rocketyard/article-schema-scraper').call({"startUrls":[{"url":"https://en.wikipedia.org/wiki/Web_scraping"}],"maxItems":100});
const { items } = await client.dataset(run.defaultDatasetId).listItems();

from apify_client import ApifyClient
client = ApifyClient("<APIFY_TOKEN>")
run = client.actor("rocketyard/article-schema-scraper").call(run_input={"startUrls":[{"url":"https://en.wikipedia.org/wiki/Web_scraping"}],"maxItems":100})
items = client.dataset(run["defaultDatasetId"]).list_items().items

The synchronous endpoint returns the records directly. For long crawls, start the run asynchronously and read the dataset when it finishes.

MCP

Apify’s MCP server exposes this Actor as a tool to Claude, Cursor and other MCP clients. Open this Actor’s MCP setup page for setup steps per client, or connect it with this URL:

https://mcp.apify.com?tools=rocketyard/article-schema-scraper

Claude Code:

claude mcp add --transport http apify "https://mcp.apify.com?tools=rocketyard/article-schema-scraper"

Cursor (.cursor/mcp.json) and other clients that read an mcpServers config:

{
  "mcpServers": {
    "apify": { "url": "https://mcp.apify.com?tools=rocketyard/article-schema-scraper" }
  }
}

On first use your browser opens so you can sign in to Apify. Then ask the agent, for example: “Collect headline/author/date metadata across many publishers”. The tool returns JSON records, and the runs are billed to that Apify account at $1.40 per 1,000 results.

Automation

n8n, Make and Zapier all connect to Apify. Each example below uses Actor rocketyard/article-schema-scraper with this input:

{
  "startUrls": [
    {
      "url": "https://en.wikipedia.org/wiki/Web_scraping"
    }
  ],
  "maxItems": 100
}

n8n

Copy this workflow and paste it onto an n8n canvas. It runs the Actor every morning through an HTTP Request node and returns the records as n8n items.

{
  "name": "Article Scraper daily",
  "nodes": [
    {
      "parameters": {
        "rule": {
          "interval": [
            {
              "field": "days",
              "daysInterval": 1,
              "triggerAtHour": 7
            }
          ]
        }
      },
      "id": "c1f6a2d0-4a57-4b5e-9d7a-0a1b2c3d4e01",
      "name": "Every day at 07:00",
      "type": "n8n-nodes-base.scheduleTrigger",
      "typeVersion": 1.2,
      "position": [
        0,
        0
      ]
    },
    {
      "parameters": {
        "method": "POST",
        "url": "https://api.apify.com/v2/acts/rocketyard~article-schema-scraper/run-sync-get-dataset-items",
        "authentication": "genericCredentialType",
        "genericAuthType": "httpBearerAuth",
        "sendBody": true,
        "specifyBody": "json",
        "jsonBody": "{\n  \"startUrls\": [\n    {\n      \"url\": \"https://en.wikipedia.org/wiki/Web_scraping\"\n    }\n  ],\n  \"maxItems\": 100\n}",
        "options": {
          "timeout": 300000
        }
      },
      "id": "c1f6a2d0-4a57-4b5e-9d7a-0a1b2c3d4e02",
      "name": "Run Article Scraper",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.2,
      "position": [
        260,
        0
      ],
      "credentials": {
        "httpBearerAuth": {
          "name": "Apify API token"
        }
      }
    }
  ],
  "connections": {
    "Every day at 07:00": {
      "main": [
        [
          {
            "node": "Run Article Scraper",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  },
  "settings": {}
}

Then open the HTTP Request node and create an HTTP Bearer Auth credential named Apify API token, with your token from Apify Console as the bearer token. The token stays in n8n’s credential store; the workflow only names the credential. Change the input in the node’s JSON body to your own. The node waits up to 300 seconds, which is also the limit of Apify’s synchronous endpoint. An input that yields nothing comes back as an item with an error field, which you can drop with a Filter node. Apify also publishes a community node, @apify/n8n-nodes-apify, with a Run an Actor and get dataset operation.

Make

Add Apify’s Run an Actor module and connect your Apify account.
Choose rocketyard/article-schema-scraper, paste the input above and set Run synchronously to Yes. Make limits how long a synchronous run can take, depending on your plan. For longer runs, start the scenario from Watch Actor Runs instead.
Add Get Dataset Items with the run’s default dataset ID.
Send the items wherever you need them, for example with Google Sheets Add a Row, mapping headline, description, author and publisher.

Zapier

Add Apify’s Run Actor action, connect your Apify account, choose rocketyard/article-schema-scraper and paste the input above.
Zapier stops synchronous runs after 30 seconds. For anything larger than a small run, let the run continue in the background and start a second Zap from the Finished Actor run trigger.
Add Fetch dataset items with the run’s dataset ID, then map headline, description, author and publisher into your app.
Test with a small input and check that the Zap handles every record before you turn it on.

Limits

Pages without schema.org Article markup return no record.
Markup added only by JavaScript in the browser is not read: the Actor reads the HTML the server sends.
Pages behind a login or paywall are not accessed, and paths that robots.txt disallows are skipped.

FAQ

How much does it cost?

$1.40 per 1,000 results plus $0.00005 per run, billed by Apify. 10,000 results cost $14.00. There is no subscription, and you can cap any run with Maximum cost per run. Your Apify plan sets your discount tier (Platinum and Diamond are enterprise tiers):

Tier	Per 1,000 results
Free	$1.40
Bronze	$1.40
Silver	$1.12
Gold	$0.91
Platinum	$0.91
Diamond	$0.91

How do I call it from the API?

One request starts a run and returns its results, using your Apify API token:

curl -X POST "https://api.apify.com/v2/acts/rocketyard~article-schema-scraper/run-sync-get-dataset-items" -H "Authorization: Bearer $APIFY_TOKEN" -H "Content-Type: application/json" -d '{"startUrls":[{"url":"https://en.wikipedia.org/wiki/Web_scraping"}],"maxItems":100}'

What happens when a URL fails?

The dataset gets one free error item for it, with a code and the reason, and the other inputs in the run carry on. Error codes explains each code, whether a retry helps and what to do.

Which websites does it work on?

Any public page that publishes schema.org Article data in its HTML. Most modern sites add it for Google rich results.

What happens when a field is missing?

The field comes back as null. The Actor reports what the source publishes and never fills in guesses.

Is it allowed to scrape this data?

The Actor reads public pages only, obeys robots.txt and crawl-delay, doesn’t log in or bypass protections, and doesn’t collect people’s profiles, personal emails or personal phone numbers. You are responsible for how you use the data under each site’s terms and the law (for example copyright and GDPR).

Can I use it from an AI agent?

Yes. Connect Apify’s MCP server as shown in MCP, and the agent can call the Actor as a tool.

Alternatives

See how it compares with other article scrapers on price, users and run success, from Apify Store data. Need a field or site behaviour that isn’t covered? Open an issue on the Actor page.

Overview

Guides

Product Data API & Scraper - Prices & Stock