MachinepulseBot

Machinepulse is an open observatory of the AI agent web.

What it collects

Only public pages and public APIs, and only aggregate statistics and public metadata.

• MCP Registry (registry.modelcontextprotocol.io) — daily snapshot of public server listings (metadata only: names, descriptions, repository links, package identifiers).
• GitHub Trending (python, typescript, javascript) — one page per language per day.
• npm and PyPI download counts for a fixed list of public packages, over historical date ranges.
• GitHub REST API — daily counters (stars, forks, open issues, last push) for a fixed list of public repositories.
• Protocol release feeds (MCP, A2A, AP2, x402) via public GitHub Releases APIs.
• A2A Registry (api.a2a-registry.org) — public agent directory listings.
• Hugging Face model metadata (downloads, likes, parameter counts) for a fixed list of public models and their derived variants.
• On-chain x402 payment volume via the public DefiLlama API.

A third-party MCP catalog (Glama) is ingested for internal analytics only and is never republished in bulk or as a dataset.

How it identifies itself

Every request uses the user agent MachinepulseBot/0.1 (+https://machinepulse.tech/bot; contact@machinepulse.tech).

Frequency

Each source is collected at most once per day, at night (server time, Europe/Amsterdam); if a nightly run was missed — for example, after downtime — it is caught up at the next opportunity. A few critical sources also get minimal availability probes during the day (a single GET to one probe URL). One run is a small fixed set of calls to the listed APIs; the only larger volume is backfilling npm/PyPI historical ranges, which may mean a few hundred requests in one day. No other requests are made.

What it never does

• No scraping of sources whose terms forbid it.
• Nothing personal is taken from your pages: no emails, no user identifiers.
• No requests beyond the documented sources and rates.

robots.txt

The bot does not crawl: it fetches only the fixed, documented pages listed above, and robots.txt is not parsed automatically. API sources are outside robots.txt scope in any case.

How to opt out

• Email contact@machinepulse.tech — the source is dropped from the list.
• Or block the user agent MachinepulseBot/0.1 at your edge.
• Or ask us to honour a robots.txt Disallow for your pages — we treat it as an opt-out and apply it manually.