WhatsMCP-Knowledge

What it is

WhatsMCP lets a business answer its WhatsApp chats with an AI agent. The agent answers from a knowledge base its owner fills, and a web page can be one of its sources: usually a page of the owner's own site, such as their prices, opening hours or questions and answers.

To read such a page, we send a request that names itself WhatsMCP-Knowledge/1.0 (+https://whatsmcp.com/bot). We never send a browser's name.

What it reads

• Only an address a customer typed in. It doesn't follow links, crawl a site or read a sitemap.
• One page for each address, after your robots.txt.
• The page as your server sends it. It runs no JavaScript, sends no cookies, doesn't log in and submits nothing. It reads https only, on port 443, and follows at most five redirects.

The text it reads goes into that customer's knowledge base, for their own agent. It isn't shared with other customers or published anywhere.

When it visits

Only when a customer adds your page, or asks for it to be read again. Nothing visits on a schedule.

If your server is slow or busy, or answers with an error (a timeout, a 5xx or a 429), it tries again a few times over about six minutes, then stops. One customer can ask for at most 30 pages an hour.

How to block it

Add a group for it to your robots.txt:

User-agent: WhatsMCP-Knowledge
Disallow: /

It follows the robots.txt standard (RFC 9309): the group that names WhatsMCP-Knowledge if there is one, otherwise the group for *. It reads your file before every visit, so a change counts from the next one. If your robots.txt answers with a 4xx, such as a 404, it reads that as no rules. If it answers with a 5xx, or can't be reached, it doesn't read the page until it can read the file.

A page it may not read stays out of the customer's knowledge base.

Questions

If our reader does something on your site it shouldn't, or you'd like to ask about a visit, write to [email protected] or use the contact page.