GPTBot vs OAI-SearchBot vs ChatGPT-User: which one to allow
Updated 10 Nov 2026 · by Venu Gopal Reddy Muvva, founder of BrandInAIAnswers · official sources checked 10 Nov 2026
OAI-SearchBot decides whether your site can appear in ChatGPT search answers, GPTBot crawls for model training, and ChatGPT-User fetches pages when a user asks ChatGPT to. To be cited in ChatGPT search, OAI-SearchBot must be allowed; GPTBot can be blocked without affecting that.
What OpenAI says about each
| Bot | Purpose | robots.txt | IP list |
|---|---|---|---|
| OAI-SearchBot | Surfaces websites in ChatGPT search; opted-out sites are not shown in search answers | Applies; about 24 hours to take effect | openai.com/searchbot.json |
| GPTBot | Crawls content that may be used to train foundation models | Applies | openai.com/gptbot.json |
| ChatGPT-User | User-triggered visits from ChatGPT and Custom GPTs; not automatic crawling | May not apply, as users initiate it | openai.com/chatgpt-user.json |
Pick your setup
- Most businesses: allow all three. You can be cited and models can learn your brand.
- Publishers who do not want training: block GPTBot only. ChatGPT search still sees you.
- Sites that must not appear in ChatGPT at all: block OAI-SearchBot and GPTBot, and put private content behind a login, because ChatGPT-User may not follow robots.txt.
Example robots.txt
User-agent: GPTBot
Disallow: /
User-agent: OAI-SearchBot
Allow: /
User-agent: *
Disallow: /accountNote that OAI-SearchBot has its own group here, so the * group's "Disallow: /account" does not apply to it (RFC 9309 uses the matching group, not both). Add "Disallow: /account" to its group too if you want that.
One crawl, two uses
OpenAI notes that if a site allows both OAI-SearchBot and GPTBot, it may use the results of one crawl for both purposes to avoid duplicate crawling. That is why you may see fewer requests than expected from one of them in your logs.
How to tell them apart in your logs
Each bot sends its name in the User-Agent header, so a simple filter on 'GPTBot', 'OAI-SearchBot' and 'ChatGPT-User' separates them. Expect GPTBot to read many pages at a steady pace, OAI-SearchBot to revisit pages that change, and ChatGPT-User to appear in bursts on specific pages, usually right after someone asked ChatGPT about your product or pasted your URL.
Because names can be faked, confirm suspicious traffic against the JSON IP lists OpenAI publishes for each bot. A request that claims to be OAI-SearchBot from an address outside searchbot.json is not OpenAI.
Questions
My site is not in ChatGPT answers. Is GPTBot the reason?
Usually not. Check OAI-SearchBot in robots.txt and any firewall rules first, then whether your pages answer the question at all.
How do I verify a request really is OpenAI?
Compare the source IP with the JSON lists OpenAI publishes for each bot. See verifying AI crawler IPs.
Does ChatGPT render JavaScript when it visits?
OpenAI's crawler page does not say. Assume it reads your raw HTML and server-render key content.
See which AI answers name you today
BrandInAIAnswers's free audit checks whether AI crawlers can read your site and asks an AI with web search the questions your buyers ask, then shows who gets named. The first 14 days of the full app are free.
Sources
- OpenAI: crawlers and user agents (checked 10 Nov 2026)
- RFC 9309: Robots Exclusion Protocol (checked 10 Nov 2026)