How Perplexity picks sources, and why it cites so many

Updated 10 Nov 2026 · by Venu Gopal Reddy Muvva, founder of BrandInAIAnswers · official sources checked 10 Nov 2026

Perplexity answers every question from live web results and shows numbered citations, so pages its crawler PerplexityBot has indexed are the pool it picks from. Perplexity says PerplexityBot is used to surface and link websites in its search results and is not used to train foundation models, so allowing it is purely about being citable.

Perplexity's user agents

User agentPurpose (Perplexity's docs)robots.txt
PerplexityBotIndex pages to surface and link them in Perplexity's results; not for training AI modelsFollows robots.txt
Perplexity-UserFetch a page when a user asks Perplexity about itUser-initiated; Perplexity says it generally ignores robots.txt, since a person asked

What gets cited

  • Pages that state the answer early, often in a short paragraph or a table Perplexity can quote.
  • Comparison and list pages for 'best X for Y' questions, including honest ones that name competitors.
  • Recent pages for time-sensitive questions: put the date you last checked facts on the page.
  • Forum threads, reviews and directories for opinion questions; your own site for facts like prices, hours and specs.

Worked example: robots.txt that allows Perplexity search only

User-agent: PerplexityBot
Allow: /
Disallow: /account/

User-agent: GPTBot
Disallow: /

User-agent: *
Allow: /
Disallow: /account/

Remember that a named group replaces the * group for that crawler, so repeat any Disallow lines you need inside the PerplexityBot group. Run the AI crawler checker logic on your own file with the free checker.

Checklist

  • PerplexityBot allowed in robots.txt and not challenged by your CDN's bot protection.
  • Key pages linked from the home page and listed in the sitemap.
  • Facts in HTML text; tables for numbers.
  • Referral visits from perplexity.ai tracked as their own channel.

Common reasons a page is never cited

  • A bot-protection challenge (CAPTCHA or JavaScript check) returned to PerplexityBot instead of the page; check your CDN's bot settings.
  • The answer is buried below a long introduction, so another page states it sooner.
  • Prices and specs sit in images or PDFs that the crawler can't quote.
  • The page is new and not yet linked from anywhere on your site.
  • A competitor publishes a comparison table and you publish only a product page.

Questions

Does Perplexity use my content to train models?

Perplexity's documentation says PerplexityBot is not used to crawl content for AI foundation models.

Why does Perplexity cite a competitor for my brand name?

Usually because their page answers the exact question and yours doesn't, or yours can't be crawled. Publish a page that answers it directly.

Can I verify PerplexityBot traffic?

Perplexity publishes the IP ranges its crawlers use; compare the IPs in your logs against them. See verifying AI crawler IPs.

See which AI answers name you today

BrandInAIAnswers's free audit checks whether AI crawlers can read your site and asks an AI with web search the questions your buyers ask, then shows who gets named. The first 14 days of the full app are free.

Sources

Related

How Perplexity picks its sources: PerplexityBot explained