Log In Sign Up

billygraham.org
robots.txt

Robots Exclusion Standard data for billygraham.org

Resource Scan

Scan Details

Site Domain billygraham.org
Base Domain billygraham.org
Scan Status Ok
Last Scan2025-10-22T22:59:29+00:00
Next Scan 2025-11-21T22:59:29+00:00

Last Scan

Scanned2025-10-22T22:59:29+00:00
URL https://billygraham.org/robots.txt
Domain IPs 104.18.2.254, 104.18.29.252, 2606:4700::6812:1dfc, 2606:4700::6812:2fe
Response IP 104.18.2.254
Found Yes
Hash 393c20c49021b3c190d1caea0d70e423100d87f282e8dfba7c2d8a4d8007fc41
SimHash 197c1353e7c4

Groups

*

Rule Path
Allow /

amazonbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /
Allow /peace-with-god
Allow /es/paz-con-dios
Allow /pt/paz-com-deus
Allow /ar/salam-maa-allah
Allow /ru/mir-s-bogom
Allow /zh-hans/yu-shen-he-hao
Allow /zh-hant/yu-shen-he-hao
Allow /search-for-jesus
Allow /es/search-for-jesus
Allow /pt/search-for-jesus
Allow /ar/search-for-jesus
Allow /ru/search-for-jesus
Allow /zh-hans/search-for-jesus
Allow /zh-hant/search-for-jesus

awariorssbot
awariosmartbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claudebot

Rule Path
Disallow /
Allow /peace-with-god
Allow /es/paz-con-dios
Allow /pt/paz-com-deus
Allow /ar/salam-maa-allah
Allow /ru/mir-s-bogom
Allow /zh-hans/yu-shen-he-hao
Allow /zh-hant/yu-shen-he-hao
Allow /search-for-jesus
Allow /es/search-for-jesus
Allow /pt/search-for-jesus
Allow /ar/search-for-jesus
Allow /ru/search-for-jesus
Allow /zh-hans/search-for-jesus
Allow /zh-hant/search-for-jesus

claude-web

Rule Path
Disallow /
Allow /peace-with-god
Allow /es/paz-con-dios
Allow /pt/paz-com-deus
Allow /ar/salam-maa-allah
Allow /ru/mir-s-bogom
Allow /zh-hans/yu-shen-he-hao
Allow /zh-hant/yu-shen-he-hao
Allow /search-for-jesus
Allow /es/search-for-jesus
Allow /pt/search-for-jesus
Allow /ar/search-for-jesus
Allow /ru/search-for-jesus
Allow /zh-hans/search-for-jesus
Allow /zh-hant/search-for-jesus

cohere-ai

Rule Path
Disallow /
Allow /peace-with-god
Allow /es/paz-con-dios
Allow /pt/paz-com-deus
Allow /ar/salam-maa-allah
Allow /ru/mir-s-bogom
Allow /zh-hans/yu-shen-he-hao
Allow /zh-hant/yu-shen-he-hao
Allow /search-for-jesus
Allow /es/search-for-jesus
Allow /pt/search-for-jesus
Allow /ar/search-for-jesus
Allow /ru/search-for-jesus
Allow /zh-hans/search-for-jesus
Allow /zh-hant/search-for-jesus

dataforseobot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /
Allow /peace-with-god
Allow /es/paz-con-dios
Allow /pt/paz-com-deus
Allow /ar/salam-maa-allah
Allow /ru/mir-s-bogom
Allow /zh-hans/yu-shen-he-hao
Allow /zh-hant/yu-shen-he-hao
Allow /search-for-jesus
Allow /es/search-for-jesus
Allow /pt/search-for-jesus
Allow /ar/search-for-jesus
Allow /ru/search-for-jesus
Allow /zh-hans/search-for-jesus
Allow /zh-hant/search-for-jesus

magpie-crawler

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

peer39_crawler
peer39_crawler/1.0

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /
Allow /peace-with-god
Allow /es/paz-con-dios
Allow /pt/paz-com-deus
Allow /ar/salam-maa-allah
Allow /ru/mir-s-bogom
Allow /zh-hans/yu-shen-he-hao
Allow /zh-hant/yu-shen-he-hao
Allow /search-for-jesus
Allow /es/search-for-jesus
Allow /pt/search-for-jesus
Allow /ar/search-for-jesus
Allow /ru/search-for-jesus
Allow /zh-hans/search-for-jesus
Allow /zh-hant/search-for-jesus

facebookexternalhit

Rule Path
Allow /*?*smid=

twitterbot

Rule Path
Allow /*?*smid=

mozilla/5.0 (compatible; ttd-content; +https://www.thetradedesk.com/general/ttd-content)

Rule Path
Allow /
Back to top

Comments

  • robots.txt for https://billygraham.org/
  • Disallow Rules
  • Other Bot Rules
  • Explicitly allowing the TTD contextual crawler
  • Ensures that we're using the correct sitemap.
  • URLs in this sitemap to match www*.npr.org
  • Sitemap: https://billygraham.org/path/to/sitemap.xml
Back to top