pursuingtrivia.com
robots.txt

Robots Exclusion Standard data for pursuingtrivia.com

Resource Scan

Scan Details

Site Domain pursuingtrivia.com
Base Domain pursuingtrivia.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2025-12-03T02:43:45+00:00
Next Scan 2026-01-02T02:43:45+00:00

Last Successful Scan

Scanned2025-10-13T22:13:15+00:00
URL https://pursuingtrivia.com/robots.txt
Redirect https://www.pursuingtrivia.com/robots.txt
Redirect Domain www.pursuingtrivia.com
Redirect Base pursuingtrivia.com
Domain IPs 192.96.218.37
Redirect IPs 192.96.218.37
Response IP 192.96.218.37
Found Yes
Hash 3f3b6e4780b109f9ad7a0bd229e3d5df82f5e09012a6488a59918eb8dd0b57d3
SimHash 410d6bd07651

Groups

baiduspider
bingbot
duckduckbot
exabot
facebookexternalhit
feedfetcher-google
google-extended
google-inspectiontool
google-site-verification
google-speakr
googlebot
googlebot-image
googlebot-news
googlebot-video
mediapartners-google
msnbot
perplexitybot
perplexity-user
qwantify
speedyspider
twitterbot
yandexantivirus/2.0
yandexbot/3.0
yandeximageresizer/2.0
yandeximages/3.0
yandexmedia/3.0
yandexpagechecker/1.0
yandexwebmaster/2.0
yandexzakladki/3.0

Rule Path
Disallow /cgi-bin/

ahrefsbot
ccbot/2.0
chatgpt-user
google-cloudvertexbot
googleother
mj12bot
mojeekbot
petalbot
semrushbot
similarweb
yepbot
youbot
*

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.pursuingtrivia.com/sitemap.xml

Comments

  • Robots.txt for https://www.pursuingtrivia.com/
  • Allow the good bots in
  • Block the well-behaved unwanted bots