foogue.com
robots.txt

Robots Exclusion Standard data for foogue.com

Resource Scan

Scan Details

Site Domain foogue.com
Base Domain foogue.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-10-23T07:53:53+00:00
Next Scan 2024-11-22T07:53:53+00:00

Last Successful Scan

Scanned2024-09-24T07:03:36+00:00
URL https://foogue.com/robots.txt
Domain IPs 104.21.44.162, 172.67.201.47, 2606:4700:3031::6815:2ca2, 2606:4700:3035::ac43:c92f
Response IP 104.21.44.162
Found Yes
Hash 7c1350633a876697cf62d47d3efb7258f82d2c06524a80c95c363042e0b85022
SimHash 59005556e922

Groups

semrushbot

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

semrushbot-coub

Rule Path
Disallow /

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

Other Records

Field Value
sitemap https://foogue.com/sitemap_index.xml

Comments

  • Disallowing the OpenAI web crawler and OpenAI plugins.