foogue.com
robots.txt

Robots Exclusion Standard data for foogue.com

Resource Scan

Scan Details

Site Domain foogue.com
Base Domain foogue.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-10-02T07:53:29+00:00
Next Scan 2024-10-09T07:53:29+00:00

Last Successful Scan

Scanned2024-09-24T07:03:36+00:00
URL https://foogue.com/robots.txt
Domain IPs 104.21.44.162, 172.67.201.47, 2606:4700:3031::6815:2ca2, 2606:4700:3035::ac43:c92f
Response IP 104.21.44.162
Found Yes
Hash 7c1350633a876697cf62d47d3efb7258f82d2c06524a80c95c363042e0b85022
SimHash 59005556e922

Groups

semrushbot

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

semrushbot-coub

Rule Path
Disallow /

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

Other Records

Field Value
sitemap https://foogue.com/sitemap_index.xml

Comments

  • Disallowing the OpenAI web crawler and OpenAI plugins.