gfycow.com
robots.txt

Robots Exclusion Standard data for gfycow.com

Resource Scan

Scan Details

Site Domain gfycow.com
Base Domain gfycow.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-06-23T06:03:32+00:00
Next Scan 2024-09-21T06:03:32+00:00

Last Successful Scan

Scanned2022-11-30T19:59:09+00:00
URL https://gfycow.com/robots.txt
Domain IPs 104.21.88.80, 172.67.174.77
Response IP 172.67.174.77
Found Yes
Hash 916b88da6667efeb3c39b9881ba48fab7a98ebd88110f9b514ea6056f8b01344
SimHash 09189506f991

Groups

*
*

Rule Path
Disallow /api/

gumgum-bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://getyarn.io/sitemap.xml

Comments

  • robotstxt.org