wfmu.org
robots.txt

Robots Exclusion Standard data for wfmu.org

Resource Scan

Scan Details

Site Domain wfmu.org
Base Domain wfmu.org
Scan Status Ok
Last Scan2024-11-04T21:03:05+00:00
Next Scan 2024-12-04T21:03:05+00:00

Last Scan

Scanned2024-11-04T21:03:05+00:00
URL https://wfmu.org/robots.txt
Domain IPs 34.199.157.48, 54.205.163.85
Response IP 54.205.163.85
Found Yes
Hash bd97af802a4d67268a9a661b2bfdfb115d6fabcbb54c41b460dd09300ca1b2dc
SimHash 2808f986e881

Groups

*

Rule Path
Disallow /artistbrowser.php?action=artist

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /