awordfinder.com
robots.txt

Robots Exclusion Standard data for awordfinder.com

Resource Scan

Scan Details

Site Domain awordfinder.com
Base Domain awordfinder.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-10-20T06:59:05+00:00
Next Scan 2024-11-19T06:59:05+00:00

Last Successful Scan

Scanned2024-09-21T06:57:54+00:00
URL https://awordfinder.com/robots.txt
Domain IPs 38.242.244.127
Response IP 38.242.244.127
Found Yes
Hash aebb9d45beeb9f7d4bf5180785b0ee3cdb9aa3c64bad9dce63a06f29eb77725d
SimHash 08189d16e910

Groups

*

Rule Path
Allow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

semrushbot-seoab

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /