afagain.com
robots.txt

Robots Exclusion Standard data for afagain.com

Resource Scan

Scan Details

Site Domain afagain.com
Base Domain afagain.com
Scan Status Ok
Last Scan2025-02-14T23:39:38+00:00
Next Scan 2025-03-16T23:39:38+00:00

Last Scan

Scanned2025-02-14T23:39:38+00:00
URL https://afagain.com/robots.txt
Redirect https://AFATHINK.COM/robots.txt
Redirect Domain afathink.com
Redirect Base afathink.com
Domain IPs 104.21.33.10, 172.67.139.160, 2606:4700:3031::ac43:8ba0, 2606:4700:3033::6815:210a
Redirect IPs 104.21.65.177, 172.67.147.225, 2606:4700:3030::6815:41b1, 2606:4700:3033::ac43:93e1
Response IP 104.21.65.177
Found Yes
Hash e54e3814606982feb86a02bc4f56d1d00827f8bd31e0fc3d23954b0691493055
SimHash 6551d31555d7

Groups

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

googlebot-mobile

Rule Path
Disallow

msnbot

Rule Path
Disallow

slurp

Rule Path
Disallow

gigabot

Rule Path
Disallow

baiduspider

Rule Path
Disallow

yahoo-mmcrawler

Rule Path
Disallow

psbot

Rule Path
Disallow

yahoo-blogs/v3.9

Rule Path
Disallow

*

Rule Path
Disallow
Disallow /cgi-bin/

Other Records

Field Value
sitemap https://afadeals.com/

Warnings

  • 1 invalid line.