blocklayer.com
robots.txt

Robots Exclusion Standard data for blocklayer.com

Resource Scan

Scan Details

Site Domain blocklayer.com
Base Domain blocklayer.com
Scan Status Ok
Last Scan2024-06-12T09:26:26+00:00
Next Scan 2024-06-19T09:26:26+00:00

Last Scan

Scanned2024-06-12T09:26:26+00:00
URL https://blocklayer.com/robots.txt
Redirect https://www.blocklayer.com/robots.txt
Redirect Domain www.blocklayer.com
Redirect Base blocklayer.com
Domain IPs 67.227.173.150
Redirect IPs 67.227.173.150
Response IP 67.227.173.150
Found Yes
Hash 9dce9ef7464986d1be540e8fce6a88eba9e472945051a634f948910978b79b45
SimHash 1815d221e793

Groups

*

Rule Path
Disallow /

googlebot

Rule Path
Disallow /*.ashx
Disallow /us?i=*

mediapartners-google

Rule Path
Disallow /*.ashx

adsbot

Rule Path
Disallow /*.ashx

adsbot mobile web

Rule Path
Disallow /*.ashx

adsbot mobile web android

Rule Path
Disallow /*.ashx

bingbot

Rule Path
Disallow /*.ashx
Disallow /*.aspx/
Disallow /*.ASPX/
Disallow /us?i=*

applebot

Rule Path
Disallow /*.ashx
Disallow /us?i=*

duckduckbot

Rule Path
Disallow /*.ashx
Disallow /us?i=*

slurp

Rule Path
Disallow /*.ashx
Disallow /us?i=*

gptbot

Rule Path
Disallow /*.ashx

chatgpt-user

Rule Path
Disallow /*.ashx

archive.org_bot

Rule Path
Disallow /*.ashx
Disallow /us?i=*

awario

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /*.ashx
Disallow /us?i=*