foro.biwenger.com
robots.txt

Robots Exclusion Standard data for foro.biwenger.com

Resource Scan

Scan Details

Site Domain foro.biwenger.com
Base Domain biwenger.com
Scan Status Ok
Last Scan2024-11-09T01:24:09+00:00
Next Scan 2024-11-23T01:24:09+00:00

Last Scan

Scanned2024-11-09T01:24:09+00:00
URL https://foro.biwenger.com/robots.txt
Domain IPs 104.26.0.144, 104.26.1.144, 172.67.69.67, 2606:4700:20::681a:190, 2606:4700:20::681a:90, 2606:4700:20::ac43:4543
Response IP 104.26.0.144
Found Yes
Hash 8540699f004dffebf378d80ef33693adfe33de3af27c76084b3b2b93df830690
SimHash d24c8282eb3b

Groups

imagesiftbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

*

Rule Path
Disallow /proxy.php?*