automation-berlin.com
robots.txt

Robots Exclusion Standard data for automation-berlin.com

Resource Scan

Scan Details

Site Domain automation-berlin.com
Base Domain automation-berlin.com
Scan Status Ok
Last Scan2024-09-13T18:27:29+00:00
Next Scan 2024-10-13T18:27:29+00:00

Last Scan

Scanned2024-09-13T18:27:29+00:00
URL https://automation-berlin.com/robots.txt
Redirect https://www.automation-berlin.com/robots.txt
Redirect Domain www.automation-berlin.com
Redirect Base automation-berlin.com
Domain IPs 162.55.130.182
Redirect IPs 162.55.130.182
Response IP 162.55.130.182
Found Yes
Hash 2cbd29da503e271c617421804c99d372be6004812cd3ffd7394506519b68792c
SimHash d91c44c86c50

Groups

*

Rule Path
Disallow /admin/
Disallow /aplication/
Disallow /lib/
Disallow /media/
Disallow /modules/
Disallow /system/
Disallow /vendor/
Disallow /tmp/

blexbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

spbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.com

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

mtrobot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /