automationjet.com
robots.txt

Robots Exclusion Standard data for automationjet.com

Resource Scan

Scan Details

Site Domain automationjet.com
Base Domain automationjet.com
Scan Status Ok
Last Scan2025-11-29T11:22:11+00:00
Next Scan 2025-12-06T11:22:11+00:00

Last Scan

Scanned2025-11-29T11:22:11+00:00
URL http://automationjet.com/robots.txt
Redirect http://www.automationjet.com/robots.txt
Redirect Domain www.automationjet.com
Redirect Base automationjet.com
Domain IPs 85.207.37.70
Redirect IPs 85.207.37.70
Response IP 85.207.37.70
Found Yes
Hash ff6bed52490ef57f80e5100ef7a3cdeebcf2171388042abfc3bd05581c20bf95
SimHash 6605dc0547d4

Groups

*

Rule Path
Disallow /clearance_sale/*.htm

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10