henkel-gcc.com
robots.txt

Robots Exclusion Standard data for henkel-gcc.com

Resource Scan

Scan Details

Site Domain henkel-gcc.com
Base Domain henkel-gcc.com
Scan Status Ok
Last Scan2024-09-22T00:32:51+00:00
Next Scan 2024-10-06T00:32:51+00:00

Last Scan

Scanned2024-09-22T00:32:51+00:00
URL https://henkel-gcc.com/robots.txt
Redirect https://www.henkel-gcc.com/robots.txt
Redirect Domain www.henkel-gcc.com
Redirect Base henkel-gcc.com
Domain IPs 185.132.25.11
Redirect IPs 185.132.25.11
Response IP 185.132.25.11
Found Yes
Hash e95a7af7900ae14749254d80aa1aa1cf20174d08bc7cc5fdb7fdad38c8d84f48
SimHash c1044040c633

Groups

*

Rule Path
Allow /
Disallow /*ajax/collection

searchatlas

Rule Path
Allow /
Disallow /*ajax/collection

Other Records

Field Value
crawl-delay 6

semrushbot

Rule Path
Allow /
Disallow /*ajax/collection

Other Records

Field Value
crawl-delay 6

Other Records

Field Value
sitemap https://www.henkel-gcc.com/service-sitemap-EN-GCC-sitemap_index.xml

Warnings

  • `request-rate` is not a known field.
  • `visit-time` is not a known field.