dosm.gov.my
robots.txt

Robots Exclusion Standard data for dosm.gov.my

Resource Scan

Scan Details

Site Domain dosm.gov.my
Base Domain dosm.gov.my
Scan Status Ok
Last Scan2024-10-29T06:03:59+00:00
Next Scan 2024-11-28T06:03:59+00:00

Last Scan

Scanned2024-10-29T06:03:59+00:00
URL https://dosm.gov.my/robots.txt
Redirect https://www.dosm.gov.my/robots.txt
Redirect Domain www.dosm.gov.my
Redirect Base dosm.gov.my
Domain IPs 185.93.164.58
Redirect IPs 185.93.164.58
Response IP 185.93.164.58
Found Yes
Hash 6c1b793c8e5436d2530a204a3f3ff855eddab6a7e7ee142dea7d2a7e0269ecb2
SimHash 4854df80e373

Groups

*

Rule Path
Disallow /

Comments

  • block all web crawlers