cosmeticsnow.sg
robots.txt

Robots Exclusion Standard data for cosmeticsnow.sg

Resource Scan

Scan Details

Site Domain cosmeticsnow.sg
Base Domain cosmeticsnow.sg
Scan Status Ok
Last Scan2025-05-06T23:46:41+00:00
Next Scan 2025-06-05T23:46:41+00:00

Last Scan

Scanned2025-05-06T23:46:41+00:00
URL https://cosmeticsnow.sg/robots.txt
Domain IPs 104.21.29.225, 172.67.171.223, 2606:4700:3032::ac43:abdf, 2606:4700:3036::6815:1de1
Response IP 172.67.171.223
Found Yes
Hash e338964cd3aeeb99308219f8c1c04da3d177a91dd715c4131f7fd2ef7b47593e
SimHash e03ed462ebb9

Groups

blexbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

*

Rule Path
Disallow /track/

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5