staff.topvalu.net
robots.txt

Robots Exclusion Standard data for staff.topvalu.net

Resource Scan

Scan Details

Site Domain staff.topvalu.net
Base Domain topvalu.net
Scan Status Ok
Last Scan2025-07-17T07:08:16+00:00
Next Scan 2025-08-16T07:08:16+00:00

Last Scan

Scanned2025-07-17T07:08:16+00:00
URL https://staff.topvalu.net/robots.txt
Domain IPs 210.167.18.228
Response IP 210.167.18.228
Found Yes
Hash b5621f80fbfb826f957922ebb8e80b3be9eebe4273367ff2b8574b70bf900d09
SimHash 1a34df33e219

Groups

mj12bot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

steeler

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

atomic_email_hunter

Rule Path
Disallow /

catchbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yahoo

Rule Path
Disallow /

scrapy

Rule Path
Disallow /