argen.com
robots.txt

Robots Exclusion Standard data for argen.com

Resource Scan

Scan Details

Site Domain argen.com
Base Domain argen.com
Scan Status Ok
Last Scan2025-09-22T02:04:10+00:00
Next Scan 2025-10-22T02:04:10+00:00

Last Scan

Scanned2025-09-22T02:04:10+00:00
URL https://argen.com/robots.txt
Domain IPs 34.200.133.95, 54.198.220.175
Response IP 54.198.220.175
Found Yes
Hash 61b7d3e1ca4cfcde8c3e938a487783f996c940d47596889704e12fe46b0a4a15
SimHash 4c3550f0c7cb

Groups

googlebot

Rule Path
Disallow

bingbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5

bingpreview

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5

adidxbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5

duckduckbot

Rule Path
Disallow

yahoo! slurp

Rule Path
Disallow

applebot

Rule Path
Disallow

baiduspider

Rule Path
Disallow

yandexbot

Rule Path
Disallow

teoma

Rule Path
Disallow

ahrefsbot

Rule Path
Disallow

twitterbot

Rule Path
Disallow

facebookexternalhit

Rule Path
Disallow

ia_archiver

Rule Path
Disallow

ahrefsbot

Rule Path
Disallow

amazonbot

Rule Path
Disallow /*/store/products

semrush*

Rule Path
Disallow /*/store/products

testomatobot

Rule Path
Disallow /

*

Rule Path
Disallow /