automation24.ir
robots.txt

Robots Exclusion Standard data for automation24.ir

Resource Scan

Scan Details

Site Domain automation24.ir
Base Domain automation24.ir
Scan Status Ok
Last Scan2025-11-20T22:03:19+00:00
Next Scan 2025-12-20T22:03:19+00:00

Last Scan

Scanned2025-11-20T22:03:19+00:00
URL https://automation24.ir/robots.txt
Redirect https://www.automation24.ir/robots.txt
Redirect Domain www.automation24.ir
Redirect Base automation24.ir
Domain IPs 80.249.115.90
Redirect IPs 80.249.115.90
Response IP 80.249.115.90
Found Yes
Hash 4759905d4bbdf877b11f766dc622204aebf8c639ed523f5f449f6bf2b6a99b44
SimHash 590c1f206e81

Groups

megaindex.ru

Rule Path
Disallow /

firefox version 10 and lower - various robots

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

dotbot

Rule Path
Disallow /admin/
Disallow /scripts/
Disallow /images/

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

crawl

Rule Path
Disallow /

*

Rule Path
Allow /

Other Records

Field Value
sitemap http://automation24.ir/sitemapindex.xml