itdell.ru
robots.txt

Robots Exclusion Standard data for itdell.ru

Resource Scan

Scan Details

Site Domain itdell.ru
Base Domain itdell.ru
Scan Status Ok
Last Scan4/13/2025, 12:40:44 AM
Next Scan 4/20/2025, 12:40:44 AM

Last Scan

Scanned4/13/2025, 12:40:44 AM
URL https://itdell.ru/robots.txt
Domain IPs 104.21.66.188, 172.67.163.118, 2606:4700:3031::ac43:a376, 2606:4700:3032::6815:42bc
Response IP 172.67.163.118
Found Yes
Hash 7ab691844912493568010a66b431eb7bfad9e653caf134aca29f9dee3bd0e9d7
SimHash 0f449e774ebd

Groups

*

Rule Path
Allow /forum/*sitemap
Allow /forum/*rss
Allow /forum/*type%3Drss
Disallow /forum/attachments/
Disallow /forum/avatars/
Disallow /forum/Packages/
Disallow /forum/Smileys/
Disallow /forum/Sources/
Disallow /forum/Themes/
Disallow /forum/Games/
Disallow /forum/*.msg
Disallow /forum/*.new
Disallow /forum/*sort
Disallow /forum/*topicseen
Disallow /forum/*wap
Disallow /forum/*imode
Disallow /forum/*action
Disallow /forum/*prev_next
Disallow /forum/*all
Disallow /forum/index.php?action=sitemap
Disallow /forum/go.php
Disallow /blocks/
Disallow /administrator/

twiceler

Rule Path
Disallow /

w3c-checklink

Rule Path
Disallow /

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 100

Other Records

Field Value
sitemap http://itdell.ru/sitemap.xml
sitemap http://itdell.ru/sitemap_forum.xml

Warnings

  • `host` is not a known field.