danhantao.com
robots.txt
Robots Exclusion Standard data for danhantao.com
Resource Scan
Scan Details
Site Domain | danhantao.com |
Base Domain | danhantao.com |
Scan Status | Ok |
Last Scan | 2024-10-12T06:48:00+00:00 |
Next Scan | 2024-11-11T06:48:00+00:00 |
Last Scan
Scanned | 2024-10-12T06:48:00+00:00 |
URL | https://danhantao.com/robots.txt |
Domain IPs | 104.21.19.156, 172.67.186.205, 2606:4700:3033::6815:139c, 2606:4700:3034::ac43:bacd |
Response IP | 172.67.186.205 |
Found | Yes |
Hash | 57d875b13219a1477e2618a65fed7bf85e459cd431bd1cb94c42cbe34652cef0 |
SimHash | f031014268d3 |
Groups
*
Rule | Path | Comment |
---|---|---|
Disallow | /admin/ | - |
Disallow | /App_Browsers/ | - |
Disallow | /App_Code/ | - |
Disallow | /App_Data/ | - |
Disallow | /App_GlobalResources/ | - |
Disallow | /bin/ | - |
Disallow | /Components/ | - |
Disallow | /Config/ | - |
Disallow | /contest/ | - |
Disallow | /controls/ | - |
Disallow | /Documentation/ | - |
Disallow | /HttpModules/ | - |
Disallow | /Install/ | - |
Disallow | /Providers/ | - |
Disallow | /Resources/ContentRotator/ | - |
Disallow | /Resources/ControlPanel/ | - |
Disallow | /Resources/Dashboard/ | - |
Disallow | /Resources/FeedBrowser/ | - |
Disallow | /Resources/OpenForceAd/ | - |
Disallow | /Resources/SkinWidgets/ | - |
Disallow | /Resources/TabStrip/ | - |
Disallow | /Resources/Widgets/ | - |
Disallow | /Activity-Feed/userId/ | Do not index user profiles |
Disallow | /*?returnurl= | - |
Disallow | /activity-feed/ | - |
msnbot
Rule | Path | Comment |
---|---|---|
Disallow | /admin/ | - |
Disallow | /App_Browsers/ | - |
Disallow | /App_Code/ | - |
Disallow | /App_Data/ | - |
Disallow | /App_GlobalResources/ | - |
Disallow | /bin/ | - |
Disallow | /Components/ | - |
Disallow | /Config/ | - |
Disallow | /contest/ | - |
Disallow | /controls/ | - |
Disallow | /Documentation/ | - |
Disallow | /HttpModules/ | - |
Disallow | /Install/ | - |
Disallow | /Providers/ | - |
Disallow | /Resources/ContentRotator/ | - |
Disallow | /Resources/ControlPanel/ | - |
Disallow | /Resources/Dashboard/ | - |
Disallow | /Resources/FeedBrowser/ | - |
Disallow | /Resources/OpenForceAd/ | - |
Disallow | /Resources/SkinWidgets/ | - |
Disallow | /Resources/TabStrip/ | - |
Disallow | /Resources/Widgets/ | - |
Disallow | /Activity-Feed/userId/ | Do not index user profiles |
Disallow | /*?returnurl= | - |
Disallow | /activity-feed/ | - |
Disallow | /*postid | - |
Other Records
Field | Value |
---|---|
crawl-delay | 30 |
slurp
Rule | Path | Comment |
---|---|---|
Disallow | /*/ctl/ | Slurp permits * |
Disallow | /admin/ | - |
Disallow | /App_Browsers/ | - |
Disallow | /App_Code/ | - |
Disallow | /App_Data/ | - |
Disallow | /App_GlobalResources/ | - |
Disallow | /bin/ | - |
Disallow | /Components/ | - |
Disallow | /Config/ | - |
Disallow | /contest/ | - |
Disallow | /controls/ | - |
Disallow | /Documentation/ | - |
Disallow | /HttpModules/ | - |
Disallow | /Install/ | - |
Disallow | /Providers/ | - |
Disallow | /Resources/ContentRotator/ | - |
Disallow | /Resources/ControlPanel/ | - |
Disallow | /Resources/Dashboard/ | - |
Disallow | /Resources/FeedBrowser/ | - |
Disallow | /Resources/OpenForceAd/ | - |
Disallow | /Resources/SkinWidgets/ | - |
Disallow | /Resources/TabStrip/ | - |
Disallow | /Resources/Widgets/ | - |
Disallow | /Activity-Feed/userId/ | Do not index user profiles |
Disallow | /*?returnurl= | - |
Disallow | /activity-feed/ | - |
Disallow | /*postid | - |
Other Records
Field | Value |
---|---|
crawl-delay | 30 |
googlebot
Rule | Path | Comment |
---|---|---|
Disallow | /*/ctl/ | Googlebot permits * |
Disallow | /admin/ | - |
Disallow | /App_Browsers/ | - |
Disallow | /App_Code/ | - |
Disallow | /App_Data/ | - |
Disallow | /App_GlobalResources/ | - |
Disallow | /bin/ | - |
Disallow | /Components/ | - |
Disallow | /Config/ | - |
Disallow | /contest/ | - |
Disallow | /controls/ | - |
Disallow | /Documentation/ | - |
Disallow | /HttpModules/ | - |
Disallow | /Install/ | - |
Disallow | /Providers/ | - |
Disallow | /Resources/ContentRotator/ | - |
Disallow | /Resources/ControlPanel/ | - |
Disallow | /Resources/Dashboard/ | - |
Disallow | /Resources/FeedBrowser/ | - |
Disallow | /Resources/OpenForceAd/ | - |
Disallow | /Resources/SkinWidgets/ | - |
Disallow | /Resources/TabStrip/ | - |
Disallow | /Resources/Widgets/ | - |
Disallow | /Activity-Feed/userId/ | Do not index user profiles |
Disallow | /*?returnurl= | - |
Disallow | /activity-feed/ | - |
Disallow | /*postid | - |
yahoo pipes 1.0
Rule | Path | Comment |
---|---|---|
Disallow | /admin/ | - |
Disallow | /App_Browsers/ | - |
Disallow | /App_Code/ | - |
Disallow | /App_Data/ | - |
Disallow | /App_GlobalResources/ | - |
Disallow | /bin/ | - |
Disallow | /Components/ | - |
Disallow | /Config/ | - |
Disallow | /contest/ | - |
Disallow | /controls/ | - |
Disallow | /Documentation/ | - |
Disallow | /HttpModules/ | - |
Disallow | /Install/ | - |
Disallow | /Providers/ | - |
Disallow | /Resources/ContentRotator/ | - |
Disallow | /Resources/ControlPanel/ | - |
Disallow | /Resources/Dashboard/ | - |
Disallow | /Resources/FeedBrowser/ | - |
Disallow | /Resources/OpenForceAd/ | - |
Disallow | /Resources/SkinWidgets/ | - |
Disallow | /Resources/TabStrip/ | - |
Disallow | /Resources/Widgets/ | - |
Disallow | /Activity-Feed/userId/ | Do not index user profiles |
Disallow | /*?returnurl= | - |
Disallow | /activity-feed/ | - |
Disallow | /*postid | - |
Warnings
- 18 invalid lines.
Comments