archives.aapg.org
robots.txt
Robots Exclusion Standard data for archives.aapg.org
Resource Scan
Scan Details
| Site Domain | archives.aapg.org |
| Base Domain | aapg.org |
| Scan Status | Ok |
| Last Scan | 2024-05-25T07:24:00+00:00 |
| Next Scan | 2024-06-24T07:24:00+00:00 |
Last Scan
| Scanned | 2024-05-25T07:24:00+00:00 |
| URL | https://archives.aapg.org/robots.txt |
| Redirect | http://www.aapg.org/robots.txt |
| Redirect Domain | www.aapg.org |
| Redirect Base | aapg.org |
| Domain IPs | 65.38.19.24 |
| Redirect IPs | 104.22.34.192, 104.22.35.192, 172.67.7.79, 2606:4700:10::6816:22c0, 2606:4700:10::6816:23c0, 2606:4700:10::ac43:74f |
| Response IP | 104.22.35.192 |
| Found | Yes |
| Hash | 806325ac756c0c899ebd8e69afd46606f2879f77d74f5ec996264761d8dff122 |
| SimHash | 730d0511cd1d |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /admin/ |
| Disallow | /App_Browsers/ |
| Disallow | /App_Code/ |
| Disallow | /App_Data/ |
| Disallow | /App_GlobalResources/ |
| Disallow | /bin/ |
| Disallow | /Components/ |
| Disallow | /Config/ |
| Disallow | /contest/ |
| Disallow | /controls/ |
| Disallow | /Documentation/ |
| Disallow | /HttpModules/ |
| Disallow | /images/ |
| Disallow | /Install/ |
| Disallow | /Profile/ |
| Disallow | /Providers/ |
| Disallow | /Resources/ContentRotator/ |
| Disallow | /Resources/ControlPanel/ |
| Disallow | /Resources/Dashboard/ |
| Disallow | /Resources/FeedBrowser/ |
| Disallow | /Resources/OpenForceAd/ |
| Disallow | /Resources/Search/ |
| Disallow | /Resources/Shared/ |
| Disallow | /Resources/SkinWidgets/ |
| Disallow | /Resources/TabStrip/ |
| Disallow | /Resources/Widgets/ |
| Disallow | /Staff/ |
| Disallow | /Activity-Feed/userId/ |
| Disallow | /science/pfcatid/ |
| Allow | /DesktopModules/DnnSharp/DnnApiEndpoint/ |
msnbot
| Rule | Path |
|---|---|
| Disallow | /admin/ |
| Disallow | /App_Browsers/ |
| Disallow | /App_Code/ |
| Disallow | /App_Data/ |
| Disallow | /App_GlobalResources/ |
| Disallow | /bin/ |
| Disallow | /Components/ |
| Disallow | /Config/ |
| Disallow | /contest/ |
| Disallow | /controls/ |
| Disallow | /DesktopModules/ |
| Disallow | /Documentation/ |
| Disallow | /HttpModules/ |
| Disallow | /images/ |
| Disallow | /Install/ |
| Disallow | /Profile/ |
| Disallow | /Providers/ |
| Disallow | /Resources/ContentRotator/ |
| Disallow | /Resources/ControlPanel/ |
| Disallow | /Resources/Dashboard/ |
| Disallow | /Resources/FeedBrowser/ |
| Disallow | /Resources/OpenForceAd/ |
| Disallow | /Resources/Search/ |
| Disallow | /Resources/Shared/ |
| Disallow | /Resources/SkinWidgets/ |
| Disallow | /Resources/TabStrip/ |
| Disallow | /Resources/Widgets/ |
| Disallow | /Staff/ |
| Disallow | /Activity-Feed/userId/ |
| Disallow | /science/pfcatid/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
slurp
| Rule | Path |
|---|---|
| Disallow | /*/ctl/ |
| Disallow | /admin/ |
| Disallow | /App_Browsers/ |
| Disallow | /App_Code/ |
| Disallow | /App_Data/ |
| Disallow | /App_GlobalResources/ |
| Disallow | /bin/ |
| Disallow | /Components/ |
| Disallow | /Config/ |
| Disallow | /contest/ |
| Disallow | /controls/ |
| Disallow | /DesktopModules/ |
| Disallow | /Documentation/ |
| Disallow | /HttpModules/ |
| Disallow | /images/ |
| Disallow | /Install/ |
| Disallow | /Profile/ |
| Disallow | /Providers/ |
| Disallow | /Resources/ContentRotator/ |
| Disallow | /Resources/ControlPanel/ |
| Disallow | /Resources/Dashboard/ |
| Disallow | /Resources/FeedBrowser/ |
| Disallow | /Resources/OpenForceAd/ |
| Disallow | /Resources/Search/ |
| Disallow | /Resources/Shared/ |
| Disallow | /Resources/SkinWidgets/ |
| Disallow | /Resources/TabStrip/ |
| Disallow | /Resources/Widgets/ |
| Disallow | /Staff/ |
| Disallow | /Activity-Feed/userId/ |
| Disallow | /science/pfcatid/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
googlebot
| Rule | Path |
|---|---|
| Disallow | /*/ctl/ |
| Disallow | /admin/ |
| Disallow | /App_Browsers/ |
| Disallow | /App_Code/ |
| Disallow | /App_Data/ |
| Disallow | /App_GlobalResources/ |
| Disallow | /bin/ |
| Disallow | /Components/ |
| Disallow | /Config/ |
| Disallow | /contest/ |
| Disallow | /controls/ |
| Disallow | /Documentation/ |
| Disallow | /HttpModules/ |
| Disallow | /images/ |
| Disallow | /Install/ |
| Disallow | /Profile/ |
| Disallow | /Providers/ |
| Disallow | /Resources/ContentRotator/ |
| Disallow | /Resources/ControlPanel/ |
| Disallow | /Resources/Dashboard/ |
| Disallow | /Resources/FeedBrowser/ |
| Disallow | /Resources/OpenForceAd/ |
| Disallow | /Resources/Search/ |
| Disallow | /Resources/Shared/ |
| Disallow | /Resources/SkinWidgets/ |
| Disallow | /Resources/TabStrip/ |
| Disallow | /Resources/Widgets/ |
| Disallow | /Staff/ |
| Disallow | /Activity-Feed/userId/ |
| Disallow | /science/pfcatid/ |
yahoo pipes 1.0
| Rule | Path |
|---|---|
| Disallow | /admin/ |
| Disallow | /App_Browsers/ |
| Disallow | /App_Code/ |
| Disallow | /App_Data/ |
| Disallow | /App_GlobalResources/ |
| Disallow | /bin/ |
| Disallow | /Components/ |
| Disallow | /Config/ |
| Disallow | /contest/ |
| Disallow | /controls/ |
| Disallow | /DesktopModules/ |
| Disallow | /Documentation/ |
| Disallow | /HttpModules/ |
| Disallow | /images/ |
| Disallow | /Install/ |
| Disallow | /Profile/ |
| Disallow | /Providers/ |
| Disallow | /Resources/ContentRotator/ |
| Disallow | /Resources/ControlPanel/ |
| Disallow | /Resources/Dashboard/ |
| Disallow | /Resources/FeedBrowser/ |
| Disallow | /Resources/OpenForceAd/ |
| Disallow | /Resources/Search/ |
| Disallow | /Resources/Shared/ |
| Disallow | /Resources/SkinWidgets/ |
| Disallow | /Resources/TabStrip/ |
| Disallow | /Resources/Widgets/ |
| Disallow | /Staff/ |
| Disallow | /Activity-Feed/userId/ |
| Disallow | /science/pfcatid/ |
Other Records
| Field | Value |
|---|---|
| sitemap | http://www.aapg.org/sitemap.aspx |