adoptapet.com
robots.txt
Robots Exclusion Standard data for adoptapet.com
Resource Scan
Scan Details
| Site Domain | adoptapet.com |
| Base Domain | adoptapet.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-05-27T10:42:01+00:00 |
| Next Scan | 2024-08-25T10:42:01+00:00 |
Last Successful Scan
| Scanned | 2023-08-02T08:46:12+00:00 |
| URL | https://adoptapet.com/robots.txt |
| Redirect | https://www.adoptapet.com/robots.txt |
| Redirect Domain | www.adoptapet.com |
| Redirect Base | adoptapet.com |
| Domain IPs | 174.129.196.91, 34.197.156.203 |
| Redirect IPs | 13.33.88.104, 13.33.88.107, 13.33.88.70, 13.33.88.98 |
| Response IP | 18.66.97.86 |
| Found | Yes |
| Hash | d4bb4a9a484cf9d9ec110ae73b223458418d10971cdcdb2e3bc8024955d1fb88 |
| SimHash | 911846086316 |
Groups
*
| Rule | Path |
|---|---|
| Allow | /cgi-bin/public/spread-word.cgi/any_page_form |
| Allow | /cgi-bin/public/search-saver.cgi/send_mgmt_link_form |
| Disallow | /cgi-bin |
| Disallow | /errordocs |
| Disallow | /library |
| Disallow | /save-a-pet-show/plugin_detect.html |
| Disallow | /save-a-pet-show/gallery/images |
| Disallow | /save-a-pet-show/images |
| Disallow | /save-a-pet-show/old_episodes/images |
| Disallow | /save-a-pet-show/video |
| Disallow | /save-a-pet-show/theme_song/song.mp3 |
| Disallow | /save-a-pet-show/theme_song/Final_web_track_44khz_stereo_80kbps.mp3 |
| Disallow | /save-a-pet-show/theme_song/1-800-Save-A-Pet_Song.mp3 |
| Disallow | /save-a-pet-show/theme_song/SAP_SONG.mov |
| Disallow | /save-a-pet-show/theme_song/final_song.swf |
| Disallow | /shelter/pointers |
| Disallow | /shelter/tour/images |
| Disallow | /shelter/tour/media_files |
| Disallow | /shelter-search |
| Disallow | /staff |
| Disallow | /Templates/ |
| Disallow | /pet-search-display |
| Disallow | /pet-search-process |
| Disallow | /dog-adoption/search |
| Disallow | /cat-adoption/search |
| Disallow | /pet-adoption/search |
| Disallow | /animal-shelter-search |
| Disallow | /add_favorite_pet |
| Disallow | /remove_favorite_pet |
| Disallow | /update-favorite-pet-list-rankings |
| Disallow | /check_if_favorite_pet |
| Disallow | /site/ |
| Disallow | /pet/*/email-shelter |
| Disallow | /pet-search |
| Disallow | /livewire/ |
| Disallow | /blog/*/feed |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.adoptapet.com/sitemaps/sitemapindex.xml |