abcnews.go.com
robots.txt
Robots Exclusion Standard data for abcnews.go.com
Resource Scan
Scan Details
| Site Domain | abcnews.go.com |
| Base Domain | go.com |
| Scan Status | Ok |
| Last Scan | 2024-05-05T05:33:14+00:00 |
| Next Scan | 2024-05-12T05:33:14+00:00 |
Last Scan
| Scanned | 2024-05-05T05:33:14+00:00 |
| URL | https://abcnews.go.com/robots.txt |
| Domain IPs | 13.226.2.103, 13.226.2.111, 13.226.2.81, 13.226.2.96 |
| Response IP | 18.165.171.93 |
| Found | Yes |
| Hash | 3e361304c984191241da7c8afa4b7c53b42a97ac9096b07e33088818db06b9d2 |
| SimHash | 022bdbd2e672 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /search?searchtext=* |
| Disallow | /disneyid/* |
| Disallow | /assets/static/ads/* |
| Disallow | /cgi |
| Disallow | /xls |
| Disallow | /imp |
| Disallow | /kmail |
| Disallow | /map |
| Disallow | /log |
| Disallow | /gif |
| Disallow | /panel |
| Disallow | /0/ |
| Disallow | /promo/ |
| Disallow | /abclinks/ |
| Disallow | /houseads/ |
| Allow | /xmldata/mrss |
| Allow | /xmldata/rss |
| Allow | /xmldata/xmlPodcast |
| Allow | /xmldata/config |
| Allow | /xmldata/feed |
| Disallow | /sendtofriend/ |
| Allow | /meta/sitemap |
| Disallow | /meta/ |
| Disallow | /staging/ |
| Disallow | /test/ |
| Disallow | /swen/ |
| Disallow | /intro/ |
| Disallow | /go/ |
| Disallow | /news/go/ |
| Disallow | /widgets/ |
| Disallow | /vp2/ |
| Disallow | /Video/*playerIndex |
| Disallow | /*carousel/ |
| Disallow | /*videoLogin? |
| Disallow | /video/browse/ |
| Disallow | /*popup? |
| Disallow | /alerts-v1/ |
| Disallow | /not-allowed/ |
| Disallow | /beta-story-container/* |
| Disallow | /video/embed/* |
| Disallow | /video/amp/embed/* |
| Disallow | /responder |
Other Records
| Field | Value |
|---|---|
| sitemap | https://abcnews.go.com/xmap |
| sitemap | https://abcnews.go.com/xmlLatestStories |
| sitemap | https://abcnews.go.com/xmlLatestVideos |
Comments