gao-town.com
robots.txt
Robots Exclusion Standard data for gao-town.com
Resource Scan
Scan Details
| Site Domain | gao-town.com |
| Base Domain | gao-town.com |
| Scan Status | Ok |
| Last Scan | 2024-11-09T03:20:40+00:00 |
| Next Scan | 2024-11-16T03:20:40+00:00 |
Last Scan
| Scanned | 2024-11-09T03:20:40+00:00 |
| URL | https://gao-town.com/robots.txt |
| Domain IPs | 195.35.33.1, 2a02:4780:b:1271:0:2033:cf78:10 |
| Response IP | 195.35.33.1 |
| Found | Yes |
| Hash | 8dd95f1310c490d67fdbaff4e534b8743b1925fdd79a433706fdd79ecbd416ee |
| SimHash | 002b19c2e672 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /search?searchtext=* |
| Disallow | /disneyid/* |
| Disallow | /assets/static/ads/* |
| Disallow | /cgi |
| Disallow | /xls |
| Disallow | /imp |
| Disallow | /kmail |
| Disallow | /map |
| Disallow | /log |
| Disallow | /gif |
| Disallow | /panel |
| Disallow | /0/ |
| Disallow | /promo/ |
| Disallow | /abclinks/ |
| Disallow | /houseads/ |
| Allow | /xmldata/mrss |
| Allow | /xmldata/rss |
| Allow | /xmldata/xmlPodcast |
| Allow | /xmldata/config |
| Allow | /xmldata/feed |
| Disallow | /sendtofriend/ |
| Allow | /meta/sitemap |
| Disallow | /meta/ |
| Disallow | /staging/ |
| Disallow | /test/ |
| Disallow | /swen/ |
| Disallow | /intro/ |
| Disallow | /go/ |
| Disallow | /news/go/ |
| Disallow | /widgets/ |
| Disallow | /vp2/ |
| Disallow | /Video/*playerIndex |
| Disallow | /*carousel/ |
| Disallow | /*videoLogin? |
| Disallow | /video/browse/ |
| Disallow | /*popup? |
| Disallow | /alerts-v1/ |
| Disallow | /not-allowed/ |
| Disallow | /beta-story-container/* |
| Disallow | /video/embed/* |
| Disallow | /video/amp/embed/* |
| Disallow | /responder |
Other Records
| Field | Value |
|---|---|
| sitemap | /xmap |
| sitemap | /xmlLatestStories |
| sitemap | /xmlLatestVideos |
Comments