austinchronicle.com
robots.txt

Robots Exclusion Standard data for austinchronicle.com

Resource Scan

Scan Details

Site Domain austinchronicle.com
Base Domain austinchronicle.com
Scan Status Ok
Last Scan2024-05-18T06:43:52+00:00
Next Scan 2024-05-25T06:43:52+00:00

Last Scan

Scanned2024-05-18T06:43:52+00:00
URL https://austinchronicle.com/robots.txt
Redirect https://www.austinchronicle.com/robots.txt
Redirect Domain www.austinchronicle.com
Redirect Base austinchronicle.com
Domain IPs 104.21.39.205, 172.67.148.123, 2606:4700:3032::ac43:947b, 2606:4700:3036::6815:27cd
Redirect IPs 104.21.39.205, 172.67.148.123, 2606:4700:3032::ac43:947b, 2606:4700:3036::6815:27cd
Response IP 172.67.148.123
Found Yes
Hash e2797d68875f9c7a2403f8af527342b2a4e1e926c115c1787e836ef14332e2c0
SimHash 29520e4b6fd0

Groups

ahrefsbot

Rule Path
Disallow /

*

Rule Path
Disallow /*/music.clubs/
Disallow /ac/
Disallow /awstats/
Disallow /banners/
Disallow /blogads/
Disallow /cafe/
Disallow /check/
Disallow /cinematexasnotes/
Disallow /cityguide/
Disallow /class/
Disallow /devel/
Disallow /documents/
Disallow /eatdrinkmovies/
Disallow /election/
Disallow /email/
Disallow /forum/
Disallow /gallery/
Disallow /giftguide/
Disallow /giftideas/
Disallow /guides/
Disallow /gyrobase/Blogs/Beta/
Disallow /gyrobase/Data/
Disallow /gyrobase/Testing/
Disallow /htdocs/
Disallow /includes/
Disallow /internal/
Disallow /ipod/
Disallow /links/
Disallow /maps/
Disallow /market/
Disallow /marketplace/
Disallow /media/
Disallow /movies_list/
Disallow /mri/
Disallow /mrpants/
Disallow /musicawards/
Disallow /musicpoll/
Disallow /nav/
Disallow /personals/
Disallow /posters/
Disallow /printads/
Disallow /public/
Disallow /rates/
Disallow /retired/
Disallow /screenings/
Disallow /sections/
Disallow /shortstory/
Disallow /survey/
Disallow /test/
Disallow /valentines/
Disallow /whishlists
Disallow /contact/pitch/success/

Comments

  • robots.txt for /
  • This file describes areas of the Chronicle's site that should
  • *not* be indexed by visiting WWW robots which follow the
  • robots.txt standard.
  • webmaster webmaster@austinchronicle.com
  • User-agent: Mediapartners-Google*
  • Disallow:
  • Disallow: /events/
  • Disallow: /images/
  • Disallow: /images_07/
  • Disallow: /search/
  • BW Additions