skyandtelescope.com
robots.txt

Robots Exclusion Standard data for skyandtelescope.com

Resource Scan

Scan Details

Site Domain skyandtelescope.com
Base Domain skyandtelescope.com
Scan Status Ok
Last Scan2024-11-11T22:50:56+00:00
Next Scan 2024-11-18T22:50:56+00:00

Last Scan

Scanned2024-11-11T22:50:56+00:00
URL https://skyandtelescope.com/robots.txt
Domain IPs 104.21.43.161, 172.67.181.111, 2606:4700:3031::6815:2ba1, 2606:4700:3031::ac43:b56f
Response IP 104.21.43.161
Found Yes
Hash a2c208e1321fd49dbd641498fdffe7e82845e506aa9518c46ee601b06cc3ca39
SimHash 295d9292e600

Groups

*

Rule Path
Disallow /wp-admin
Disallow /wp-includes
Disallow /wp-content/cache
Disallow */trackback
Disallow */feed
Disallow */comments
Disallow /*?*
Disallow /*?
Disallow /*.png
Disallow /*trackback
Disallow /*/feed/$
Disallow /*/feed/rss/$
Disallow /blog/tag/
Disallow /*.gz$
Disallow /*.cgi$
Disallow /*.wmv$
Disallow /*.png$
Disallow /*.gif$
Disallow /*.cgi$
Disallow /*.xhtml$
Disallow /*.php*
Disallow */trackback*
Disallow /wp-content/observing-tools/moonphase/moon.html
Disallow /wp-content/observing-tools/almanac/almanac.html
Disallow /wp-content/uploads/meteors-free-ebook.html
Disallow /wp-content/observing-tools/mars_profiler/mars.html
Disallow /wp-content/observing-tools/jupiter_moons/jupiter.html
Disallow /wp-content/observing-tools/almanac/welcome.htm

googlebot

Rule Path
Allow *.js*
Allow *.css*
Allow /wp-content/uploads/

mediapartners-google*

Rule Path
Disallow
Allow /*

googlebot-image

Rule Path
Disallow /wp-content/plugins/
Disallow /wp-content/cache/
Disallow /wp-content/themes/
Disallow /wp-includes/
Allow /*

ia_archiver

Rule Path
Disallow /

duggmirror

Rule Path
Disallow /

updownerbot

Rule Path
Disallow /

Other Records

Field Value
sitemap http://www.skyandtelescope.com/sitemap_index.xml
sitemap http://www.skyandtelescope.com/news-sitemap.xml

Comments

  • Allow AdSense bot on Entire Site
  • Allow Google Image Bot to Search all Images
  • Disallow Archiving Site
  • Disable Duggmirror
  • Disable Updowner