skyandtelescope.org
robots.txt

Robots Exclusion Standard data for skyandtelescope.org

Resource Scan

Scan Details

Site Domain skyandtelescope.org
Base Domain skyandtelescope.org
Scan Status Ok
Last Scan2024-06-08T07:02:08+00:00
Next Scan 2024-06-15T07:02:08+00:00

Last Scan

Scanned2024-06-08T07:02:08+00:00
URL https://skyandtelescope.org/robots.txt
Domain IPs 104.26.4.119, 104.26.5.119, 172.67.75.119, 2606:4700:20::681a:477, 2606:4700:20::681a:577, 2606:4700:20::ac43:4b77
Response IP 104.26.4.119
Found Yes
Hash a2c208e1321fd49dbd641498fdffe7e82845e506aa9518c46ee601b06cc3ca39
SimHash 295d9292e600

Groups

*

Rule Path
Disallow /wp-admin
Disallow /wp-includes
Disallow /wp-content/cache
Disallow */trackback
Disallow */feed
Disallow */comments
Disallow /*?*
Disallow /*?
Disallow /*.png
Disallow /*trackback
Disallow /*/feed/$
Disallow /*/feed/rss/$
Disallow /blog/tag/
Disallow /*.gz$
Disallow /*.cgi$
Disallow /*.wmv$
Disallow /*.png$
Disallow /*.gif$
Disallow /*.cgi$
Disallow /*.xhtml$
Disallow /*.php*
Disallow */trackback*
Disallow /wp-content/observing-tools/moonphase/moon.html
Disallow /wp-content/observing-tools/almanac/almanac.html
Disallow /wp-content/uploads/meteors-free-ebook.html
Disallow /wp-content/observing-tools/mars_profiler/mars.html
Disallow /wp-content/observing-tools/jupiter_moons/jupiter.html
Disallow /wp-content/observing-tools/almanac/welcome.htm

googlebot

Rule Path
Allow *.js*
Allow *.css*
Allow /wp-content/uploads/

mediapartners-google*

Rule Path
Disallow
Allow /*

googlebot-image

Rule Path
Disallow /wp-content/plugins/
Disallow /wp-content/cache/
Disallow /wp-content/themes/
Disallow /wp-includes/
Allow /*

ia_archiver

Rule Path
Disallow /

duggmirror

Rule Path
Disallow /

updownerbot

Rule Path
Disallow /

Other Records

Field Value
sitemap http://www.skyandtelescope.com/sitemap_index.xml
sitemap http://www.skyandtelescope.com/news-sitemap.xml

Comments

  • Allow AdSense bot on Entire Site
  • Allow Google Image Bot to Search all Images
  • Disallow Archiving Site
  • Disable Duggmirror
  • Disable Updowner