Log In Sign Up

astrology.com.au
robots.txt

Robots Exclusion Standard data for astrology.com.au

Resource Scan

Scan Details

Site Domain astrology.com.au
Base Domain astrology.com.au
Scan Status Ok
Last Scan2024-11-09T11:56:30+00:00
Next Scan 2024-11-16T11:56:30+00:00

Last Scan

Scanned2024-11-09T11:56:30+00:00
URL https://astrology.com.au/robots.txt
Domain IPs 172.66.40.164, 172.66.43.92
Response IP 172.66.43.92
Found Yes
Hash f2a6bbf14af829482dd1459727d923ecc8270336c708ab83f5515bfa3020fd2d
SimHash 29048b498632

Groups

*

Rule Path
Allow /
Disallow /*?*
Disallow /*.asp$
Disallow /wp-activate.php
Disallow /wp-blog-header.php
Disallow /comments-post.php
Disallow /wp-config-sample.php
Disallow /wp-config.php
Disallow /wp-cron.php
Disallow /wp-links-opml.php
Disallow /wp-load.php
Disallow /wp-login.php
Disallow /wp-mail.php
Disallow /wp-settings.php
Disallow /wp-signup.php
Disallow /wp-trackback.php
Disallow /xmlrpc.php
Disallow /functions.php
Disallow /wp-login.php
Back to top

Other Records

Field Value
sitemap https://astrology.com.au/sitemap_index.xml
Back to top

Comments

  • Disallow: /wp-content/plugins/
  • Disallow: /wp-admin/
  • Disallow: /readme.html
  • Disallow: /refer/
  • Directories
  • Disallow: /wp-content/uploads/
  • Disallow: /wp-admin
  • Disallow: /wp-includes
  • Disallow: /misc/
  • Disallow: /modules/
  • Disallow: /profiles/
  • Disallow: /scripts/
  • Disallow: /wp-content/themes/
  • Disallow: /default/
  • Files
Back to top

Warnings

  • `noindex` is not a known field.
Back to top