Log In Sign Up

survivorfandom.com
robots.txt

Robots Exclusion Standard data for survivorfandom.com

Resource Scan

Scan Details

Site Domain survivorfandom.com
Base Domain survivorfandom.com
Scan Status Ok
Last Scan2024-09-27T18:14:48+00:00
Next Scan 2024-10-04T18:14:48+00:00

Last Scan

Scanned2024-09-27T18:14:48+00:00
URL https://survivorfandom.com/robots.txt
Domain IPs 64.91.249.62
Response IP 64.91.249.62
Found Yes
Hash c67ac14cda992cac33df325fe7b6c048794a412b29a9148ff6acd2609b1356e3
SimHash 2afb4d1144d7

Groups

*

Rule Path
Disallow /cgi-bin/
Disallow /*.inc$
Disallow /*.txt$
Disallow /wp-admin/
Disallow /wp-content/upgrade/
Disallow /wp-content/w3tc/
Disallow /ads/
Disallow /cwfl/
Disallow /docs/
Disallow /misc/
Disallow /stats*

googlebot-image

Rule Path
Disallow
Allow /*
Back to top

Other Records

Field Value
sitemap http://survivorfandom.com/sitemap.xml.gz
Back to top

Comments

  • disallow files in the following folders
  • disallow all files ending in .php
  • Disallow: /*.js$
  • Disallow: /*.css$
  • disallow all files in /wp- directorys
  • Disallow: /wp-includes/
  • Disallow: /wp-content/plugins/
  • Disallow: /wp-content/themes/
  • Disallow: /wp-content/wptouch-data/
  • disallow all files with ? in url
  • disallow any files that are stats related
  • Sitemap location for auto-discovery
Back to top