ancientmysteries.org
robots.txt

Robots Exclusion Standard data for ancientmysteries.org

Resource Scan

Scan Details

Site Domain ancientmysteries.org
Base Domain ancientmysteries.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-12-05T19:17:35+00:00
Next Scan 2026-01-04T19:17:35+00:00

Last Successful Scan

Scanned2025-11-05T02:24:21+00:00
URL https://ancientmysteries.org/robots.txt
Domain IPs 2a02:4780:38:ca91:cc9f:9b5e:d7f9:9fb1, 2a02:4780:39:23cb:32df:621c:1d34:4b5e, 84.32.84.144, 84.32.84.98
Response IP 77.37.75.92
Found Yes
Hash 5823b3dda2d747f09dc158397bdc761e899ad5f13317c4c35e3316c05b272085
SimHash a088c943459a

Groups

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /wp-content/uploads/

adsbot-google

Rule Path
Allow /

*

Rule Path Comment
Disallow /wp-admin/ -
Disallow /wp-includes/ -
Disallow /cgi-bin/ -
Disallow /trackback/ -
Disallow /feed/ -
Disallow /comments/ -
Disallow /search/ Fixed search disallow rule
Disallow /*?replytocom= -
Disallow /*?s= -
Disallow /*%26sort%3D -
Disallow /*?utm_= -
Disallow /*?fbclid= -
Disallow /*?gclid= -
Disallow /tag/*?paged= -
Disallow /category/*?paged= -
Disallow /author/*?paged= -
Disallow /page/*?paged= -
Disallow /private-research.pdf -
Disallow /*?orderby= -
Disallow /*?filter= -
Disallow /*?preview= -
Disallow /*?mode= -
Disallow /*?dir= -
Disallow /archives/ -
Disallow /?attachment_id= -
Allow /wp-content/uploads/ -
Allow /wp-content/themes/ -
Allow /wp-content/plugins/ -

Other Records

Field Value
sitemap https://ancientmysteries.org/sitemap_index.xml

Comments

  • Block duplicate content from common URL parameters
  • Prevent unnecessary archive indexing
  • Allow important directories for proper site rendering
  • Sitemap for better indexing