sociodreams.com
robots.txt

Robots Exclusion Standard data for sociodreams.com

Resource Scan

Scan Details

Site Domain sociodreams.com
Base Domain sociodreams.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-10-14T15:44:33+00:00
Next Scan 2025-12-13T15:44:33+00:00

Last Successful Scan

Scanned2025-08-16T15:42:41+00:00
URL https://sociodreams.com/robots.txt
Redirect https://www.sociodreams.com/robots.txt
Redirect Domain www.sociodreams.com
Redirect Base sociodreams.com
Domain IPs 69.10.40.66
Redirect IPs 69.10.40.66
Response IP 69.10.40.66
Found Yes
Hash ffccccfd7a7fc4183d7f881110a4ab3a8aa3fe6968a310d8627dfd9be7255f74
SimHash d3665d414376

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /wp-includes/
Allow /wp-includes/js/
Allow /wp-includes/images/
Disallow /trackback/
Disallow /wp-login.php
Disallow /wp-register.php
Disallow /downloads/
Allow /uploads/

magicsearchbot

Rule Path
Disallow /uploads/

httrack

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

teleport

Rule Path
Disallow /

wget

Rule Path
Disallow /

curl

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.sociodreams.com/sitemap_index.xml

Comments

  • This virtual robots.txt file was created by the Virtual Robots.txt WordPress plugin: https://www.wordpress.org/plugins/pc-robotstxt/
  • General rules for all bots
  • Allow uploads for all bots except specific ones
  • Specific rules for magicsearchbot
  • Block specific scraping and cloning bots
  • Sitemap (optional but recommended)