c-resorts.com
robots.txt

Robots Exclusion Standard data for c-resorts.com

Resource Scan

Scan Details

Site Domain c-resorts.com
Base Domain c-resorts.com
Scan Status Ok
Last Scan2025-07-14T06:43:17+00:00
Next Scan 2025-08-13T06:43:17+00:00

Last Scan

Scanned2025-07-14T06:43:17+00:00
URL https://c-resorts.com/robots.txt
Domain IPs 162.159.134.42
Response IP 162.159.134.42
Found Yes
Hash e5a32a8476e48047d93d9e4124a58f8cf7937e239c8c32b255d116742a6f69fd
SimHash 60ea48f7e865

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-content/plugins/
Disallow /wp-content/cache/
Disallow /wp-json/
Disallow /cgi-bin/
Allow /wp-content/uploads/

Other Records

Field Value Comment
crawl-delay 600 Adjust the number of seconds as needed

facebookexternalhit/1.1

Rule Path
Disallow

Other Records

Field Value Comment
crawl-delay 20 Adjust this if necessary

meta-externalagent/1.1

Rule Path
Disallow

Other Records

Field Value Comment
crawl-delay 20 Adjust this as needed

Comments

  • Disallow certain directories to reduce server load
  • Allow crawling of WordPress content that should be indexed
  • Specific rules for certain known bots (example: Facebook's crawler)