curtis.com
robots.txt

Robots Exclusion Standard data for curtis.com

Resource Scan

Scan Details

Site Domain curtis.com
Base Domain curtis.com
Scan Status Ok
Last Scan2024-10-02T19:50:11+00:00
Next Scan 2024-10-09T19:50:11+00:00

Last Scan

Scanned2024-10-02T19:50:11+00:00
URL https://curtis.com/robots.txt
Redirect https://www.curtis.com/robots.txt
Redirect Domain www.curtis.com
Redirect Base curtis.com
Domain IPs 23.54.118.36, 23.54.118.39
Redirect IPs 2600:1413:b000:6::17d5:2bd5, 2600:1413:b000:6::17d5:2bdf, 96.17.96.20, 96.17.96.7
Response IP 23.59.168.121
Found Yes
Hash b09ea79de5146ab61f9d0eebaa5f9dede0d3482370e769bbdaa688253c234177
SimHash 636c9f322f9a

Groups

*

Rule Path
Disallow /cpresources/
Disallow /vendor/
Disallow /.env
Disallow /cache/
Disallow /*?*

Other Records

Field Value
sitemap https://www.curtis.com/sitemaps-1-sitemap.xml
sitemap https://curtis-prod-48439636.us-east-1.elb.amazonaws.com/es/sitemaps-1-sitemap.xml
sitemap https://curtis-prod-48439636.us-east-1.elb.amazonaws.com/it/sitemaps-1-sitemap.xml
sitemap https://curtis-prod-48439636.us-east-1.elb.amazonaws.com/de/sitemaps-1-sitemap.xml
sitemap https://curtis-prod-48439636.us-east-1.elb.amazonaws.com/fr/sitemaps-1-sitemap.xml

Comments

  • robots.txt for https://www.curtis.com/
  • live - don't allow web crawlers to index cpresources/ or vendor/