Log In Sign Up

skincarisma.com
robots.txt

Robots Exclusion Standard data for skincarisma.com

Resource Scan

Scan Details

Site Domain skincarisma.com
Base Domain skincarisma.com
Scan Status Ok
Last Scan2024-10-26T02:22:43+00:00
Next Scan 2024-11-02T02:22:43+00:00

Last Scan

Scanned2024-10-26T02:22:43+00:00
URL https://skincarisma.com/robots.txt
Redirect https://www.skincarisma.com/robots.txt
Redirect Domain www.skincarisma.com
Redirect Base skincarisma.com
Domain IPs 138.68.37.40
Redirect IPs 138.68.37.40
Response IP 138.68.37.40
Found Yes
Hash 3fd3511b9ad85a1ca8d62bc58ce45b701f728e0ea192510173fb91a53f0b8225
SimHash 23842d8d6550

Groups

*
*

Rule Path
Disallow staging-skincarisma.herokuapp.com
Disallow staging.skicarisma.com
Disallow /forum
Disallow /la*
Disallow /adm*
Disallow /users/*/notifications
Disallow /users/*/reviews
Disallow /users/*/lists
Disallow /users/*/ingredients
Disallow /test
Back to top

Other Records

Field Value
sitemap https://s3-ap-southeast-1.amazonaws.com/skincarisma-staging/sitemaps/sitemap.xml.gz
Back to top

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-agent: *
  • Disallow: /
Back to top