vaporshark.com
robots.txt

Robots Exclusion Standard data for vaporshark.com

Resource Scan

Scan Details

Site Domain vaporshark.com
Base Domain vaporshark.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-04-21T09:30:33+00:00
Next Scan 2024-07-20T09:30:33+00:00

Last Successful Scan

Scanned2023-09-01T05:29:41+00:00
URL https://vaporshark.com/robots.txt
Redirect https://www.vaporshark.com/robots.txt
Redirect Domain www.vaporshark.com
Redirect Base vaporshark.com
Domain IPs 104.26.12.40, 104.26.13.40, 172.67.69.201, 2606:4700:20::681a:c28, 2606:4700:20::681a:d28, 2606:4700:20::ac43:45c9
Redirect IPs 104.26.12.40, 104.26.13.40, 172.67.69.201, 2606:4700:20::681a:c28, 2606:4700:20::681a:d28, 2606:4700:20::ac43:45c9
Response IP 104.26.13.40
Found Yes
Hash 9ba746238a26e5846cdd5e284074be126f70f3008a84544734a0c041f59a469e
SimHash ec50db026793

Groups

*

Rule Path
Disallow /tag/
Disallow /review/
Disallow /catalog/category/view/
Disallow /catalog/product/view/
Disallow /catalogsearch/
Disallow /checkout/
Disallow /customer/
Disallow /*?

Comments

  • Enables robots.txt rules for all crawlers
  • Disable Tag & Review
  • Disable Raw URLs
  • Disable Checkout, Customer & Search
  • Disable Query Strings