Log In Sign Up

penvape.net
robots.txt

Robots Exclusion Standard data for penvape.net

Resource Scan

Scan Details

Site Domain penvape.net
Base Domain penvape.net
Scan Status Ok
Last Scan2026-01-15T20:56:22+00:00
Next Scan 2026-02-14T20:56:22+00:00

Last Scan

Scanned2026-01-15T20:56:22+00:00
URL https://penvape.net/robots.txt
Domain IPs 104.21.14.133, 172.67.159.45, 2606:4700:3035::ac43:9f2d, 2606:4700:3036::6815:e85
Response IP 172.67.159.45
Found Yes
Hash 6e8c69efcec5a80cb75a5bc26da778f2ae19f3851016e8f816ce9b8d0c12aa90
SimHash 4899fa10efd3

Groups

googlebot

Rule Path
Disallow

bingbot

Rule Path
Disallow

slurp

Rule Path
Disallow

duckduckbot

Rule Path
Disallow

yandexbot

Rule Path
Disallow

chatgpt-user

Rule Path
Disallow

gptbot

Rule Path
Disallow

perplexitybot

Rule Path
Disallow

youbot

Rule Path
Disallow

google-extended

Rule Path
Disallow

anthropicbot

Rule Path
Disallow

neevabot

Rule Path
Disallow

amazonbot

Rule Path
Disallow

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ubersuggest

Rule Path
Disallow /

*

Rule Path
Disallow /
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow */?add_to_cart*
Disallow */?add_to_wishlist*
Disallow */?add-to-cart*
Disallow */?add-to-wishlist*
Disallow */?category-view-mode*
Disallow */?product_count*
Disallow */?orderby*
Disallow /*feed*
Disallow /*rss*
Disallow /wp-content/uploads/wc-logs/
Disallow /wp-content/uploads/woocommerce_transient_files/
Disallow /wp-content/uploads/woocommerce_uploads/
Back to top

Other Records

Field Value
sitemap https://penvape.net/sitemap_index.xml
Back to top

Comments

  • Allow trusted search engine bots
  • Allow reputable AI bots
  • Block SEO/backlink crawlers
  • Catch-all: Block everything else
  • Block dynamic query URLs (for all bots)
  • Block log, cache, and transient directories
  • Sitemap location
Back to top