petshoppak.com
robots.txt

Robots Exclusion Standard data for petshoppak.com

Resource Scan

Scan Details

Site Domain petshoppak.com
Base Domain petshoppak.com
Scan Status Ok
Last Scan2026-02-09T07:29:06+00:00
Next Scan 2026-02-16T07:29:06+00:00

Last Scan

Scanned2026-02-09T07:29:06+00:00
URL https://petshoppak.com/robots.txt
Domain IPs 2a02:4780:84:9e27:faa:d35e:b415:154d, 2a02:4780:84:a46d:68cb:9929:e040:6ba9, 77.37.75.92, 91.108.100.188
Response IP 77.37.75.107
Found Yes
Hash a5d58038944e079495a060186375af31c05e9f4b7341b18d87a5de39b2a156f2
SimHash 6aa14840e5a3

Groups

*

Rule Path
Disallow /wp-content/uploads/wc-logs/
Disallow /wp-content/uploads/woocommerce_transient_files/
Disallow /wp-content/uploads/woocommerce_uploads/
Disallow /wp-admin/
Allow /wp-content/uploads/
Allow /wp-content/themes/
Allow /wp-content/plugins/
Allow /wp-json/
Disallow /cgi-bin/
Disallow /wp-includes/
Disallow /xmlrpc.php
Disallow /trackback/
Disallow /feed/
Disallow /comments/
Disallow /?s=
Disallow /search

Other Records

Field Value
crawl-delay 15

*

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

google-extended

Rule Path
Allow /

gptbot

Rule Path
Allow /

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

twitterbot

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

Other Records

Field Value
sitemap https://petshoppak.com/sitemap_index.xml

Comments

  • Allow search engines to crawl everything important
  • Disallow unnecessary system folders and duplicate content
  • START YOAST BLOCK
  • ---------------------------
  • ---------------------------
  • END YOAST BLOCK