prettynetty.com
robots.txt

Robots Exclusion Standard data for prettynetty.com

Resource Scan

Scan Details

Site Domain prettynetty.com
Base Domain prettynetty.com
Scan Status Ok
Last Scan2025-10-10T08:48:25+00:00
Next Scan 2025-10-17T08:48:25+00:00

Last Scan

Scanned2025-10-10T08:48:25+00:00
URL https://prettynetty.com/robots.txt
Redirect https://www.prettynetty.com/robots.txt
Redirect Domain www.prettynetty.com
Redirect Base prettynetty.com
Domain IPs 46.253.116.84
Redirect IPs 46.253.116.84
Response IP 46.253.116.84
Found Yes
Hash e1b375d457d1d85516e3b0f30116c72948de45a1d8e09b6f22b9da3df953b1d6
SimHash 557cdd70fe2f

Groups

claude-web
claudebot
semrushbot
blexbot
ahrefsbot
dotbot

Rule Path
Disallow /

pinterestbot

Rule Path
Allow /
Disallow /shop/category

Other Records

Field Value
crawl-delay 1.0

gptbot
amazonbot

Rule Path
Allow /
Disallow /shop/category

*

Rule Path
Allow /
Disallow /cart

Other Records

Field Value
crawl-delay 0.2

Comments

  • Disallowed crawlers
  • Pinterest
  • Amazon (Alexa), OpenAI GPTBot
  • All crawlers