Log In Sign Up

pcrookie.com
robots.txt

Robots Exclusion Standard data for pcrookie.com

Resource Scan

Scan Details

Site Domain pcrookie.com
Base Domain pcrookie.com
Scan Status Ok
Last Scan2025-10-28T18:31:38+00:00
Next Scan 2025-11-04T18:31:38+00:00

Last Scan

Scanned2025-10-28T18:31:38+00:00
URL https://pcrookie.com/robots.txt
Domain IPs 61.216.106.4
Response IP 61.216.106.4
Found Yes
Hash 96c9308dfa50b86850837dcb5ecc4328ad64f6b67ef62303dfe4405424ab0d17
SimHash 14b45c402fd2

Groups

amazonbot

Product Comment
amazonbot Amazon's user agent
Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

*

Product Comment
* any robot
Rule Path Comment
Disallow /not-allowed/ disallow this directory
Back to top

Other Records

Field Value
sitemap https://pcrookie.com/sitemap.xml
Back to top

Comments

  • User-agent: baiduspider
  • Disallow: /
  • BEGIN XML-SITEMAP-PLUGIN
  • END XML-SITEMAP-PLUGIN
Back to top