puzzledepot.com
robots.txt

Robots Exclusion Standard data for puzzledepot.com

Resource Scan

Scan Details

Site Domain puzzledepot.com
Base Domain puzzledepot.com
Scan Status Ok
Last Scan2025-10-27T04:33:00+00:00
Next Scan 2025-11-03T04:33:00+00:00

Last Scan

Scanned2025-10-27T04:33:00+00:00
URL https://puzzledepot.com/robots.txt
Domain IPs 172.66.41.11, 172.66.42.245, 2606:4700:3108::ac42:290b, 2606:4700:3108::ac42:2af5
Response IP 172.66.41.11
Found Yes
Hash 8571424a282fa437f889177944e8c40f9dbe4de434937860a31ec5b2118e046c
SimHash 291c4320e5a1

Groups

*

Rule Path
Allow /
Allow /html/
Allow /rebus-word-puzzles/
Allow /trivia/
Disallow /cgi-bin/
Disallow /logs/
Disallow /private/
Disallow /admin/
Disallow /tmp/
Disallow /temp/
Disallow /*.log$
Disallow /*.tmp$
Disallow /backup/
Disallow /.git/
Disallow /.htaccess
Disallow /server-status
Disallow /server-info
Disallow /*.bak$
Disallow /config/
Disallow /dev-tools/
Disallow /test/
Disallow /testing/

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

slurp

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

semrushbot

Rule Path
Allow /

ahrefsbot

Rule Path
Allow /

mj12bot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://puzzledepot.com/sitemap.xml

Comments

  • Allow crawling of main content areas
  • Disallow sensitive and unnecessary areas
  • Disallow crawling of development and utility files
  • Allow popular search engines
  • Block unwanted bots and crawlers
  • Sitemap location (if you have one)
  • Crawl delay for respectful crawling