pickitz.ai
robots.txt

Robots Exclusion Standard data for pickitz.ai

Resource Scan

Scan Details

Site Domain pickitz.ai
Base Domain pickitz.ai
Scan Status Ok
Last Scan2025-09-16T00:02:24+00:00
Next Scan 2025-10-16T00:02:24+00:00

Last Scan

Scanned2025-09-16T00:02:24+00:00
URL https://pickitz.ai/robots.txt
Domain IPs 34.111.179.208
Response IP 34.111.179.208
Found Yes
Hash e6430551e0e6b990eac2e2d632fd3b06fac7066d92ca70ad31f0b970f4af5f84
SimHash 2223d2e0c524

Groups

*

Rule Path
Allow /
Allow /lottery-app.html
Allow /algorithms.html
Allow /quantum-page.html
Allow /organic.html
Allow /wheeling.html
Allow /cash-pop.html
Allow /news.html
Allow /about.html
Allow /contact.html
Allow /news-page-*.html
Allow /articles/
Allow /sitemap.xml
Allow /favicon.ico
Allow /favicon.png
Disallow /admin/
Disallow /private/
Disallow /.git/
Disallow /node_modules/
Disallow /*.log
Disallow /*.json
Disallow /package.json
Disallow /package-lock.json

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://pickitz.ai/sitemap.xml

Comments

  • Important pages for crawling
  • News archive pages
  • Articles directory
  • Important resources
  • Block admin areas
  • Block log files and development files
  • Sitemap location
  • Crawl delay for respectful crawling