Log In Sign Up

pxlworld.com
robots.txt

Robots Exclusion Standard data for pxlworld.com

Resource Scan

Scan Details

Site Domain pxlworld.com
Base Domain pxlworld.com
Scan Status Ok
Last Scan2025-07-07T09:28:01+00:00
Next Scan 2025-08-06T09:28:01+00:00

Last Scan

Scanned2025-07-07T09:28:01+00:00
URL https://pxlworld.com/robots.txt
Domain IPs 104.26.14.38, 104.26.15.38, 172.67.70.70, 2606:4700:20::681a:e26, 2606:4700:20::681a:f26, 2606:4700:20::ac43:4646
Response IP 104.26.14.38
Found Yes
Hash 7fe5cf2afa3ce02c614230ab90c53db5bfe6004860a7c5e4a3aadb1137be813a
SimHash 403ce8026796

Groups

*

Rule Path
Disallow /check-jobs-table.js
Disallow /reset-password.html

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

dataforseo

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://pxlworld.com/sitemap.xml
Back to top

Comments

  • Allow all major search engine bots
  • Block known AI bots
  • Optional: Block archivers / general scraping
Back to top