something.global
robots.txt

Robots Exclusion Standard data for something.global

Resource Scan

Scan Details

Site Domain something.global
Base Domain something.global
Scan Status Ok
Last Scan2025-12-03T18:13:22+00:00
Next Scan 2025-12-10T18:13:22+00:00

Last Scan

Scanned2025-12-03T18:13:22+00:00
URL https://something.global/robots.txt
Domain IPs 104.21.90.39, 172.67.194.236, 2606:4700:3030::ac43:c2ec, 2606:4700:3034::6815:5a27
Response IP 104.21.90.39
Found Yes
Hash 403afec78402ab608d4e38a1792d4f870d4962700d4dbe6dbed5f3accca404e2
SimHash 43289f563f12

Groups

*

Rule Path
Disallow /cpresources/
Disallow /vendor/
Disallow /.env
Disallow /cache/

Other Records

Field Value
sitemap https://something.global/sitemaps-1-sitemap.xml

Comments

  • robots.txt for https://something.global/
  • default - don't allow web crawlers to index cpresources/ or vendor/