expresscargas.site
robots.txt

Robots Exclusion Standard data for expresscargas.site

Resource Scan

Scan Details

Site Domain expresscargas.site
Base Domain expresscargas.site
Scan Status Ok
Last Scan2025-11-08T20:42:39+00:00
Next Scan 2025-12-08T20:42:39+00:00

Last Scan

Scanned2025-11-08T20:42:39+00:00
URL https://expresscargas.site/robots.txt
Domain IPs 104.21.41.91, 172.67.163.181, 2606:4700:3035::6815:295b, 2606:4700:3035::ac43:a3b5
Response IP 172.67.163.181
Found Yes
Hash 9887f57a518377dc643e9a0c907b7660bc83e97f3e457e959513f7efeaafaa9a
SimHash 48545b718476

Groups

gptbot

Rule Path
Disallow /

chatgpt

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

Other Records

Field Value
sitemap https://expresscargas.site/sitemap.xml

Comments

  • Bloquear User-Agents específicos que indicam scrapers
  • Permitir apenas bots do Google
  • Sitemap