galgo.com
robots.txt

Robots Exclusion Standard data for galgo.com

Resource Scan

Scan Details

Site Domain galgo.com
Base Domain galgo.com
Scan Status Ok
Last Scan2024-09-17T16:53:02+00:00
Next Scan 2024-10-17T16:53:02+00:00

Last Scan

Scanned2024-09-17T16:53:02+00:00
URL https://galgo.com/robots.txt
Domain IPs 108.156.133.19, 108.156.133.38, 108.156.133.42, 108.156.133.95
Response IP 108.156.133.19
Found Yes
Hash 09be6d916fa1da956bd6bcc2cffe1b10b4228c42645e54bcb52ad94627de8a58
SimHash 44571c558f52

Groups

*

Rule Path
Disallow /cl/checkout*
Disallow /pe/checkout*
Disallow /mx/checkout*

siteauditbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.galgo.com/sitemap.xml

Comments

  • Robots.txt for www.galgo.com
  • Checkout rules
  • Disallow Semrush crawler to avoid conflicts with Segment
  • Sitemap