foxsports.com.mx
robots.txt

Robots Exclusion Standard data for foxsports.com.mx

Resource Scan

Scan Details

Site Domain foxsports.com.mx
Base Domain foxsports.com.mx
Scan Status Ok
Last Scan2024-05-12T22:16:55+00:00
Next Scan 2024-05-19T22:16:55+00:00

Last Scan

Scanned2024-05-12T22:16:55+00:00
URL https://foxsports.com.mx/robots.txt
Redirect https://www.foxsports.com.mx:443/robots.txt
Redirect Domain www.foxsports.com.mx
Redirect Base foxsports.com.mx
Domain IPs 15.197.252.249, 3.33.250.31
Redirect IPs 125.56.219.34, 2600:1413:b000:14::b857:c147, 2600:1413:b000:14::b857:c160, 96.17.72.80
Response IP 42.99.140.161
Found Yes
Hash afc3f6308b98b2d79a317dc769ff663eb4eb4bf5b56a72e318cb8ddb5d4bc006
SimHash a242fee05b57

Groups

*
slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

petalbot
grapeshot

Rule Path
Disallow

Comments

  • Disallow: /author/*
  • Disallow: /page/latest/*
  • Disallow: */whatsapp$
  • Disallow: */twitter$
  • Disallow: /index.php*
  • Disallow: *.jpg$
  • Disallow: *.jpeg$
  • Disallow: *.gif$
  • Disallow: *.png$