Log In Sign Up

zocalo.com.mx
robots.txt

Robots Exclusion Standard data for zocalo.com.mx

Resource Scan

Scan Details

Site Domain zocalo.com.mx
Base Domain zocalo.com.mx
Scan Status Ok
Last Scan2024-06-25T23:00:40+00:00
Next Scan 2024-07-02T23:00:40+00:00

Last Scan

Scanned2024-06-25T23:00:40+00:00
URL https://zocalo.com.mx/robots.txt
Domain IPs 104.26.4.92, 104.26.5.92, 172.67.73.96, 2606:4700:20::681a:45c, 2606:4700:20::681a:55c, 2606:4700:20::ac43:4960
Response IP 172.67.73.96
Found Yes
Hash a1ef53f3efa0f74bd8fc425bfb8a412bfa87f834a8a42e0e946b1c1396222524
SimHash 4da4da6068b5

Groups

twitterbot

Rule Path
Allow /
Allow /wp-content/uploads/

mediapartners-google

Rule Path
Disallow

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /.tmb/
Disallow /.quarantine/
Allow /

Other Records

Field Value
crawl-delay 15

googlebot

Rule Path
Disallow /wp-admin/
Disallow /wp-content/
Disallow /wp-includes/
Disallow /.tmb/
Disallow /.quarantine/
Allow /
Back to top

Comments

  • Exclude Files From All Robots:
  • Disallow: /wp-content/
  • UserAgent: Facebot
  • Disallow: /
  • UserAgent: facebookexternalhit/1.0 (+http://www.facebook.com/externalhit_uatext.php)
  • Disallow: /
  • UserAgent: facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
  • Disallow: /
  • UserAgent: CrowdTanglebot/1.0
  • Disallow: /
Back to top