experimentality.co
robots.txt

Robots Exclusion Standard data for experimentality.co

Resource Scan

Scan Details

Site Domain experimentality.co
Base Domain experimentality.co
Scan Status Ok
Last Scan2024-10-14T18:14:10+00:00
Next Scan 2024-11-13T18:14:10+00:00

Last Scan

Scanned2024-10-14T18:14:10+00:00
URL https://www.experimentality.co/robots.txt
Domain IPs 108.157.254.13, 108.157.254.24, 108.157.254.85, 108.157.254.87, 2600:9000:2753:2600:18:2adc:2800:93a1, 2600:9000:2753:3800:18:2adc:2800:93a1, 2600:9000:2753:6400:18:2adc:2800:93a1, 2600:9000:2753:9200:18:2adc:2800:93a1, 2600:9000:2753:b000:18:2adc:2800:93a1, 2600:9000:2753:c800:18:2adc:2800:93a1, 2600:9000:2753:d000:18:2adc:2800:93a1, 2600:9000:2753:e800:18:2adc:2800:93a1
Response IP 108.157.254.13
Found Yes
Hash 089120dcfa361299e02d0e6f848d997c7b1584421de76cae307e5f1addf9cd39
SimHash a429bd710cf4

Groups

*

Rule Path
Disallow /img/*
Disallow /account/*
Disallow /login/*
Disallow /checkout/*
Disallow /busca/*
Disallow /quick-view/*
Disallow /espiar/*
Disallow /buscapagina/*

Other Records

Field Value
sitemap https://www.experimentality.co/sitemap.xml

Comments

  • This robots.txt file controls crawling of URLs under https://example.com.
  • All crawlers are disallowed to crawl files in the "includes" directory, such
  • as .css, .js, but Google needs them for rendering, so Googlebot is allowed
  • to crawl them.
  • Instituciones