Log In Sign Up

vercalendario.info
robots.txt

Robots Exclusion Standard data for vercalendario.info

Resource Scan

Scan Details

Site Domain vercalendario.info
Base Domain vercalendario.info
Scan Status Ok
Last Scan2024-06-07T12:55:48+00:00
Next Scan 2024-06-14T12:55:48+00:00

Last Scan

Scanned2024-06-07T12:55:48+00:00
URL https://vercalendario.info/robots.txt
Redirect https://www.vercalendario.info/robots.txt
Redirect Domain www.vercalendario.info
Redirect Base vercalendario.info
Domain IPs 162.214.1.207
Redirect IPs 162.214.1.207
Response IP 162.214.1.207
Found Yes
Hash 4eca8866347ee3fa9ef6e3d8d7c3511a4cda6c701b06f3fc9aafeab0afe01b08
SimHash 101e4772c9db

Groups

ahrefsbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

sogou

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

phpcrawl

Rule Path
Disallow /

biglotron

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

portalbspider

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

yahoo pipes 1.0

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

tadcrawler

Rule Path
Disallow /

scoutjet

Rule Path
Disallow /
Back to top

Comments

  • Baiduspider
  • Yandex
  • Sogou web spider
  • https://mj12bot.com/
  • https://www.exalead.com/search/webmasterguide
  • Alexa https://support.alexa.com/hc/en-us/articles/200450194-Alexa-s-Web-and-Site-Audit-Crawlers
  • http://phpcrawl.cuab.de/
  • BIGLOTRON (Beta 2;GNU/Linux)
  • the following list comes from timeanddate.com/robots.txt
  • disallow any urls with ? in
Back to top