Log In Sign Up

tesionline.com
robots.txt

Robots Exclusion Standard data for tesionline.com

Resource Scan

Scan Details

Site Domain tesionline.com
Base Domain tesionline.com
Scan Status Ok
Last Scan2024-11-14T12:36:20+00:00
Next Scan 2024-11-21T12:36:20+00:00

Last Scan

Scanned2024-11-14T12:36:20+00:00
URL https://tesionline.com/robots.txt
Domain IPs 104.21.5.37, 172.67.132.237, 2606:4700:3031::6815:525, 2606:4700:3036::ac43:84ed
Response IP 104.21.5.37
Found Yes
Hash 5281b9c1f0eec62dcf90dcf51c1aa954630bbe9c70b4086379aeeb5807cfd38d
SimHash 690c8463c7b0

Groups

ia_archiver

Rule Path
Disallow /curr/viewcurr.asp
Disallow /google-scholar-data/

*

Rule Path
Disallow /__PDF/
Disallow /v2/appunto-pdf.jsp*
Disallow /tesiteca_docs/
Disallow /inviotesi/
Disallow /tesiteca/
Disallow /logons/logon.asp?*
Disallow /consult/cart.jsp?*

bingbot

Rule Path
Disallow /tesiteca/

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

boldbrains

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.tesionline.it/sitemapindex.xml
Back to top

Comments

  • Crawl-delay: 1
  • User-Agent: YahooSeeker/M1A1-R2D2
  • User-Agent: MSNBOT_Mobile
  • Allow: /mobile/
  • Disallow: /mobile/
  • User-agent: Yandex
  • Disallow: /
Back to top