Log In Sign Up

tesionline.com
robots.txt

Robots Exclusion Standard data for tesionline.com

Resource Scan

Scan Details

Site Domain tesionline.com
Base Domain tesionline.com
Scan Status Ok
Last Scan2024-09-19T09:52:35+00:00
Next Scan 2024-09-26T09:52:35+00:00

Last Scan

Scanned2024-09-19T09:52:35+00:00
URL https://tesionline.com/robots.txt
Domain IPs 104.21.5.37, 172.67.132.237, 2606:4700:3031::6815:525, 2606:4700:3036::ac43:84ed
Response IP 172.67.132.237
Found Yes
Hash db15d087617afbf914d3ef90b819973590958b57666c0ba9bf4209680a3c9efa
SimHash 490c8463c7b0

Groups

ia_archiver

Rule Path
Disallow /curr/viewcurr.asp
Disallow /google-scholar-data/

*

Rule Path
Disallow /__PDF/
Disallow /v2/appunto-pdf.jsp*
Disallow /tesiteca_docs/
Disallow /inviotesi/
Disallow /tesiteca/

bingbot

Rule Path
Disallow /tesiteca/

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

boldbrains

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.tesionline.it/sitemapindex.xml
Back to top

Comments

  • Crawl-delay: 1
  • User-Agent: YahooSeeker/M1A1-R2D2
  • User-Agent: MSNBOT_Mobile
  • Allow: /mobile/
  • Disallow: /mobile/
  • User-agent: Yandex
  • Disallow: /
Back to top