tesiverified.it
robots.txt

Robots Exclusion Standard data for tesiverified.it

Resource Scan

Scan Details

Site Domain tesiverified.it
Base Domain tesiverified.it
Scan Status Ok
Last Scan2024-09-14T07:57:34+00:00
Next Scan 2024-09-21T07:57:34+00:00

Last Scan

Scanned2024-09-14T07:57:34+00:00
URL https://tesiverified.it/robots.txt
Domain IPs 104.21.84.74, 172.67.188.164, 2606:4700:3033::ac43:bca4, 2606:4700:3037::6815:544a
Response IP 172.67.188.164
Found Yes
Hash db15d087617afbf914d3ef90b819973590958b57666c0ba9bf4209680a3c9efa
SimHash 490c8463c7b0

Groups

ia_archiver

Rule Path
Disallow /curr/viewcurr.asp
Disallow /google-scholar-data/

*

Rule Path
Disallow /__PDF/
Disallow /v2/appunto-pdf.jsp*
Disallow /tesiteca_docs/
Disallow /inviotesi/
Disallow /tesiteca/

bingbot

Rule Path
Disallow /tesiteca/

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

boldbrains

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.tesionline.it/sitemapindex.xml

Comments

  • Crawl-delay: 1
  • User-Agent: YahooSeeker/M1A1-R2D2
  • User-Agent: MSNBOT_Mobile
  • Allow: /mobile/
  • Disallow: /mobile/
  • User-agent: Yandex
  • Disallow: /