tug.org
robots.txt
Robots Exclusion Standard data for tug.org
Resource Scan
Scan Details
Site Domain | tug.org |
Base Domain | tug.org |
Scan Status | Ok |
Last Scan | 2024-05-21T07:29:41+00:00 |
Next Scan | 2024-06-20T07:29:41+00:00 |
Last Scan
Scanned | 2024-05-21T07:29:41+00:00 |
URL | https://tug.org/robots.txt |
Domain IPs | 46.4.94.215 |
Response IP | 46.4.94.215 |
Found | Yes |
Hash | ed3a5949ed00e50f18494ddbc1965ed3d803e04e1ab284b875d01a33b3b0dc6b |
SimHash | a884746b44b3 |
Groups
*
Rule | Path |
---|---|
Disallow | /RCS/ |
Disallow | /cgi-bin |
Disallow | /devsrc/ |
Disallow | /ftp/ |
Disallow | /i-packages/ |
Disallow | /images/ |
Disallow | /proTeXt/ |
Disallow | /svn/ |
Disallow | /teTeX/ |
Disallow | /tetex/release |
Disallow | /tetex/tetex-src |
Disallow | /tetex/tetex-texmfdist |
Disallow | /tetex/tetex-texmfmain |
Disallow | /tetex/texmf |
Disallow | /tetex/texmf-dist |
Disallow | /tetex/texmf-local |
Disallow | /texlive/Contents/ |
Disallow | /texlive/Images/ |
Disallow | /texlive/devsrc/ |
Disallow | /texlive/svn/ |
Disallow | /texmf/ |
Disallow | /texmf-dist/ |
Disallow | /twg/MacTeX/ |
Disallow | /viewvc-docroot/ |
Disallow | /~rlevien/scan/ |
Other Records
Field | Value |
---|---|
crawl-delay | 1 |
Warnings
- 16 invalid lines.
Comments