webapp1.dlib.indiana.edu
robots.txt
Robots Exclusion Standard data for webapp1.dlib.indiana.edu
Resource Scan
Scan Details
| Site Domain | webapp1.dlib.indiana.edu |
| Base Domain | indiana.edu |
| Scan Status | Ok |
| Last Scan | 2024-11-03T15:06:54+00:00 |
| Next Scan | 2024-12-03T15:06:54+00:00 |
Last Scan
| Scanned | 2024-11-03T15:06:54+00:00 |
| URL | https://webapp1.dlib.indiana.edu/robots.txt |
| Domain IPs | 156.56.241.6 |
| Response IP | 156.56.241.6 |
| Found | Yes |
| Hash | 71c1830832d4c5125d43f3146f8593499c5c3adad9bf6aff0839a42c5ccc7b25 |
| SimHash | a8048b32c514 |
Groups
gsa-crawler
| Rule | Path |
|---|---|
| Allow | /sheetmusic/devincent.do |
| Disallow | /fli/ |
| Disallow | /cushman/browse/ |
| Disallow | /cushman/search/ |
| Disallow | /cushman/results/result.do |
| Disallow | /cushman/overview/ordering.jsp |
| Disallow | /collections/dido/ |
| Disallow | /confluence/ |
| Disallow | /newton/search |
| Disallow | /newton/mss |
| Disallow | /metsnav/ |
| Disallow | /metsnav2/ |
| Disallow | /metsnav3/ |
| Disallow | /metsnav3service/ |
| Disallow | /inharmony/search.do |
| Disallow | /imagesearch/ |
| Disallow | /findingaids/search* |
| Disallow | /findingaids/browse* |
| Disallow | /findingaids/lilly |
| Disallow | /findingaids/archives |
| Disallow | /findingaids/wmi |
| Disallow | /vwwp/search* |
| Disallow | /vwwp/browse* |
| Disallow | /inauthors/search* |
| Disallow | /inauthors/browse* |
| Disallow | /brevier/* |
| Disallow | /TEIgeneral/search* |
| Disallow | /TEIgeneral/browse* |
| Disallow | /archivesphotos/search/ |
| Disallow | /archivesphotos/selections.do |
| Disallow | /archivesphotos/requestmaster.do |
| Disallow | /hohenberger/selections.do |
| Disallow | /imagesearch/ |
| Disallow | /filmstudyguides/ |
| Disallow | /dcs/ |
| Disallow | /dcs* |
| Disallow | /biblio |
| Disallow | /bibliocat |
googlebot
| Rule | Path |
|---|---|
| Disallow | /fli/ |
| Disallow | /cushman/browse/ |
| Disallow | /cushman/search/ |
| Disallow | /cushman/results/result.do |
| Disallow | /cushman/overview/ordering.jsp |
| Disallow | /collections/dido/ |
| Disallow | /confluence/ |
| Disallow | /sheetmusic/devincent.do |
| Disallow | /newton/search |
| Disallow | /newton/mss |
| Disallow | /metsnav/ |
| Disallow | /metsnav2/ |
| Disallow | /metsnav3/ |
| Disallow | /metsnav3service/ |
| Disallow | /inharmony/search.do |
| Disallow | /imagesearch/ |
| Disallow | /findingaids/search* |
| Disallow | /findingaids/browse* |
| Disallow | /findingaids/lilly |
| Disallow | /findingaids/archives |
| Disallow | /findingaids/wmi |
| Disallow | /vwwp/search* |
| Disallow | /vwwp/browse* |
| Disallow | /inauthors/search* |
| Disallow | /inauthors/browse* |
| Disallow | /TEIgeneral/search* |
| Disallow | /TEIgeneral/browse* |
| Disallow | /brevier/search* |
| Disallow | /brevier/browse* |
| Disallow | /archivesphotos/search/ |
| Disallow | /archivesphotos/selections.do |
| Disallow | /archivesphotos/requestmaster.do |
| Disallow | /hohenberger/selections.do |
| Disallow | /imagesearch/ |
| Disallow | /filmstudyguides/ |
| Disallow | /dcs/ |
| Disallow | /dcs* |
| Disallow | /biblio |
| Disallow | /bibliocat |
googlebot/2.1
| Rule | Path |
|---|---|
| Disallow | /fli/ |
| Disallow | /cushman/browse/ |
| Disallow | /cushman/search/ |
| Disallow | /cushman/results/result.do |
| Disallow | /cushman/overview/ordering.jsp |
| Disallow | /collections/dido/ |
| Disallow | /confluence/ |
| Disallow | /sheetmusic/devincent.do |
| Disallow | /newton/search |
| Disallow | /newton/mss |
| Disallow | /metsnav/ |
| Disallow | /metsnav2/ |
| Disallow | /metsnav3/ |
| Disallow | /metsnav3service/ |
| Disallow | /inharmony/search.do |
| Disallow | /imagesearch/ |
| Disallow | /findingaids/search* |
| Disallow | /findingaids/browse* |
| Disallow | /findingaids/lilly |
| Disallow | /findingaids/archives |
| Disallow | /findingaids/wmi |
| Disallow | /vwwp/search* |
| Disallow | /vwwp/browse* |
| Disallow | /inauthors/search* |
| Disallow | /inauthors/browse* |
| Disallow | /brevier/search* |
| Disallow | /brevier/browse* |
| Disallow | /TEIgeneral/search* |
| Disallow | /TEIgeneral/browse* |
| Disallow | /archivesphotos/search/ |
| Disallow | /archivesphotos/selections.do |
| Disallow | /archivesphotos/requestmaster.do |
| Disallow | /hohenberger/selections.do |
| Disallow | /imagesearch/ |
| Disallow | /filmstudyguides/ |
| Disallow | /dcs/ |
| Disallow | /dcs* |
| Disallow | /biblio |
| Disallow | /bibliocat |
googlebot-image
| Rule | Path |
|---|---|
| Disallow | /fli/ |
| Disallow | /cushman/browse/ |
| Disallow | /cushman/search/ |
| Disallow | /cushman/results/result.do |
| Disallow | /cushman/overview/ordering.jsp |
| Disallow | /collections/dido/ |
| Disallow | /confluence/ |
| Disallow | /sheetmusic/devincent.do |
| Disallow | /newton/search |
| Disallow | /newton/mss |
| Disallow | /metsnav/ |
| Disallow | /metsnav2/ |
| Disallow | /metsnav3/ |
| Disallow | /metsnav3service/ |
| Disallow | /inharmony/search.do |
| Disallow | /imagesearch/ |
| Disallow | /findingaids/search* |
| Disallow | /findingaids/browse* |
| Disallow | /findingaids/lilly |
| Disallow | /findingaids/archives |
| Disallow | /findingaids/wmi |
| Disallow | /vwwp/search* |
| Disallow | /vwwp/browse* |
| Disallow | /inauthors/search* |
| Disallow | /inauthors/browse* |
| Disallow | /brevier/search* |
| Disallow | /brevier/browse* |
| Disallow | /TEIgeneral/search* |
| Disallow | /TEIgeneral/browse* |
| Disallow | /archivesphotos/search/ |
| Disallow | /archivesphotos/selections.do |
| Disallow | /archivesphotos/requestmaster.do |
| Disallow | /hohenberger/selections.do |
| Disallow | /imagesearch/ |
| Disallow | /filmstudyguides/ |
| Disallow | /dcs/ |
| Disallow | /dcs* |
| Disallow | /biblio |
| Disallow | /bibliocat |
iu-crawler
| Rule | Path |
|---|---|
| Disallow | /cushman/search/ |
| Disallow | /cushman/browse/ |
| Disallow | /cushman/results/ |
| Disallow | /cushman/overview/ordering.jsp |
| Disallow | /collections/cushman/ |
| Disallow | /fli/ |
| Disallow | /collections/dido/ |
| Disallow | /confluence/ |
| Disallow | /sheetmusic/devincent.do |
| Disallow | /newton/search |
| Disallow | /newton/mss |
| Disallow | /metsnav/ |
| Disallow | /metsnav2/ |
| Disallow | /metsnav3/ |
| Disallow | /metsnav3service/ |
| Disallow | /inharmony/search.do |
| Disallow | /imagesearch/ |
| Disallow | /findingaids/search* |
| Disallow | /findingaids/browse* |
| Disallow | /findingaids/lilly |
| Disallow | /findingaids/archives |
| Disallow | /findingaids/wmi |
| Disallow | /vwwp/search* |
| Disallow | /vwwp/browse* |
| Disallow | /inauthors/search* |
| Disallow | /inauthors/browse* |
| Disallow | /brevier/search* |
| Disallow | /brevier/browse* |
| Disallow | /TEIgeneral/search* |
| Disallow | /TEIgeneral/browse* |
| Disallow | /archivesphotos/search/ |
| Disallow | /archivesphotos/selections.do |
| Disallow | /archivesphotos/requestmaster.do |
| Disallow | /hohenberger/selections.do |
| Disallow | /imagesearch/ |
| Disallow | /filmstudyguides/ |
| Disallow | /dcs/ |
| Disallow | /dcs* |
| Disallow | /biblio |
| Disallow | /bibliocat |
applebot/0.1
| Rule | Path |
|---|---|
| Disallow | /fli/ |
| Disallow | /cushman/browse/ |
| Disallow | /cushman/search/ |
| Disallow | /cushman/results/result.do |
| Disallow | /cushman/overview/ordering.jsp |
| Disallow | /collections/dido/ |
| Disallow | /confluence/ |
| Disallow | /sheetmusic/devincent.do |
| Disallow | /newton/search |
| Disallow | /newton/mss |
| Disallow | /metsnav/ |
| Disallow | /metsnav2/ |
| Disallow | /metsnav3/ |
| Disallow | /metsnav3service/ |
| Disallow | /inharmony/search.do |
| Disallow | /imagesearch/ |
| Disallow | /findingaids/search* |
| Disallow | /findingaids/browse* |
| Disallow | /findingaids/lilly |
| Disallow | /findingaids/archives |
| Disallow | /findingaids/wmi |
| Disallow | /vwwp/search* |
| Disallow | /vwwp/browse* |
| Disallow | /inauthors/search* |
| Disallow | /inauthors/browse* |
| Disallow | /brevier/search* |
| Disallow | /brevier/browse* |
| Disallow | /TEIgeneral/search* |
| Disallow | /TEIgeneral/browse* |
| Disallow | /archivesphotos/search/ |
| Disallow | /archivesphotos/selections.do |
| Disallow | /archivesphotos/requestmaster.do |
| Disallow | /hohenberger/selections.do |
| Disallow | /imagesearch/ |
| Disallow | /filmstudyguides/ |
| Disallow | /dcs/ |
| Disallow | /dcs* |
| Disallow | /biblio |
| Disallow | /bibliocat |
*
| Rule | Path |
|---|---|
| Disallow | /cushman/search/ |
| Disallow | /cushman/browse/ |
| Disallow | /cushman/results/ |
| Disallow | /cushman/overview/ordering.jsp |
| Disallow | /collections/cushman/ |
| Disallow | /fli/ |
| Disallow | /collections/dido/ |
| Disallow | /confluence/ |
| Disallow | /sheetmusic/devincent.do |
| Disallow | /newton/search |
| Disallow | /newton/mss |
| Disallow | /metsnav/ |
| Disallow | /metsnav2/ |
| Disallow | /metsnav3/ |
| Disallow | /metsnav3service/ |
| Disallow | /inharmony/search.do |
| Disallow | /imagesearch/ |
| Disallow | /findingaids |
| Disallow | /vwwp |
| Disallow | /archivesphotos/search/ |
| Disallow | /archivesphotos/selections.do |
| Disallow | /archivesphotos/requestmaster.do |
| Disallow | /hohenberger/selections.do |
| Disallow | /imagesearch/ |
| Disallow | /filmstudyguides/ |
| Disallow | /inauthors |
| Disallow | /brevier/* |
| Disallow | /TEIgeneral |
| Disallow | /dcs/ |
| Disallow | /dcs* |
| Disallow | /images/search.htm |
| Disallow | /images/contact.htm |
| Disallow | /biblio |
| Disallow | /bibliocat |
Comments