academia-assets.com
robots.txt
Robots Exclusion Standard data for academia-assets.com
Resource Scan
Scan Details
| Site Domain | academia-assets.com |
| Base Domain | academia-assets.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-08-05T12:20:42+00:00 |
| Next Scan | 2024-11-03T12:20:42+00:00 |
Last Successful Scan
| Scanned | 2023-09-10T05:06:10+00:00 |
| URL | https://www.academia-assets.com/robots.txt |
| Domain IPs | 13.227.254.104, 13.227.254.123, 13.227.254.45, 13.227.254.93 |
| Response IP | 52.222.174.95 |
| Found | Yes |
| Hash | 6a17667efb7a610c053a66f2c46fa5d8bc20dacbc7656f3f178d3c268e08606d |
| SimHash | 62154310c595 |
Groups
googlebot
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
mediapartners-google
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
msnbot
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
bingbot
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
slurp
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
duckduckbot
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
baiduspider
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
seznambot
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
naverbot
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
teoma
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
yandex
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
scoutjet
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
gurujibot
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
exabot
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
solbot
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
fast
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
voilabot
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
daumoa
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
architextspider
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
twitterbot
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
socscibot
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
ia_archiver
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
coccoc
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
browsershots
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
sitearc
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
cliqzbot
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
proximic
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
oncrawlbot
| Rule | Path |
|---|---|
| Disallow | /attachments/*/download_file |
| Disallow | /Jobs/Search |
| Disallow | /people/search |
| Disallow | /ac_search |
| Disallow | /users/search |
| Disallow | /ac_universities |
| Disallow | /ac_university_departments |
| Disallow | /v0/arbitrary_event |
| Disallow | /v0/server_notifications |
| Disallow | /v0/impressions |
| Disallow | /v0/ab_test |
| Disallow | /v0/onsite_pings |
| Allow | / |
*
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://sitemap.academia.edu/works_index.xml |
Comments