20220902t135734-dot-unipage-pro.ew.r.appspot.com
robots.txt
Robots Exclusion Standard data for 20220902t135734-dot-unipage-pro.ew.r.appspot.com
Resource Scan
Scan Details
| Site Domain | 20220902t135734-dot-unipage-pro.ew.r.appspot.com |
| Base Domain | 20220902t135734-dot-unipage-pro.ew.r.appspot.com |
| Scan Status | Ok |
| Last Scan | 2024-05-22T02:16:09+00:00 |
| Next Scan | 2024-05-29T02:16:09+00:00 |
Last Scan
| Scanned | 2024-05-22T02:16:09+00:00 |
| URL | https://20220902t135734-dot-unipage-pro.ew.r.appspot.com/robots.txt |
| Domain IPs | 2404:6800:4003:c02::99, 74.125.68.153 |
| Response IP | 64.233.170.153 |
| Found | Yes |
| Hash | c3aeeaf4771d537a854ce09a53ae980c5b02b9e1d3e560d32d68d026daa279cb |
| SimHash | 490881f00e42 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /message/* |
| Disallow | /document/* |
| Disallow | /user/* |
| Disallow | /debug/default/* |
| Disallow | /en/request/create |
| Disallow | /ru/request/create |
| Disallow | /en/user/signup?* |
| Disallow | /ru/user/signup?* |
| Disallow | /*/admin$ |
| Disallow | /*%26page%3D* |
| Disallow | /*?page=* |
| Disallow | /*%26sort%3D* |
| Disallow | /*?sort=* |
| Disallow | /*uni_sort%3D* |
| Disallow | /*%26per-page%3D* |
| Disallow | /*%26country%3D* |
| Disallow | /*%26major%3D* |
| Disallow | /*%26limit%3D* |
| Disallow | /*%26degree%3D* |
| Disallow | /*?__cf_chl_captcha_tk__=* |
| Disallow | /*%26__cf_chl_captcha_tk__%3D* |
| Disallow | /*?__cf_chl_tk=* |
| Disallow | /*%26__cf_chl_tk%3D* |
| Disallow | /*?__cf_chl_f_tk=* |
| Disallow | /*%26__cf_chl_f_tk%3D* |
| Disallow | /*uni_page%3D* |
| Disallow | /*?event=* |
| Disallow | /roll/* |
| Disallow | /page/update* |
| Disallow | /page/* |
| Disallow | /uni/* |
| Disallow | /institute/* |
| Disallow | /institution/* |
| Disallow | /site/* |
| Disallow | /block/* |
| Disallow | /about/* |
| Disallow | /program/* |
| Disallow | /course/* |
| Disallow | /user/* |
| Disallow | /account/* |
| Disallow | /favorites/* |
| Disallow | /request/* |
| Disallow | /company/* |
| Disallow | /ru/cities?* |
| Disallow | /en/cities?* |
| Disallow | /en/search?* |
| Disallow | /ru/search?* |
| Disallow | /ru/countries?* |
| Disallow | /en/countries?* |
| Disallow | /ru/courses?* |
| Disallow | /en/courses?* |
| Disallow | /ru/schools?* |
| Disallow | /en/schools?* |
| Disallow | /ru/universities/best/*?* |
| Disallow | /en/universities/best/*?* |
| Disallow | /universities?* |
| Disallow | /en/universities?* |
| Disallow | /ru/universities?* |
| Disallow | /ru/universities?city=* |
| Disallow | /ru/universities?country=* |
| Disallow | /ru/universities?uni_sort=* |
| Disallow | /ru/programs?* |
| Disallow | /en/programs?* |
| Disallow | /ru/program/* |
| Disallow | /en/program/* |
| Disallow | /ru/news?* |
| Allow | /roll/blocks?* |
mediapartners-google*
| Rule | Path |
|---|---|
| Allow | /en/universities?* |
| Allow | /ru/universities?* |
| Allow | /favorites/* |
| Allow | /ru/cities?* |
| Allow | /en/cities?* |
| Allow | /en/search?* |
| Allow | /ru/search?* |
| Allow | /ru/countries?* |
| Allow | /en/countries?* |
| Allow | /ru/courses?* |
| Allow | /en/courses?* |
| Allow | /ru/schools?* |
| Allow | /en/schools?* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.unipage.net/sitemap.xml |
| sitemap | https://www.unipage.net/cities_universities.xml |
| sitemap | https://www.unipage.net/institutes.xml |
| sitemap | https://www.unipage.net/institutes_ru.xml |
Warnings
- `host` is not a known field.
Comments