nyu.edu
robots.txt
Robots Exclusion Standard data for nyu.edu
Resource Scan
Scan Details
| Site Domain | nyu.edu |
| Base Domain | nyu.edu |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-04-25T07:48:36+00:00 |
| Next Scan | 2024-07-24T07:48:36+00:00 |
Last Successful Scan
| Scanned | 2023-12-05T07:14:16+00:00 |
| URL | https://nyu.edu/robots.txt |
| Redirect | https://www.nyu.edu/robots.txt |
| Redirect Domain | www.nyu.edu |
| Redirect Base | nyu.edu |
| Domain IPs | 216.165.47.10 |
| Redirect IPs | 18.155.68.111, 18.155.68.33, 18.155.68.68, 18.155.68.77, 2600:9000:23d2:2400:1:f7e2:cb00:93a1, 2600:9000:23d2:4800:1:f7e2:cb00:93a1, 2600:9000:23d2:6000:1:f7e2:cb00:93a1, 2600:9000:23d2:9000:1:f7e2:cb00:93a1, 2600:9000:23d2:9600:1:f7e2:cb00:93a1, 2600:9000:23d2:c000:1:f7e2:cb00:93a1, 2600:9000:23d2:dc00:1:f7e2:cb00:93a1, 2600:9000:23d2:e400:1:f7e2:cb00:93a1 |
| Response IP | 13.32.145.104 |
| Found | Yes |
| Hash | 9d977f80b595cd6fb35af84f580ee009a2c931aabe85bf997c9c2565369c6c25 |
| SimHash | 66044b92c20b |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /about/leadership-university-administration/board-of-trustees/newsletter/ |
| Disallow | /alumni/the-hub/ |
| Disallow | /bursar/ |
| Disallow | /dental/dh/ |
| Disallow | /dental/endo/ |
| Disallow | /dental/perio/ |
| Disallow | /dental/histo/ |
| Disallow | /dental/students/ |
| Disallow | /dental/intranet/ |
| Disallow | /fast-facts/ |
| Disallow | /maps/ |
| Disallow | /nyucard/campuscash/admin/ |
| Disallow | /public.affairs/gata/ |
| Disallow | /residential.education/archive/ |
| Disallow | /residential.education/templates/ |
| Disallow | /sustainability/ |
| Disallow | /wagner/newsite/ |
| Disallow | /wagner/newsiteTEST/ |
| Disallow | /*? |
googlebot-image
| Rule | Path |
|---|---|
| Disallow | /about/leadership-university-administration/board-of-trustees/newsletter/ |
| Disallow | /alumni/the-hub/ |
| Disallow | /fast-facts/ |
| Disallow | /*? |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.nyu.edu/sitemap.xml |