physorg.com
robots.txt
Robots Exclusion Standard data for physorg.com
Resource Scan
Scan Details
| Site Domain | physorg.com |
| Base Domain | physorg.com |
| Scan Status | Ok |
| Last Scan | 2024-11-11T22:31:12+00:00 |
| Next Scan | 2024-11-18T22:31:12+00:00 |
Last Scan
| Scanned | 2024-11-11T22:31:12+00:00 |
| URL | http://physorg.com/robots.txt |
| Redirect | https://phys.org/robots.txt |
| Redirect Domain | phys.org |
| Redirect Base | phys.org |
| Domain IPs | 72.251.236.55 |
| Redirect IPs | 2001:48c8:13:5::52, 72.251.233.232 |
| Response IP | 72.251.233.232 |
| Found | Yes |
| Hash | 86b733568072ef75a2f2e787bf79bf0dec8c63bfaeb219bbd0869d24b9badadd |
| SimHash | 701c5843e0e3 |
Groups
Other Records
| Field | Value |
|---|---|
| sitemap | https://phys.org/sitemap/indx/ |
Warnings
- `​crawl-delay` is not a known field.