psychcentral.com
robots.txt
Robots Exclusion Standard data for psychcentral.com
Resource Scan
Scan Details
| Site Domain | psychcentral.com |
| Base Domain | psychcentral.com |
| Scan Status | Ok |
| Last Scan | 2024-05-17T13:31:15+00:00 |
| Next Scan | 2024-05-24T13:31:15+00:00 |
Last Scan
| Scanned | 2024-05-17T13:31:15+00:00 |
| URL | https://psychcentral.com/robots.txt |
| Domain IPs | 18.155.68.100, 18.155.68.119, 18.155.68.129, 18.155.68.21 |
| Response IP | 18.155.68.119 |
| Found | Yes |
| Hash | 14e629660dfc00ce98cb39ca56c012947fd36087c3520516e496f3943f8e24f0 |
| SimHash | b125da64cc71 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /lib/psy-* |
| Disallow | /blog/psy-* |
| Disallow | /disorders/psy-* |
| Disallow | /drugs/psy-* |
| Disallow | /quizzes/psy-* |
| Disallow | /program/psy-* |
| Disallow | /health/psych-central-block-sampler |
| Disallow | /test/ |
| Disallow | /api/* |
| Disallow | */null$ |
| Disallow | */inline$ |
| Disallow | /*?s= |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://psychcentral.com/sitemap.xml |
Comments