taylorfrancis.com
robots.txt
Robots Exclusion Standard data for taylorfrancis.com
Resource Scan
Scan Details
| Site Domain | taylorfrancis.com |
| Base Domain | taylorfrancis.com |
| Scan Status | Ok |
| Last Scan | 2025-10-20T10:27:53+00:00 |
| Next Scan | 2025-11-19T10:27:53+00:00 |
Last Scan
| Scanned | 2025-10-20T10:27:53+00:00 |
| URL | https://taylorfrancis.com/robots.txt |
| Redirect | https://www.taylorfrancis.com/robots.txt |
| Redirect Domain | www.taylorfrancis.com |
| Redirect Base | taylorfrancis.com |
| Domain IPs | 104.18.33.121, 172.64.154.135 |
| Redirect IPs | 104.18.33.121, 172.64.154.135 |
| Response IP | 172.64.154.135 |
| Found | Yes |
| Hash | c252e18ba6c2d480730db6c7c3479e6e7b1f2c295958c940593970de8450c26b |
| SimHash | 01154156c511 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /articles* |
| Disallow | /search* |
| Disallow | /content/ |
| Disallow | /cdn-cgi/ |
| Disallow | /contentsitemap*.xml$ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.taylorfrancis.com/sitemap.xml |
| sitemap | https://www.taylorfrancis.com/contentsitemap.xml |