alleyinsider.com
robots.txt
Robots Exclusion Standard data for alleyinsider.com
Resource Scan
Scan Details
Site Domain | alleyinsider.com |
Base Domain | alleyinsider.com |
Scan Status | Ok |
Last Scan | 2024-05-17T09:38:12+00:00 |
Next Scan | 2024-05-24T09:38:12+00:00 |
Last Scan
Scanned | 2024-05-17T09:38:12+00:00 |
URL | http://alleyinsider.com/robots.txt |
Redirect | https://www.businessinsider.com/robots.txt |
Redirect Domain | www.businessinsider.com |
Redirect Base | businessinsider.com |
Domain IPs | 151.101.0.64, 151.101.128.64, 151.101.192.64, 151.101.64.64 |
Redirect IPs | 151.101.1.171, 151.101.129.171, 151.101.193.171, 151.101.65.171 |
Response IP | 199.232.45.171 |
Found | Yes |
Hash | ab5b3458202bd32b79d30232a6113ce46ccb0fcf7de384ce9976e51e357bf823 |
SimHash | 8343c04deb5f |
Groups
*
Rule | Path |
---|---|
Disallow | /*?utm_campaign=Monitor& |
Disallow | /adframe |
Disallow | /afp$ |
Disallow | /ajax/ |
Disallow | /archives |
Disallow | /associated-press$ |
Disallow | /authentication$ |
Disallow | /author/*/date |
Disallow | /author/*/mostread |
Disallow | /categories |
Disallow | /cms/ |
Disallow | /comments$ |
Disallow | /cross-domain$ |
Disallow | /document/ |
Disallow | /fonts-async.css |
Disallow | /insider$ |
Disallow | /partner/ |
Disallow | /reuters$ |
Disallow | /reviews/out? |
Disallow | /s? |
Disallow | /guides/s? |
Disallow | /personal-finance/s? |
Disallow | /search$ |
Disallow | /track.gif |
Disallow | /uk$ |
Disallow | /ws/ |
Disallow | /.well-known/amphtml/apikey.pub |
Disallow | /business-insider$ |
Disallow | /*/contributor |
Disallow | /bi$ |
Disallow | /news-insider$ |
Other Records
Field | Value |
---|---|
sitemap | https://www.businessinsider.com/sitemap/latest.xml |
sitemap | https://www.businessinsider.com/sitemap/google-news.xml |
sitemap | https://www.businessinsider.com/sitemap/index.xml |
sitemap | https://www.businessinsider.com/sitemap/landing-pages.xml |
sitemap | https://www.businessinsider.com/sitemap/insider/migrated.xml |
sitemap | https://feeds.businessinsider.com/custom/tech-sj-sitemap |
sitemap | https://feeds.businessinsider.com/custom/pfi-sitemap |