whattoexpect.com
robots.txt
Robots Exclusion Standard data for whattoexpect.com
Resource Scan
Scan Details
| Site Domain | whattoexpect.com |
| Base Domain | whattoexpect.com |
| Scan Status | Ok |
| Last Scan | 2024-05-17T06:05:59+00:00 |
| Next Scan | 2024-05-24T06:05:59+00:00 |
Last Scan
| Scanned | 2024-05-17T06:05:59+00:00 |
| URL | https://whattoexpect.com/robots.txt |
| Redirect | https://www.whattoexpect.com:443/robots.txt |
| Redirect Domain | www.whattoexpect.com |
| Redirect Base | whattoexpect.com |
| Domain IPs | 34.236.180.160, 54.87.124.244 |
| Redirect IPs | 23.52.40.91, 23.52.40.97 |
| Response IP | 104.88.70.139 |
| Found | Yes |
| Hash | 95952d7e60b0dc310df55bc64b48c46c998ea3c0e022bb6ce39227981e901133 |
| SimHash | 8d090d5465d0 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /Index.aspx?puid=D16F18B4-73A8-48F5-B506-A9A4CE0119FF* |
| Disallow | /Index.aspx?puid=CFB00753-F97C-4587-966D-DE9ECE6BEBDC* |
| Disallow | /baby-pictures/ |
| Disallow | /destroy/ |
| Disallow | /post/ |
| Disallow | /infoheavy.html |
| Disallow | /imageheavy.html |
| Disallow | /?order= |
| Disallow | /?filter= |
| Disallow | /aolcat%3D |
| Disallow | /?RedirectUrl |
| Disallow | /?redirecturl |
| Disallow | /login/ |
| Disallow | /apple-app-site-association |
| Disallow | /archives/ |
| Disallow | /updateutp* |
| Disallow | /updateUtp* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.whattoexpect.com/sitemap.full.xml |
| sitemap | https://community.whattoexpect.com/forums/sitemap.xml |
| sitemap | https://community.whattoexpect.com/forums/recent.xml |
| sitemap | https://www.whattoexpect.com/news/sitemap.xml |
| sitemap | https://registry.whattoexpect.com/baby-registry/sitemap.xml |