rajeshpahan.com
robots.txt
Robots Exclusion Standard data for rajeshpahan.com
Resource Scan
Scan Details
| Site Domain | rajeshpahan.com |
| Base Domain | rajeshpahan.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Couldn't connect to server. |
| Last Scan | 2024-09-14T23:01:44+00:00 |
| Next Scan | 2024-12-13T23:01:44+00:00 |
Last Successful Scan
| Scanned | 2023-06-29T19:36:56+00:00 |
| URL | https://rajeshpahan.com/robots.txt |
| Redirect | https://www.rajeshpahan.com/robots.txt |
| Redirect Domain | www.rajeshpahan.com |
| Redirect Base | rajeshpahan.com |
| Domain IPs | 216.239.32.21, 216.239.34.21, 216.239.36.21, 216.239.38.21 |
| Redirect IPs | 2404:6800:4003:c11::79, 74.125.200.121 |
| Response IP | 172.217.194.121 |
| Found | Yes |
| Hash | d299149861ea1f8bf8a1b2442c4c78ea5a217d95d5ee32dccf5078ed73cf6c2e |
| SimHash | 43049c217c53 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /search |
| Allow | / |
| Disallow | /https%3A//frases.rajeshpahan.com |
| Disallow | /https%3A//bomdia.rajeshpahan.com |
| Disallow | /https%3A//www.rajeshpahan.com/2020/07/what-do-bed-bugs-smell-like.html |
| Disallow | /https%3A//www.rajeshpahan.com/2020/07/what-do-bed-bugs-look-like-to-human-eye.html |
| Disallow | /https%3A//www.rajeshpahan.com/2020/07/bedbugs-live-in-your-hair.html |
| Disallow | /https%3A//www.rajeshpahan.com/2020/07/how-do-bed-bugs-affect-your-health.html |
| Disallow | /https%3A//www.rajeshpahan.com/2020/07/how-to-find-bed-bugs.html |
| Disallow | /https%3A//www.rajeshpahan.com/2020/07/do-bed-bugs-have-to-feed-to-survive.html |
| Disallow | /https%3A//www.rajeshpahan.com/2020/07/difference-between-bedbugs-and-scabies.html |
| Disallow | /https%3A//www.rajeshpahan.com/2020/07/what-color-are-bed-bugs.html |
| Allow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.rajeshpahan.com/atom.xml?redirect=false&start-index=1&max-results=500 |