dexpens.com
robots.txt
Robots Exclusion Standard data for dexpens.com
Resource Scan
Scan Details
| Site Domain | dexpens.com |
| Base Domain | dexpens.com |
| Scan Status | Ok |
| Last Scan | 2025-09-20T07:16:58+00:00 |
| Next Scan | 2025-09-27T07:16:58+00:00 |
Last Scan
| Scanned | 2025-09-20T07:16:58+00:00 |
| URL | https://dexpens.com/robots.txt |
| Redirect | https://www.dexpens.com:443/robots.txt |
| Redirect Domain | www.dexpens.com |
| Redirect Base | dexpens.com |
| Domain IPs | 40.118.101.67 |
| Redirect IPs | 40.118.101.67 |
| Response IP | 40.118.101.67 |
| Found | Yes |
| Hash | 3eb04d9c5b4980d031fb7734006854f629381f4dfb9da4725390bd6a697c1d9d |
| SimHash | cf95e260c9d1 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /Admin/ |
| Disallow | /Settings/ |
| Disallow | /Account/ |
| Disallow | /Friends/ |
| Disallow | /Links/ |
| Disallow | /Home/ModalSend |
| Disallow | /SignUp/ |
| Disallow | /PhotoAlbums/DeleteUserImage |
| Disallow | /Storage/RotateImage |
| Disallow | /Notifications/ReadAll |
| Disallow | /Walls/SharePost |
| Disallow | /Walls/GetSharedPopup |
| Disallow | /Walls/GetSharedPopupForImage |
| Disallow | /Responses/ |
| Disallow | /Likes/ |
| Disallow | /Comments/ |
| Disallow | /Vote/ |
| Disallow | /BotDetectCaptcha.ashx/ |
| Disallow | /node/ |
| Disallow | /ckeditor/ |
| Disallow | /ckeditor5/ |
| Disallow | /Logs/ |
| Disallow | /Temp/ |
| Disallow | *%26crawl%3Dno |
| Disallow | /Cars/UserCars/image_src |
| Disallow | /Organisation/image_src |
| Disallow | /Articles/image_src |
| Disallow | /Articles/UserArticles/image_src |
| Disallow | /Groups/UserGroups/image_src |
| Disallow | /PhotoAlbum/image_src |
| Disallow | /Questions/UserQuestions/image_src |
| Disallow | /Article/image_src |
| Disallow | /Car/image_src |
| Disallow | /Cars/image_src |
| Disallow | /Images/ |
| Allow | /.well-known/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://dexpens.com/sitemap.xml |
Warnings
- `host` is not a known field.