surfertoday.com
robots.txt
Robots Exclusion Standard data for surfertoday.com
Resource Scan
Scan Details
| Site Domain | surfertoday.com |
| Base Domain | surfertoday.com |
| Scan Status | Ok |
| Last Scan | 2024-11-04T17:05:09+00:00 |
| Next Scan | 2024-11-11T17:05:09+00:00 |
Last Scan
| Scanned | 2024-11-04T17:05:09+00:00 |
| URL | https://surfertoday.com/robots.txt |
| Redirect | https://www.surfertoday.com/robots.txt |
| Redirect Domain | www.surfertoday.com |
| Redirect Base | surfertoday.com |
| Domain IPs | 172.66.40.73, 172.66.43.183, 2606:4700:3108::ac42:2849, 2606:4700:3108::ac42:2bb7 |
| Redirect IPs | 172.66.40.73, 172.66.43.183, 2606:4700:3108::ac42:2849, 2606:4700:3108::ac42:2bb7 |
| Response IP | 172.66.40.73 |
| Found | Yes |
| Hash | ee617b4e952091bfe44bc87186e8156555c1e95227fa5ba54d1e886a0f271c09 |
| SimHash | 511cf511c808 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /administrator/ |
| Disallow | /api/ |
| Disallow | /cache/ |
| Disallow | /cli/ |
| Disallow | /components/ |
| Disallow | /includes/ |
| Disallow | /installation/ |
| Disallow | /language/ |
| Disallow | /layouts/ |
| Disallow | /libraries/ |
| Disallow | /modules/ |
| Disallow | /plugins/ |
| Disallow | /tmp/ |
| Disallow | /*? |
| Disallow | *search?q=* |
| Allow | /*.js* |
| Allow | /*.css* |
| Allow | /*.png* |
| Allow | /*.jpg* |
| Allow | /*.gif* |
| Allow | /plugins/system/jchoptimize/ |
| Allow | /plugins/system/jamp/core/images/placeholder.png |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.surfertoday.com/sitemap_xml.xml |