thespread.com
robots.txt
Robots Exclusion Standard data for thespread.com
Resource Scan
Scan Details
| Site Domain | thespread.com |
| Base Domain | thespread.com |
| Scan Status | Ok |
| Last Scan | 2024-11-15T20:56:58+00:00 |
| Next Scan | 2024-11-22T20:56:58+00:00 |
Last Scan
| Scanned | 2024-11-15T20:56:58+00:00 |
| URL | https://thespread.com/robots.txt |
| Domain IPs | 104.26.6.42, 104.26.7.42, 172.67.68.48, 2606:4700:20::681a:62a, 2606:4700:20::681a:72a, 2606:4700:20::ac43:4430 |
| Response IP | 104.26.6.42 |
| Found | Yes |
| Hash | 4ad9a9242c762f7f2bddd94cc557bbc8afa537648044d4fa2de54913311db6bf |
| SimHash | 893ce3624d71 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Allow | /*.js* |
| Allow | /*.css* |
| Allow | /*.png* |
| Allow | /*.jpg* |
| Allow | /*.gif* |
| Allow | /components/com_k2/ |
| Allow | /components/com_agorapro/assets/ |
| Disallow | /administrator/ |
| Disallow | /bin/ |
| Disallow | /cache/ |
| Disallow | /cli/ |
| Disallow | /includes/ |
| Disallow | /installation/ |
| Disallow | /language/ |
| Disallow | /layouts/ |
| Disallow | /libraries/ |
| Disallow | /logs/ |
| Disallow | /tmp/ |
| Disallow | /lines/admin/ |
| Disallow | /*?do_pdf=1* |
| Disallow | /*?tmpl=component* |
| Disallow | /cdn-cgi/ |
Other Records
| Field | Value |
|---|---|
| sitemap | http://www.thespread.com/sitemap.xml |