traders.lt
robots.txt
Robots Exclusion Standard data for traders.lt
Resource Scan
Scan Details
| Site Domain | traders.lt |
| Base Domain | traders.lt |
| Scan Status | Ok |
| Last Scan | 2024-06-29T06:49:18+00:00 |
| Next Scan | 2024-07-06T06:49:18+00:00 |
Last Scan
| Scanned | 2024-06-29T06:49:18+00:00 |
| URL | https://traders.lt/robots.txt |
| Redirect | https://www.traders.lt/robots.txt |
| Redirect Domain | www.traders.lt |
| Redirect Base | traders.lt |
| Domain IPs | 185.81.164.77 |
| Redirect IPs | 185.81.164.77 |
| Response IP | 185.81.164.77 |
| Found | Yes |
| Hash | dc3b56ca34a6fae465830242aa9d7188424beb7c0fb7a39f985f0cc26e25388b |
| SimHash | a46488144168 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /cgi-bin |
| Disallow | /datas/ads |
| Disallow | /datas/downloads |
| Disallow | /datas/tmp |
| Disallow | /datas/videos |
| Disallow | /datas/thumbs |
| Disallow | /plugins |
| Disallow | /robots |
| Disallow | /system |
| Disallow | /ta |
| Disallow | /wap |
| Disallow | /openx |
| Disallow | /fncharts |
| Disallow | /jpgraph |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |