presearch.com
robots.txt
Robots Exclusion Standard data for presearch.com
Resource Scan
Scan Details
| Site Domain | presearch.com |
| Base Domain | presearch.com |
| Scan Status | Ok |
| Last Scan | 2024-11-05T11:37:45+00:00 |
| Next Scan | 2024-11-12T11:37:45+00:00 |
Last Scan
| Scanned | 2024-11-05T11:37:45+00:00 |
| URL | https://presearch.com/robots.txt |
| Domain IPs | 13.229.27.62, 18.136.117.30, 18.139.20.247 |
| Response IP | 18.139.20.247 |
| Found | Yes |
| Hash | a0a7569329d37b3f521207fa7797437487e300bd564466077441f7beea13260b |
| SimHash | 68505c32eac1 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /search |
| Disallow | /*?q= |
| Disallow | /images |
| Disallow | /videos |
| Disallow | /news |
| Disallow | /shopping |
| Disallow | /results |
| Disallow | /ai-results |
| Disallow | /background-url/ |
| Disallow | /travel |
| Disallow | /map |
| Disallow | /*?ref= |
| Disallow | /*?session_id= |
| Disallow | /*?page= |
| Disallow | /*.pdf$ |
| Disallow | /*.jpg$ |
| Disallow | /*.png$ |
| Disallow | /*.css$ |
| Disallow | /*.js$ |
| Disallow | /*.json$ |
| Disallow | /*.csv$ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 20 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.presearch.com/sitemap.xml |
Comments