dealbase.com
robots.txt
Robots Exclusion Standard data for dealbase.com
Resource Scan
Scan Details
| Site Domain | dealbase.com |
| Base Domain | dealbase.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Couldn't connect to server. |
| Last Scan | 2024-05-16T19:34:12+00:00 |
| Next Scan | 2024-05-30T19:34:12+00:00 |
Last Successful Scan
| Scanned | 2024-05-01T19:33:40+00:00 |
| URL | https://dealbase.com/robots.txt |
| Redirect | https://www.dealbase.com/robots.txt |
| Redirect Domain | www.dealbase.com |
| Redirect Base | dealbase.com |
| Domain IPs | 18.155.68.120, 18.155.68.35, 18.155.68.36, 18.155.68.74 |
| Redirect IPs | 18.155.68.120, 18.155.68.35, 18.155.68.36, 18.155.68.74, 2600:9000:23d2:1000:c:9fc7:f540:93a1, 2600:9000:23d2:200:c:9fc7:f540:93a1, 2600:9000:23d2:6200:c:9fc7:f540:93a1, 2600:9000:23d2:7c00:c:9fc7:f540:93a1, 2600:9000:23d2:8000:c:9fc7:f540:93a1, 2600:9000:23d2:ba00:c:9fc7:f540:93a1, 2600:9000:23d2:c400:c:9fc7:f540:93a1, 2600:9000:23d2:f800:c:9fc7:f540:93a1 |
| Response IP | 18.155.68.35 |
| Found | Yes |
| Hash | 7794f4cbeb7ca2eb2a5cb211286f9317fa37534a547ca45aac96ffdbb38d8d47 |
| SimHash | 04b79842e7b6 |
Groups
mediapartners-google
adsbot-google
adsbot-google-mobile
adidxbot
bingpreview
yandexdirect
yadirectfetcher
baiduspider-ads
seznambot
naverbot
sln24857
google-hoteladsverifier
googlebot
| Rule | Path |
|---|---|
| Disallow |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
*
| Rule | Path |
|---|---|
| Disallow | /CS/ |
| Disallow | /DA/ |
| Disallow | /ET/ |
| Disallow | /Hotel/ |
| Disallow | /ID/ |
| Disallow | /LV/ |
| Disallow | /PB/ |
| Disallow | /Place/ |
| Disallow | /RU/ |
| Disallow | /SK/ |
| Disallow | /TR/ |
Comments