Log In Sign Up

resto.fr
robots.txt

Robots Exclusion Standard data for resto.fr

Resource Scan

Scan Details

Site Domain resto.fr
Base Domain resto.fr
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-05-03T13:16:32+00:00
Next Scan 2024-07-02T13:16:32+00:00

Last Successful Scan

Scanned2024-02-11T13:15:17+00:00
URL https://resto.fr/robots.txt
Redirect https://en.resto.fr/robots.txt
Redirect Domain en.resto.fr
Redirect Base resto.fr
Domain IPs 13.225.233.129
Redirect IPs 2600:9000:21f8:2c00:14:5716:68c0:93a1, 2600:9000:21f8:3a00:14:5716:68c0:93a1, 2600:9000:21f8:3c00:14:5716:68c0:93a1, 2600:9000:21f8:8400:14:5716:68c0:93a1, 2600:9000:21f8:a800:14:5716:68c0:93a1, 2600:9000:21f8:b400:14:5716:68c0:93a1, 2600:9000:21f8:e600:14:5716:68c0:93a1, 2600:9000:21f8:f400:14:5716:68c0:93a1, 3.160.188.39, 3.160.188.5, 3.160.188.51, 3.160.188.56
Response IP 99.86.4.127
Found Yes
Hash 39d08f3f47a10c1417e727ce7a2b9cd106ca3b9c3a848339c193722166aab94c
SimHash b111c0d6a8b8

Groups

*

Rule Path
Disallow /business/search
Disallow /business/search/
Disallow /business/feedback
Disallow /business/feedback/
Disallow /business/like
Disallow /business/like/
Disallow /business/add
Disallow /business/add/
Disallow /business/unlike
Disallow /business/unlike/
Disallow /restaurant/like
Disallow /restaurant/like/
Disallow /restaurant/unlike
Disallow /restaurant/unlike/
Disallow /restaurant/preview/cms
Disallow /restaurant/preview/cms/
Disallow /logBusinessLinkInterceptor
Disallow /logTelephoneInterceptor
Disallow /image/upload
Disallow /image/upload/
Disallow /image/remove
Disallow /image/remove/
Disallow /image/crop
Disallow /image/crop/
Disallow /error
Disallow /error/

Other Records

Field Value
crawl-delay 1
Back to top

Warnings

  • 2 invalid lines.
Back to top