bons-plans-bonnes-affaires.fr
robots.txt

Robots Exclusion Standard data for bons-plans-bonnes-affaires.fr

Resource Scan

Scan Details

Site Domain bons-plans-bonnes-affaires.fr
Base Domain bons-plans-bonnes-affaires.fr
Scan Status Ok
Last Scan2024-06-28T23:46:47+00:00
Next Scan 2024-07-05T23:46:47+00:00

Last Scan

Scanned2024-06-28T23:46:47+00:00
URL https://bons-plans-bonnes-affaires.fr/robots.txt
Domain IPs 2a02:2350:5:10a:8022:52c5:bd37:3c0, 46.30.213.36
Response IP 46.30.213.36
Found Yes
Hash 832fa066996a3d100fff0f1d7bdfe0e623e788cb2d340f4755c6e4593ce062ed
SimHash 4510d3126711

Groups

scrapy

Rule Path
Allow /

ia_archiver disallow: /

No rules defined. All paths allowed.