bel-bo.be
robots.txt
Robots Exclusion Standard data for bel-bo.be
Resource Scan
Scan Details
| Site Domain | bel-bo.be |
| Base Domain | bel-bo.be |
| Scan Status | Ok |
| Last Scan | 2024-11-02T01:14:06+00:00 |
| Next Scan | 2024-12-02T01:14:06+00:00 |
Last Scan
| Scanned | 2024-11-02T01:14:06+00:00 |
| URL | https://bel-bo.be/robots.txt |
| Redirect | https://www.bel-bo.be/robots.txt |
| Redirect Domain | www.bel-bo.be |
| Redirect Base | bel-bo.be |
| Domain IPs | 2001:4860:4802:32::15, 2001:4860:4802:34::15, 2001:4860:4802:36::15, 2001:4860:4802:38::15, 216.239.32.21, 216.239.34.21, 216.239.36.21, 216.239.38.21 |
| Redirect IPs | 2404:6800:4003:c0f::79, 74.125.68.121 |
| Response IP | 142.251.10.121 |
| Found | Yes |
| Hash | e9998a292dc241510b5e93d5a151b2c050386ef60b4e00103ad5331db98655e3 |
| SimHash | 64327d53c7f3 |
Groups
*
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
*
| Rule | Path |
|---|---|
| Allow | /*?page= |
| Allow | /*?p= |
| Allow | /*html?$ |
| Disallow | /index.php/ |
| Disallow | /*? |
| Disallow | /*?page=*& |
| Disallow | /*?p=& |
| Disallow | /checkout/ |
| Disallow | /app/ |
| Disallow | /lib/ |
| Disallow | /*.php$ |
| Disallow | /pkginfo/ |
| Disallow | /report/ |
| Disallow | /var/ |
| Disallow | /catalog/ |
| Disallow | /customer/ |
| Disallow | /sendfriend/ |
| Disallow | /review/ |
| Disallow | /*SID%3D |
| Disallow | /sales/ |
| Disallow | /mydata/ |
| Disallow | /graphql |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.bel-bo.be/media/sitemap/sitemap_fr_be_index.xml |
| sitemap | https://www.bel-bo.be/media/sitemap/sitemap_nl_be_index.xml |
Comments