Log In Sign Up

boels.com
robots.txt

Robots Exclusion Standard data for boels.com

Resource Scan

Scan Details

Site Domain boels.com
Base Domain boels.com
Scan Status Ok
Last Scan2024-06-07T22:51:46+00:00
Next Scan 2024-07-07T22:51:46+00:00

Last Scan

Scanned2024-06-07T22:51:46+00:00
URL https://boels.com/robots.txt
Redirect https://www.boels.com/robots.txt
Redirect Domain www.boels.com
Redirect Base boels.com
Domain IPs 20.105.224.24
Redirect IPs 13.107.246.59, 2620:1ec:bdf::59
Response IP 13.107.246.59
Found Yes
Hash dce5d3fda1250b17fecb6aabb15390f3a747f6d9337646e649d47ad64dcc5fb2
SimHash 49083f74f792

Groups

*

Rule Path
Disallow /admin/
Disallow /*?sortCode=name-asc
Disallow /*%26sortCode%3Dname-asc
Disallow /*?sortCode=relevance
Disallow /*%26sortCode%3Drelevance
Disallow /*?sortCode=name-desc
Disallow /*%26sortCode%3Dname-desc
Disallow /search/*/*
Disallow /index.php/*/*

blexbot

Rule Path
Disallow /

baidu spider

Rule Path
Disallow /

yandex bot

Rule Path
Disallow /

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

duckduckgo bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://www.boels.com/sitemap.xml
sitemap https://www.boels.com/NL_NL_sitemap.xml
sitemap https://www.boels.com/EN_NL_sitemap.xml
sitemap https://www.boels.com/NL_BE_sitemap.xml
sitemap https://www.boels.com/FR_BE_sitemap.xml
sitemap https://www.boels.com/EN_BE_sitemap.xml
sitemap https://www.boels.com/CS_CZ_sitemap.xml
sitemap https://www.boels.com/EN_CZ_sitemap.xml
sitemap https://www.boels.com/DE_DE_sitemap.xml
sitemap https://www.boels.com/EN_DE_sitemap.xml
sitemap https://www.boels.com/IT_IT_sitemap.xml
sitemap https://www.boels.com/EN_IT_sitemap.xml
sitemap https://www.boels.com/DE_LU_sitemap.xml
sitemap https://www.boels.com/FR_LU_sitemap.xml
sitemap https://www.boels.com/EN_LU_sitemap.xml
sitemap https://www.boels.com/HU_HU_sitemap.xml
sitemap https://www.boels.com/EN_HU_sitemap.xml
sitemap https://www.boels.com/DE_AT_sitemap.xml
sitemap https://www.boels.com/EN_AT_sitemap.xml
sitemap https://www.boels.com/PL_PL_sitemap.xml
sitemap https://www.boels.com/EN_PL_sitemap.xml
sitemap https://www.boels.com/DE_CH_sitemap.xml
sitemap https://www.boels.com/FR_CH_sitemap.xml
sitemap https://www.boels.com/IT_CH_sitemap.xml
sitemap https://www.boels.com/EN_CH_sitemap.xml
sitemap https://www.boels.com/SK_SK_sitemap.xml
sitemap https://www.boels.com/EN_SK_sitemap.xml
sitemap https://www.boels.com/EN_GB_sitemap.xml
Back to top

Comments

  • Paths (no clean URLs)
Back to top