jbfo.nl
robots.txt

Robots Exclusion Standard data for jbfo.nl

Resource Scan

Scan Details

Site Domain jbfo.nl
Base Domain jbfo.nl
Scan Status Ok
Last Scan2025-10-11T22:42:10+00:00
Next Scan 2025-11-10T22:42:10+00:00

Last Scan

Scanned2025-10-11T22:42:10+00:00
URL https://jbfo.nl/robots.txt
Redirect https://www.jbfo.nl/robots.txt
Redirect Domain www.jbfo.nl
Redirect Base jbfo.nl
Domain IPs 104.21.53.208, 172.67.218.212, 2606:4700:3030::6815:35d0, 2606:4700:3031::ac43:dad4
Redirect IPs 104.21.53.208, 172.67.218.212, 2606:4700:3030::6815:35d0, 2606:4700:3031::ac43:dad4
Response IP 104.21.53.208
Found Yes
Hash bce1262d29ad3c8c32a33348509c35aeaeaf226e66ec26edca572b53cfb1fd65
SimHash 2c10dd310691

Groups

*

Rule Path
Disallow /apollo-graphql
Disallow /catalogsearch/

googlebot

Rule Path
Disallow /apollo-graphql
Disallow /catalogsearch/

googlebot-image

Rule Path
Disallow /apollo-graphql
Disallow /catalogsearch/

baiduspider

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

fasterfox

Rule Path
Disallow /

ingrid

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

jyxobot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

yahoo! slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

Other Records

Field Value
sitemap https://www.jbfo.nl/sitemap.xml

Comments

  • robots.txt
  • Website Sitemap
  • Crawlers Setup
  • Disallow baidu (chinse SEO)