Log In Sign Up

babyoutlet.nl
robots.txt

Robots Exclusion Standard data for babyoutlet.nl

Resource Scan

Scan Details

Site Domain babyoutlet.nl
Base Domain babyoutlet.nl
Scan Status Ok
Last Scan2024-06-14T02:26:52+00:00
Next Scan 2024-06-21T02:26:52+00:00

Last Scan

Scanned2024-06-14T02:26:52+00:00
URL https://babyoutlet.nl/robots.txt
Redirect https://www.babyoutlet.nl/robots.txt
Redirect Domain www.babyoutlet.nl
Redirect Base babyoutlet.nl
Domain IPs 104.26.4.250, 104.26.5.250, 172.67.75.93, 2606:4700:20::681a:4fa, 2606:4700:20::681a:5fa, 2606:4700:20::ac43:4b5d
Redirect IPs 104.26.4.250, 104.26.5.250, 172.67.75.93, 2606:4700:20::681a:4fa, 2606:4700:20::681a:5fa, 2606:4700:20::ac43:4b5d
Response IP 172.67.75.93
Found Yes
Hash 96520bf6f4abf1b15d3071674a92f0f6c1cfcc8919b9dc11d648ea6e75a61f44
SimHash 3803b213c491

Groups

*

Rule Path
Allow /

baiduspider

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

fasterfox

Rule Path
Disallow /

ingrid

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

jyxobot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

yahoo! slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2
Back to top

Other Records

Field Value
sitemap https://www.babyoutlet.nl/sitemap.xml
Back to top

Comments

  • Hi! Trying to reverse engineer something?
  • Maybe you should come work with us.
  • Apply at www.atmospherecms.nl and mention this comment.
Back to top