Log In Sign Up

marktplaats.nl
robots.txt

Robots Exclusion Standard data for marktplaats.nl

Resource Scan

Scan Details

Site Domain marktplaats.nl
Base Domain marktplaats.nl
Scan Status Ok
Last Scan2024-05-06T19:08:41+00:00
Next Scan 2024-05-07T19:08:41+00:00

Last Scan

Scanned2024-05-06T19:08:41+00:00
URL https://www.marktplaats.nl/robots.txt
Domain IPs 18.155.192.22, 18.155.192.4, 18.155.192.45, 18.155.192.47
Response IP 18.165.171.38
Found Yes
Hash 9d891fa0f122dc2e8867475dbc72802af36919a03dde666fea374c5e1ada3a62
SimHash 2c892a1a8776

Groups

*

Rule Path
Disallow /experiences/*
Disallow /*/asq/
Disallow /4282/mpnl.*
Disallow /LR/*
Disallow /4282/web/mpnl.*
Disallow /messages*
Disallow /notifications*
Disallow /plaats/*
Disallow /i/adinfo/*
Disallow /admanager/*
Disallow /u/*/*/q/*
Disallow /u/*/*/l/*
Disallow /u/*/*/f/*
Disallow /account/wachtwoordvergeten.html
Disallow /account/wachtwoordinstellen.html
Disallow /account/password-reset/initiate.html
Disallow /account/password-reset/confirm.html
Disallow /account/confirm.html
Disallow /account/login.html?target=*
Disallow /account/createAccount.html
Disallow /account/loginSuccess.html
Disallow /mymp/
Disallow /asq.html
Disallow /asq
Disallow /syi/
Disallow /flag/
Disallow /bid/
Disallow /externalUrl/
Disallow /EVIP/*
Disallow /VIP/*
Disallow /L1PAGE/*
Disallow /L1.5PAGE/*
Disallow /*ebayimg.com/*/s/*
Disallow /metrics/
Disallow /wijzigAdvertentie/
Disallow /search?q=*
Disallow /widget/new-and-popular/click.html*
Disallow /lrp/api/audience-targeting
Disallow /px/fb*
Disallow /a/*/*/m*.html?c=*
Disallow /a/*/*/a*.html?c=*
Disallow /v/*/*/m*?c=*
Disallow /v/*/*/a*?c=*
Disallow /a/*/*/m*.html*correlationId%3D*
Disallow /a/*/*/a*.html*correlationId%3D*
Disallow /v/*/*/m*correlationId%3D*
Disallow /v/*/*/a*correlationId%3D*
Disallow /?type=*
Disallow /kijkinuwwijk.html?m_i=*
Disallow */undefined*
Back to top

Other Records

Field Value
sitemap https://www.marktplaats.nl/sitemap/sitemap.xml
Back to top

Comments

  • Here is our sitemap (this line is independent of UA blocks, per the spec)
  • Please keep blocking of all URLs in place for at least 2 years after removing a specific module
  • internal URLs
  • SOI subpage
  • login, confirm and forgot password pages
  • mymp pages
  • ASQ pages
  • SYI Pages
  • Flagging/tipping ads
  • bidding on ads
  • external url redirects
  • google analytics
  • korean spam
  • widgets
  • prevent unnecessary crawling
  • Block VIPs with parameters
  • block homepage feeds
  • block kijkinuwwijk parameters
  • block undefined URLs
Back to top