maersk.com
robots.txt

Robots Exclusion Standard data for maersk.com

Resource Scan

Scan Details

Site Domain maersk.com
Base Domain maersk.com
Scan Status Ok
Last Scan2024-11-06T04:47:04+00:00
Next Scan 2024-11-20T04:47:04+00:00

Last Scan

Scanned2024-11-06T04:47:04+00:00
URL https://maersk.com/robots.txt
Redirect https://www.maersk.com/robots.txt
Redirect Domain www.maersk.com
Redirect Base maersk.com
Domain IPs 23.215.7.12, 23.215.7.30
Redirect IPs 23.215.7.12, 23.215.7.30
Response IP 23.215.7.12
Found Yes
Hash c13086ff41ca637272b45d0f5bfe6c724c17e54ee12895cc190095dfd679d360
SimHash c18ec850c012

Groups

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

baiduspider

Rule Path
Disallow /

yandex

Rule Path
Allow /tracking/$
Allow /tracking/air/$
Allow /tracking/assets/
Allow /tracking/lcl/$
Allow /tracking/parcel$
Disallow /careers/vacancies/
Disallow /schedules/*%3D
Disallow /search*%3D
Disallow /share-via
Disallow /tracking/*
Disallow /vacancies/

*

Rule Path
Allow /*-*/careers/technology
Allow /insights/filter$
Allow /logistics-explained/filter$
Allow /onlinequote/$
Allow /onlinequote/assets/
Allow /portaluser/login$
Allow /portaluser/register$
Allow /tracking/$
Allow /tracking/air/$
Allow /tracking/assets/
Allow /tracking/lcl/$
Allow /tracking/parcel$
Allow /de-de/local-information/europe/contract-logistics/*/bremen
Allow /de-de/local-information/europe/contract-logistics/*/bremerhaven
Allow /de-de/local-information/europe/contract-logistics/*/duisburg
Allow /de-de/local-information/europe/contract-logistics/*/hamburg
Allow /de-de/local-information/europe/contract-logistics/*/hanover
Allow /es-mx/local-information/europe/contract-logistics/*/barcelona
Allow /es-mx/local-information/europe/contract-logistics/*/madrid
Allow /tr-tr/local-information/europe/contract-logistics/*/eskisehir
Allow /tr-tr/local-information/europe/contract-logistics/*/istanbul
Disallow /*-*/about/executive-leadership-team
Disallow /*-*/about/our-history
Disallow /*-*/beta
Disallow /*-*/brexit
Disallow /*-*/careers
Disallow /*-*/growth
Disallow /*-*/insights/tags/
Disallow /*-*/live-events
Disallow /*-*/local-information/europe/contract-logistics
Disallow /*-*/logistics-explained$
Disallow /*-*/onlinequote
Disallow /*-*/portaluser
Disallow /*-*/procurement
Disallow /*-*/schedules
Disallow /*-*/search
Disallow /*-*/share-via
Disallow /*-*/support/case
Disallow /*-*/support/chat
Disallow /*-*/tracking
Disallow /*.doc
Disallow /*.ppt
Disallow /*.xls
Disallow /*/filter
Disallow /*/news/tags
Disallow /*/treatment/
Disallow /*cHash%3D
Disallow /*elqTrackId%3D
Disallow /*exit%3D
Disallow /*link.aspx_id%3D
Disallow /*locationId%3D
Disallow /*originalUrl%3D
Disallow /*tx_
Disallow /*url%3D
Disallow /about/cfs
Disallow /about/do-copy
Disallow /about/edo
Disallow /acm/
Disallow /akam/
Disallow /allocations
Disallow /beta
Disallow /billoflading
Disallow /book
Disallow /captcha
Disallow /careers/vacancies?
Disallow /careers/vacancies/sv
Disallow /cargo-insurance
Disallow /commodity/
Disallow /common/
Disallow /dashboard
Disallow /demurrage
Disallow /doLogin
Disallow /dqcs
Disallow /en/careers
Disallow /es-ES
Disallow /essential/location
Disallow /fixedSpotBooking
Disallow /flow/
Disallow /forms/
Disallow /go$
Disallow /hub
Disallow /ibooking
Disallow /ign/
Disallow /import/
Disallow /inlandpricelookup
Disallow /instantPrice
Disallow /intermodal/
Disallow /lcl
Disallow /liveEngage
Disallow /loggedin
Disallow /logistics-explained/newsletter
Disallow /logistics-explained/tags/
Disallow /Maersk/Home/
Disallow /multilanguageapp/
Disallow /my-customs
Disallow /myfinance
Disallow /mymaersk
Disallow /news/category/*?
Disallow /news/category/planned-maintenance
Disallow /news/category/website-enhancement
Disallow /nl-NL
Disallow /onlinequote/*
Disallow /payer
Disallow /pdf/
Disallow /ping
Disallow /portallang
Disallow /portaluser
Disallow /pt-pt
Disallow /pt-PT
Disallow /public
Disallow /ratelookup/
Disallow /ru-ru/insights
Disallow /ru-ru/local-information
Disallow /ru-ru/logistics-explained
Disallow /ru-ru/news
Disallow /ru-ru/transportation-services/air-freight
Disallow /rules-lookup/
Disallow /schedules/*%3D
Disallow /scp/
Disallow /search*%3D
Disallow /security/
Disallow /share-via
Disallow /shipment
Disallow /shipping
Disallow /sitecore
Disallow /subscriptions/
Disallow /support/chat/?
Disallow /tasks/
Disallow /tracking/*
Disallow /user/
Disallow /vacancies/
Disallow /vessels
Disallow /web/
Disallow /webuser

Other Records

Field Value
sitemap https://www.maersk.com/sitemap.xml
sitemap https://www.maersk.com/sitemap-cs.xml
sitemap https://www.maersk.com/sitemap-is.xml

Comments

  • Want to be part of the next big idea?
  • Maersk.ai
  • Uncomment to block AI scrapers
  • User-agent: Applebot-Extended
  • User-agent: Bytespider
  • User-agent: CCBot
  • User-agent: ClaudeBot
  • User-agent: Diffbot
  • User-agent: FacebookBot
  • User-agent: Google-Extended
  • User-agent: GPTBot
  • User-agent: Meta-ExternalAgent
  • User-agent: omgili
  • User-agent: Timpibot
  • Disallow: /

Warnings

  • `clean-param` is not a known field.