Log In Sign Up

roberthalf.com
robots.txt

Robots Exclusion Standard data for roberthalf.com

Resource Scan

Scan Details

Site Domain roberthalf.com
Base Domain roberthalf.com
Scan Status Ok
Last Scan2024-11-06T19:08:51+00:00
Next Scan 2024-11-20T19:08:51+00:00

Last Scan

Scanned2024-11-06T19:08:51+00:00
URL https://roberthalf.com/robots.txt
Redirect https://www.roberthalf.com/robots.txt
Redirect Domain www.roberthalf.com
Redirect Base roberthalf.com
Domain IPs 3.165.102.106, 3.165.102.113, 3.165.102.30, 3.165.102.62
Redirect IPs 104.18.18.164, 104.18.19.164, 2606:4700::6812:12a4, 2606:4700::6812:13a4
Response IP 104.18.18.164
Found Yes
Hash f5d623fed6030d0ba32f5f9a49373fbb9f8c7eb8ec79d7ccb2f6d3d5a45a45db
SimHash 3ad3f209edd1

Groups

*

Rule Path
Disallow /cn/zh/*
Disallow /cn/en/*
Disallow /content/dam/roberthalf/documents/cn/zh/non-indexed/*
Disallow /content/dam/roberthalf/documents/cn/en/non-indexed/*
Disallow /web.config
Disallow /content/roberthalf.sitemap.cn-sitemap.xml
Disallow /content/roberthalf.sitemap.cn-external-job-details-sitemap.xml
Disallow /content/roberthalf.sitemap.cn-en-external-job-search-sitemap.xml
Disallow /content/roberthalf.sitemap.cn-en-external-role-detail-no-salary-sitemap.xml
Disallow /content/roberthalf.sitemap.cn-en-external-role-details-sitemap.xml
Disallow /content/roberthalf.sitemap.cn-zh-external-job-search-sitemap.xml
Disallow /content/roberthalf.sitemap.cn-zh-external-role-detail-no-salary-sitemap.xml
Disallow /content/roberthalf.sitemap.cn-zh-external-role-details-sitemap.xml
Disallow /us/en/insights/salary-guide/details/*
Disallow /us/en/insights/salary-guide-preview*
Disallow /us/en/job-details-preview*
Disallow /us/en/contact/search?param=*
Disallow /content/dam/roberthalf/documents/us/en/non-indexed/*
Disallow /ca/en/insights/salary-guide/details/*
Disallow /ca/en/insights/salary-guide-preview*
Disallow /ca/en/job-details-preview*
Disallow /ca/fr/perspectives/guide-salarial/details/*
Disallow /ca/fr/job-details-preview*
Disallow /ca/fr/perspectives/apercu-guide-salarial*
Disallow /ca/en/search?param=*
Disallow /ca/fr/recherche?param=*
Disallow /content/dam/roberthalf/documents/ca/en/non-indexed/*
Disallow /content/dam/roberthalf/documents/ca/fr/non-indexed/*
Disallow /insights/gehaltsuebersicht/job-details/*
Disallow /de/de/suche?param=*
Disallow /content/dam/roberthalf/documents/de/de/non-indexed/*
Disallow /gb/en/search?param=*
Disallow /gb/en/contact/search?param=*
Disallow /content/dam/roberthalf/documents/gb/en/non-indexed/*
Disallow /au/en/search?param=*
Disallow /au/en/contact/search?param=*
Disallow /content/dam/roberthalf/documents/au/en/non-indexed/*
Disallow /br/pt/search?param=*
Disallow /content/dam/roberthalf/documents/br/pt/non-indexed/*
Disallow /fr/fr/search?param=*
Disallow /content/dam/roberthalf/documents/fr/fr/non-indexed/*
Disallow /be/fr/search?param=*
Disallow /be/nl/search?param=*
Disallow /be/en/search?param=*
Disallow /content/dam/roberthalf/documents/be/en/non-indexed/*
Disallow /content/dam/roberthalf/documents/be/fr/non-indexed/*
Disallow /content/dam/roberthalf/documents/be/nl/non-indexed/*
Disallow /jp/en/search?param=*
Disallow /jp/ja/search?param=*
Disallow /content/dam/roberthalf/documents/jp/en/non-indexed/*
Disallow /content/dam/roberthalf/documents/jp/jp/non-indexed/*
Disallow /sg/en/search?param=*
Disallow /sg/en/contact/search?param=*
Disallow /content/dam/roberthalf/documents/sg/en/non-indexed/*
Disallow /nz/en/search?param=*
Disallow /nz/en/contact/search?param=*
Disallow /content/dam/roberthalf/documents/nz/en/non-indexed/*
Disallow /ae/en/search?param=*
Disallow /ae/en/contact/search?param=*
Disallow /content/dam/roberthalf/documents/ae/en/non-indexed/*
Disallow /cl/es/search?param=*
Disallow /cl/es/contact/search?param=*
Disallow /content/dam/roberthalf/documents/cl/es/non-indexed/*
Disallow /ch/en/search?param=*
Disallow /ch/en/contact/search?param=*
Disallow /ch/fr/search?param=*
Disallow /ch/fr/contact/search?param=*
Disallow /ch/de/search?param=*
Disallow /ch/de/contact/search?param=*
Disallow /content/dam/roberthalf/documents/ch/en/non-indexed/*
Disallow /content/dam/roberthalf/documents/ch/fr/non-indexed/*
Disallow /content/dam/roberthalf/documents/ch/de/non-indexed/*
Disallow /nl/en/search?param=*
Disallow /nl/en/contact/search?param=*
Disallow /nl/nl/search?param=*
Disallow /nl/nl/contact/search?param=*
Disallow /content/dam/roberthalf/documents/nl/en/non-indexed/*
Disallow /content/dam/roberthalf/documents/nl/nl/non-indexed/*
Disallow /hk/en/search?param=*
Disallow /hk/en/contact/search?param=*
Disallow /content/dam/roberthalf/documents/hk/en/non-indexed/*
Disallow /ie/en/search?param=*
Disallow /ie/en/contact/search?param=*
Disallow /content/dam/roberthalf/documents/ie/en/non-indexed/*
Disallow /lu/en/search?param=*
Disallow /lu/en/contact/search?param=*
Disallow /content/dam/roberthalf/documents/lu/en/non-indexed/*
Disallow /lu/fr/search?param=*
Disallow /lu/fr/contact/search?param=*
Disallow /content/dam/roberthalf/documents/lu/fr/non-indexed/*
Disallow /us/en/privacy/qualtrics
Disallow /ca/en/privacy/qualtrics
Disallow /ca/fr/confidentialite/qualtrics
Disallow /gb/en/privacy-notice/qualtrics
Disallow /au/en/privacy/qualtrics
Disallow /nz/en/privacy/qualtrics
Disallow /be/en/privacy/qualtrics
Disallow /be/fr/vie-privee/qualtrics
Disallow /be/nl/privacy/qualtrics
Disallow /nl/en/privacy/qualtrics
Disallow /nl/nl/privacy/qualtrics
Disallow /fr/fr/protection-donnees-personnelles/qualtrics
Disallow /de/de/datenschutzerklaerung/qualtrics
Disallow /ch/en/data-privacy-statement/qualtrics
Disallow /br/pt/privacidade/qualtrics
Disallow /ae/en/privacy-notice/qualtrics
Disallow /ch/de/datenschutzerklaerung/qualtrics
Disallow /ch/fr/clause-de-respect-de-la-vie-privee/qualtrics
Disallow /cl/es/politica-de-privacidad/qualtrics
Disallow /jp/ja/privacy/qualtrics
Disallow /cn/zh/privacy-statement/qualtrics
Disallow /cn/en/privacy-statement/qualtrics
Disallow /hk/en/privacy-statement/qualtrics
Disallow /sg/en/privacy/qualtrics
Disallow /ca/fr/privacy/qualtrics
Disallow /gb/en/privacy/qualtrics
Disallow /be/fr/privacy/qualtrics
Disallow /fr/fr/privacy/qualtrics
Disallow /de/de/privacy/qualtrics
Disallow /ch/en/privacy/qualtrics
Disallow /br/pt/privacy/qualtrics
Disallow /ae/en/privacy/qualtrics
Disallow /ch/de/privacy/qualtrics
Disallow /ch/fr/privacy/qualtrics
Disallow /cl/es/privacy/qualtrics
Disallow /ie/en/privacy-notice/qualtrics
Disallow /lu/en/privacy/qualtrics
Disallow /lu/fr/vie-privee/qualtrics
Disallow /cn/zh/privacy/qualtrics
Disallow /hk/en/privacy-statement/qualtrics
Disallow /ie/en/privacy/qualtrics
Disallow /hk/en/privacy/qualtrics
Back to top

Other Records

Field Value
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.us-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.us-en-external-job-search-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.us-en-external-role-detail-no-salary-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.us-external-job-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.us-en-external-role-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.ca-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.ca-external-job-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.ca-en-external-job-search-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.ca-en-external-role-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.ca-en-external-role-detail-no-salary-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.ca-fr-external-job-search-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.ca-fr-external-role-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.ca-fr-external-role-detail-no-salary-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.de-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.de-external-job-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.de-de-external-job-search-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.de-de-external-role-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.de-de-external-role-detail-no-salary-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.gb-en-external-job-search-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.gb-en-external-role-detail-no-salary-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.gb-en-external-role-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.gb-external-job-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.gb-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.au-en-external-job-search-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.au-en-external-role-detail-no-salary-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.au-en-external-role-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.au-external-job-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.au-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.br-pt-external-job-search-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.br-pt-external-role-detail-no-salary-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.br-pt-external-role-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.br-external-job-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.br-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.fr-fr-external-job-search-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.fr-fr-external-role-detail-no-salary-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.fr-fr-external-role-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.fr-external-job-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.fr-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.be-en-external-job-search-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.be-en-external-role-detail-no-salary-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.be-en-external-role-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.be-external-job-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.be-fr-external-job-search-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.be-fr-external-role-detail-no-salary-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.be-fr-external-role-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.be-nl-external-job-search-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.be-nl-external-role-detail-no-salary-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.be-nl-external-role-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.be-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.jp-ja-external-job-search-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.jp-ja-external-role-detail-no-salary-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.jp-ja-external-role-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.jp-en-external-job-search-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.jp-en-external-role-detail-no-salary-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.jp-en-external-role-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.jp-external-job-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.jp-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.sg-en-external-job-search-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.sg-en-external-role-detail-no-salary-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.sg-en-external-role-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.sg-external-job-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.sg-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.nz-en-external-job-search-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.nz-en-external-role-detail-no-salary-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.nz-en-external-role-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.nz-external-job-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.nz-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.ae-en-external-job-search-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.ae-en-external-role-detail-no-salary-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.ae-en-external-role-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.ae-external-job-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.ae-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.cl-es-external-job-search-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.cl-es-external-role-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.cl-external-job-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.cl-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.ch-en-external-job-search-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.ch-en-external-role-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.ch-de-external-job-search-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.ch-de-external-role-detail-no-salary-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.ch-de-external-role-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.ch-fr-external-job-search-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.ch-fr-external-role-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.ch-external-job-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.ch-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.nl-en-external-job-search-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.nl-en-external-role-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.nl-nl-external-job-search-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.nl-nl-external-role-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.nl-external-job-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.nl-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.hk-en-external-role-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.hk-en-external-job-search-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.hk-en-external-role-detail-no-salary-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.hk-external-job-details-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.hk-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.ie-sitemap.xml
sitemap https://www.roberthalf.com/content/roberthalf.sitemap.lu-sitemap.xml
Back to top

Comments

  • robots.txt
  • This file is to prevent the crawling and indexing of certain parts
  • of your site by web crawlers and spiders run by sites like Yahoo!
  • and Google. By telling these "robots" where not to go on your site,
  • you save bandwidth and server resources.
  • This file will be ignored unless it is at the root of your host:
  • Used: http://example.com/robots.txt
  • Ignored: http://example.com/site/robots.txt
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/robotstxt.html
  • Universal Rules
  • CSS, JS, Images
  • work with FED team for exceptions
  • Directories
  • Disallow CN
  • Files
  • Disallow CN Sitemaps
  • FED Team
  • SEO Team
  • US (.com/us/en) rules
  • XML sitemap
  • FED Team
  • SEO Team
  • CA (.com/ca/en|.com/ca/fr) rules
  • XML
  • FED Team
  • SEO Team
  • DE (.com/de/de) rules
  • XML sitemap
  • FED Team
  • SEO Team
  • UK (.com/gb) rules
  • XML sitemap
  • FED Team
  • SEO Team
  • AU (.com/au) rules
  • XML sitemap
  • FED Team
  • SEO Team
  • BR (.com/br) rules
  • XML sitemap
  • FED Team
  • SEO Team
  • FR (.com/FR) rules
  • XML sitemap
  • FED Team
  • SEO Team
  • BE (.com/be) rules
  • XML sitemap
  • FED Team
  • SEO Team
  • JP (.com/jp) rules
  • XML sitemap
  • FED Team
  • SEO Team
  • SG (.com/sg) rules
  • XML sitemap
  • FED Team
  • SEO Team
  • NZ (.com/NZ) rules
  • XML sitemap
  • FED Team
  • SEO Team
  • AE (.com/ae) rules
  • XML sitemap
  • FED Team
  • SEO Team
  • CL (.com/cl) rules
  • XML sitemap
  • FED Team
  • SEO Team
  • CH (.com/ch) rules
  • XML sitemap
  • FED Team
  • SEO Team
  • NL (.com/nl) rules
  • XML sitemap
  • FED Team
  • SEO Team
  • HK (.com/hk) rules
  • XML sitemap
  • FED Team
  • SEO Team
  • IE (.com/ie) rules
  • XML sitemap
  • FED Team
  • SEO Team
  • LU (.com/lu) rules
  • XML sitemap
  • FED Team
  • SEO Team
  • Qualtrix Group Blocks
Back to top