glassdoor.com.mx
robots.txt

Robots Exclusion Standard data for glassdoor.com.mx

Resource Scan

Scan Details

Site Domain glassdoor.com.mx
Base Domain glassdoor.com.mx
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-24T07:10:06+00:00
Next Scan 2024-12-23T07:10:06+00:00

Last Successful Scan

Scanned2023-03-04T13:46:27+00:00
URL https://glassdoor.com.mx/robots.txt
Redirect https://www.glassdoor.com.mx/robots.txt
Redirect Domain www.glassdoor.com.mx
Redirect Base glassdoor.com.mx
Domain IPs 104.18.84.75, 104.18.85.75
Redirect IPs 104.18.84.75, 104.18.85.75
Response IP 104.18.84.75
Found Yes
Hash c8752cec30bec32f3f12e456c676903b5a1986e5e277a509ad79b5cc787b321c
SimHash 1a91f90a45db

Groups

*

Rule Path
Disallow /*?*hostSite=*
Disallow /1347171559/
Disallow /about/board/
Disallow /about/contact/
Disallow /about/faq/
Disallow /about/forCareerCenters/
Disallow /about/forLibraries/
Disallow /about/forStudents/
Disallow /about/guidelines/
Disallow /about/index/
Disallow /about/jobs/
Disallow /about/learn/
Disallow /about/overview/
Disallow /about/privacy/
Disallow /about/privacy/
Disallow /about/syndicationCenter/
Disallow /about/team/
Disallow /about/terms/
Disallow /about/widgetTerms/
Disallow /ajax/
Disallow /abtest
Disallow /browse/
Disallow /employerinfo/
Disallow /employerInfo/
Disallow /Explorar/buscar-empresas
Disallow /getAdSlotContentsAjax.htm
Disallow /home/
Disallow /integrations/facebook/glassdoor/eep
Disallow /jobview/
Disallow /legal/
Disallow /lists/
Disallow /more/
Disallow /partner/
Disallow /partner-center/
Disallow /partners/company/
Disallow /partners/insights/
Disallow /partners/jobs/
Disallow /partners/reports/
Disallow /partners/resumeView
Disallow /partners/settings/
Disallow /parts
Disallow /profile/
Allow /profile/login_input.htm
Allow /profile/joinNow_input.htm
Disallow /Resume/user-profile/
Disallow /rss/*
Disallow /Rungs/
Disallow /search/
Disallow /Search/
Disallow /survey/
Disallow /surveys
Disallow /util/
Disallow /getAdSlotContentsAjax.htm
Disallow /developer/widget/builder/
Disallow /hammer/
Disallow /mz-survey/
Disallow /user-activation/
Disallow /member/
Disallow /resume/build/
Disallow /userprofile/
Disallow /sourcing$
Disallow /searchsuggest$
Disallow /knowyourworth/
Disallow /Evaluaciones/index.htm?
Disallow */lib$
Disallow */lib/
Disallow */globalize/
Disallow */globalize$
Disallow */ASCIISumThreshold$
Disallow */LogClient$
Disallow */MsgBuilder$
Disallow */UserAgent$
Disallow */Constants$
Disallow */init/
Disallow */init$
Disallow */LogServer$
Disallow */GDLogger$
Disallow */gd-perf$
Disallow */gd-site-hdr-dropdown$
Disallow */bundles$
Disallow */wait$
Disallow */extend$
Disallow */strings$
Disallow */strings/
Disallow */document$
Disallow */*Ajax.htm
Disallow */json$
Disallow */json/
Disallow /Compara/elegir
Disallow /employers/ec
Disallow /slink.htm
Disallow /*encryptedUserId
Disallow /*followId
Disallow /*userValidationKey
Disallow */trackClickAsync.htm
Disallow /track
Disallow /job-listing/details.htm?*
Disallow /job-listing/*_IE*.htm
Disallow /job-listing/JV.htm?*
Disallow /Empleo/*_IP*
Disallow /Empleos/*_P*.htm*
Disallow /Empleos/*_IP*.htm*
Disallow /Evaluaciones/*_P*.htm*
Disallow /Evaluaciones/*_IP*.htm*
Allow /Evaluaciones/*-evaluaciones-SRCH_*_IP2.htm*
Disallow /Entrevista/*_P*.htm*
Disallow /Entrevista/*_IP*.htm*
Disallow /Prestaciones/*_IP*.htm*
Disallow /Sueldos/*_IP*.htm*
Allow /Sueldos/*_IP2.htm*
Allow /Sueldos/*_IP3.htm*
Allow /Sueldos/*_IP4.htm*
Allow /Sueldos/*_IP5.htm*
Disallow /1060761/*

ia_archiver

Rule Path
Disallow /
Allow */index.htm

omniexplorer_bot

Rule Path
Disallow /

mediapartners-google

Rule Path
Allow /

baiduspider

Rule Path
Disallow /
Allow */index.htm

Comments

  • France
  • logging related
  • Blocking track urls (ACQ-2468)
  • Blocking non standard job view and job search URLs, and paginated job SERP URLs (TRFC-2831)
  • Blocking pagination on employer infosite
  • Blocking bots from crawling DoubleClick for Publisher and Google Analytics related URL's (which aren't real URL's)