glassdoor.it
robots.txt

Robots Exclusion Standard data for glassdoor.it

Resource Scan

Scan Details

Site Domain glassdoor.it
Base Domain glassdoor.it
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-28T15:32:46+00:00
Next Scan 2024-12-27T15:32:46+00:00

Last Successful Scan

Scanned2023-03-09T05:28:31+00:00
URL https://glassdoor.it/robots.txt
Redirect https://www.glassdoor.it/robots.txt
Redirect Domain www.glassdoor.it
Redirect Base glassdoor.it
Domain IPs 104.17.19.76, 104.17.20.76
Redirect IPs 104.17.19.76, 104.17.20.76
Response IP 104.17.19.76
Found Yes
Hash 05dac964835e324bedae4d73d600e9dcf121f43b19484ab52864b48b308d96ad
SimHash 0291f9c845f3

Groups

*

Rule Path
Disallow /*?*hostSite=*
Disallow /1347171559/
Disallow /about/board/
Disallow /about/contact/
Disallow /about/faq/
Disallow /about/forCareerCenters/
Disallow /about/forLibraries/
Disallow /about/forStudents/
Disallow /about/guidelines/
Disallow /about/index/
Disallow /about/jobs/
Disallow /about/learn/
Disallow /about/overview/
Disallow /about/privacy/
Disallow /about/privacy/
Disallow /about/syndicationCenter/
Disallow /about/team/
Disallow /about/terms/
Disallow /about/widgetTerms/
Disallow /ajax/
Disallow /abtest
Disallow /browse/
Disallow /employerinfo/
Disallow /employerInfo/
Disallow /Cerca/cerca-aziende
Disallow /getAdSlotContentsAjax.htm
Disallow /home/
Disallow /integrations/facebook/glassdoor/eep
Disallow /jobview/
Disallow /legal/
Disallow /lists/
Disallow /more/
Disallow /partner/
Disallow /partner-center/
Disallow /partners/company/
Disallow /partners/insights/
Disallow /partners/jobs/
Disallow /partners/reports/
Disallow /partners/resumeView
Disallow /partners/settings/
Disallow /parts
Disallow /profile/
Allow /profile/login_input.htm
Allow /profile/joinNow_input.htm
Disallow /Resume/user-profile/
Disallow /rss/*
Disallow /Rungs/
Disallow /search/
Disallow /Search/
Disallow /survey/
Disallow /surveys
Disallow /util/
Disallow /getAdSlotContentsAjax.htm
Disallow /developer/widget/builder/
Disallow /hammer/
Disallow /mz-survey/
Disallow /user-activation/
Disallow /member/
Disallow /resume/build/
Disallow /userprofile/
Disallow /sourcing$
Disallow /searchsuggest$
Disallow /knowyourworth/
Disallow /Recensioni/index.htm?
Disallow */lib$
Disallow */lib/
Disallow */globalize/
Disallow */globalize$
Disallow */ASCIISumThreshold$
Disallow */LogClient$
Disallow */MsgBuilder$
Disallow */UserAgent$
Disallow */Constants$
Disallow */init/
Disallow */init$
Disallow */LogServer$
Disallow */GDLogger$
Disallow */gd-perf$
Disallow */gd-site-hdr-dropdown$
Disallow */bundles$
Disallow */wait$
Disallow */extend$
Disallow */strings$
Disallow */strings/
Disallow */document$
Disallow */*Ajax.htm
Disallow */json$
Disallow */json/
Disallow /Confronta/scegli
Disallow /employers/ec
Disallow /slink.htm
Disallow /*encryptedUserId
Disallow /*followId
Disallow /*userValidationKey
Disallow */trackClickAsync.htm
Disallow /track
Disallow /job-listing/details.htm?*
Disallow /job-listing/*_IE*.htm
Disallow /job-listing/JV.htm?*
Disallow /Lavoro/*_IP*
Disallow /Lavori/*_P*.htm*
Disallow /Lavori/*_IP*.htm*
Disallow /Recensioni/*_P*.htm*
Disallow /Recensioni/*_IP*.htm*
Allow /Recensioni/*-recensioni-SRCH_*_IP2.htm*
Disallow /Colloquio/*_P*.htm*
Disallow /Colloquio/*_IP*.htm*
Disallow /Benefit/*_IP*.htm*
Disallow /Stipendi/*_IP*.htm*
Allow /Stipendi/*_IP2.htm*
Allow /Stipendi/*_IP3.htm*
Allow /Stipendi/*_IP4.htm*
Allow /Stipendi/*_IP5.htm*
Disallow /1060761/*

ia_archiver

Rule Path
Disallow /
Allow */index.htm

omniexplorer_bot

Rule Path
Disallow /

mediapartners-google

Rule Path
Allow /

baiduspider

Rule Path
Disallow /
Allow */index.htm

Comments

  • Italy
  • logging related
  • Blocking track urls (ACQ-2468)
  • Blocking non standard job view and job search URLs, and paginated job SERP URLs (TRFC-2831)
  • Blocking pagination on employer infosite
  • Blocking bots from crawling DoubleClick for Publisher and Google Analytics related URL's (which aren't real URL's)