Log In Sign Up

glassdoor.sg
robots.txt

Robots Exclusion Standard data for glassdoor.sg

Resource Scan

Scan Details

Site Domain glassdoor.sg
Base Domain glassdoor.sg
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-24T11:46:11+00:00
Next Scan 2024-12-23T11:46:11+00:00

Last Successful Scan

Scanned2023-03-04T15:54:12+00:00
URL https://glassdoor.sg/robots.txt
Redirect https://www.glassdoor.sg/robots.txt
Redirect Domain www.glassdoor.sg
Redirect Base glassdoor.sg
Domain IPs 104.16.94.86, 104.16.95.86
Redirect IPs 104.16.94.86, 104.16.95.86
Response IP 104.16.94.86
Found Yes
Hash cbe50e2411ce53256fec277ca1e7f4a9577e62abcb14e0ce3aa0933ddedc7f67
SimHash 1b9e310945d3

Groups

*

Rule Path
Disallow /*?*hostSite=*
Disallow /1347171559/
Disallow /about/board/
Disallow /about/contact/
Disallow /about/faq/
Disallow /about/forCareerCenters/
Disallow /about/forLibraries/
Disallow /about/forStudents/
Disallow /about/guidelines/
Disallow /about/index/
Disallow /about/jobs/
Disallow /about/learn/
Disallow /about/overview/
Disallow /about/privacy/
Disallow /about/privacy/
Disallow /about/syndicationCenter/
Disallow /about/team/
Disallow /about/terms/
Disallow /about/widgetTerms/
Disallow /ajax/
Disallow /abtest
Disallow /browse/
Disallow /employerinfo/
Disallow /employerInfo/
Disallow /Explore/browse-companies
Disallow /getAdSlotContentsAjax.htm
Disallow /home/
Disallow /integrations/facebook/glassdoor/eep
Disallow /jobview/
Disallow /legal/
Disallow /lists/
Disallow /more/
Disallow /partner/
Disallow /partner-center/
Disallow /partners/company/
Disallow /partners/insights/
Disallow /partners/jobs/
Disallow /partners/reports/
Disallow /partners/resumeView
Disallow /partners/settings/
Disallow /parts
Disallow /profile/
Allow /profile/login_input.htm
Allow /profile/joinNow_input.htm
Disallow /Resume/user-profile/
Disallow /rss/*
Disallow /Rungs/
Disallow /search/
Disallow /Search/
Disallow /survey/
Disallow /surveys
Disallow /util/
Disallow /getAdSlotContentsAjax.htm
Disallow /developer/widget/builder/
Disallow /hammer/
Disallow /employers/ec
Disallow /slink.htm
Disallow /*encryptedUserId
Disallow /*followId
Disallow /*userValidationKey
Disallow */trackClickAsync.htm
Disallow /mz-survey/
Disallow /user-activation/
Disallow /member/
Disallow /resume/build/
Disallow /userprofile/
Disallow /sourcing$
Disallow /searchsuggest$
Disallow /knowyourworth/
Disallow /Reviews/index.htm?
Disallow */lib$
Disallow */lib/
Disallow */globalize/
Disallow */globalize$
Disallow */ASCIISumThreshold$
Disallow */LogClient$
Disallow */MsgBuilder$
Disallow */UserAgent$
Disallow */Constants$
Disallow */init/
Disallow */init$
Disallow */LogServer$
Disallow */GDLogger$
Disallow */gd-perf$
Disallow */gd-site-hdr-dropdown$
Disallow */bundles$
Disallow */wait$
Disallow */extend$
Disallow */strings$
Disallow */strings/
Disallow */document$
Disallow */*Ajax.htm
Disallow */json$
Disallow */json/
Disallow /Compare/choose
Disallow /track
Disallow /job-listing/details.htm?*
Disallow /job-listing/*_IE*.htm
Disallow /job-listing/JV.htm?*
Disallow /Job/*_IP*
Disallow /Jobs/*_P*.htm*
Disallow /Jobs/*_IP*.htm*
Disallow /Reviews/*_P*.htm*
Disallow /Reviews/*_IP*.htm*
Allow /Reviews/*-reviews-SRCH_*_IP2.htm*
Disallow /Interview/*_P*.htm*
Disallow /Interview/*_IP*.htm*
Disallow /Benefits/*_IP*.htm*
Disallow /Salaries/*_IP*.htm*
Allow /Salaries/*_IP2.htm*
Allow /Salaries/*_IP3.htm*
Allow /Salaries/*_IP4.htm*
Allow /Salaries/*_IP5.htm*
Disallow /1060761/*
Disallow /Reviews/Barbizon-scam-*

ia_archiver

Rule Path
Disallow /
Allow */index.htm

omniexplorer_bot

Rule Path
Disallow /

mediapartners-google

Rule Path
Allow /

baiduspider

Rule Path
Disallow /
Allow */index.htm
Back to top

Comments

  • Singapore
  • Greetings, human beings!,
  • If you're sniffing around this file, and you're not a robot, we're looking to meet curious folks such as yourself.
  • Think you have what it takes to join the best white-hat SEO growth hackers on the planet, and help improve the way people everywhere find jobs?
  • Run - don't crawl - to apply to join Glassdoor's SEO team here http://jobs.glassdoor.com
  • logging related
  • Blocking track urls (ACQ-2468)
  • Blocking non standard job view and job search URLs, and paginated job SERP URLs (TRFC-2831)
  • Blocking pagination on employer infosite
  • Blocking bots from crawling DoubleClick for Publisher and Google Analytics related URL's (which aren't real URL's)
  • TRFC-4037 Block page from being indexed
  • Note that this file has the extension '.text' rather than the more-standard '.txt'
  • to keep it from being pre-compiled as a servlet. (*.txt files are precompiled, and
  • there doesn't seem to be a way to turn this off.)
Back to top