api.glassdoor.com
robots.txt

Robots Exclusion Standard data for api.glassdoor.com

Resource Scan

Scan Details

Site Domain api.glassdoor.com
Base Domain glassdoor.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-05-23T15:12:13+00:00
Next Scan 2024-08-21T15:12:13+00:00

Last Successful Scan

Scanned2023-02-28T23:56:47+00:00
URL https://api.glassdoor.com/robots.txt
Domain IPs 104.17.90.51, 104.17.91.51
Response IP 104.17.90.51
Found Yes
Hash 532ad2dc7c8ac48244c1305e28645384f0476911a048ec2e1d607bed8aeb56d6
SimHash 6824db74ebf1

Groups

*

Rule Path
Disallow /

googlebot

Rule Path
Allow /api
Allow /api-internal

Comments

  • Disallow crawlers from the api.glassdoor.com subdomain