careers.sh
robots.txt

Robots Exclusion Standard data for careers.sh

Resource Scan

Scan Details

Site Domain careers.sh
Base Domain careers.sh
Scan Status Ok
Last Scan2025-11-30T19:16:26+00:00
Next Scan 2025-12-14T19:16:26+00:00

Last Scan

Scanned2025-11-30T19:16:26+00:00
URL https://careers.sh/robots.txt
Domain IPs 104.26.0.74, 104.26.1.74, 172.67.69.195, 2606:4700:20::681a:14a, 2606:4700:20::681a:4a, 2606:4700:20::ac43:45c3
Response IP 104.26.0.74
Found Yes
Hash 850d3ba68e9aa30051d2de06d25320116f25549e7d261cf4a2ec665a7f642c20
SimHash 28001d17fa00

Groups

*

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

semrushbot

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

googlebot
adsbot-google

Rule Path
Allow /

Other Records

Field Value
sitemap https://careers.sh/pro-sitemaps-4157371.php?sn=sitemap.xml.gz