aspiringminds.com
robots.txt

Robots Exclusion Standard data for aspiringminds.com

Resource Scan

Scan Details

Site Domain aspiringminds.com
Base Domain aspiringminds.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2025-08-23T03:18:05+00:00
Next Scan 2025-09-22T03:18:05+00:00

Last Successful Scan

Scanned2025-07-02T03:00:58+00:00
URL https://www.aspiringminds.com/robots.txt
Redirect https://www.shl.com/robots.txt?redirect_source=aspiringminds.com
Redirect Domain www.shl.com
Redirect Base shl.com
Domain IPs 18.168.109.89
Redirect IPs 13.35.37.109, 13.35.37.113, 13.35.37.28, 13.35.37.62, 2600:9000:213e:1200:18:efed:aa40:93a1, 2600:9000:213e:7c00:18:efed:aa40:93a1, 2600:9000:213e:8400:18:efed:aa40:93a1, 2600:9000:213e:a00:18:efed:aa40:93a1, 2600:9000:213e:b600:18:efed:aa40:93a1, 2600:9000:213e:d600:18:efed:aa40:93a1, 2600:9000:213e:e00:18:efed:aa40:93a1, 2600:9000:213e:fe00:18:efed:aa40:93a1
Response IP 13.35.37.109
Found Yes
Hash 68ca9481d4a5eb299897bd57ef154d4fcd747b6cbe19a1aa92908e5dfcd949c6
SimHash 2789da072313

Groups

*

Rule Path
Allow /
Disallow /admin
Disallow /dev
Disallow /?elqTrackId
Disallow /?start
Disallow /?Team
Disallow /?cid
Disallow /?l=en_US
Disallow /?keyword
Disallow /contactUs.html?
Disallow /viewArticle.html?
Disallow /view.php?id
Disallow /category.html?
Disallow /categories.html?
Disallow /browserCheck.html?
Disallow /schedule.html?
Disallow /assets/logos/logo-amazon-gray.png
Disallow /assets/logos/logo-jnj-gray.png
Disallow /assets/logos/logo-astrazeneca-gray.png
Disallow /assets/logos/logo-capital-one-gray.png
Disallow /assets/logos/logo-tmobile-gray.png
Disallow /assets/logos/logo-ibm-gray.png
Disallow /assets/logos/logo-british-airways-gray-v2.png
Disallow /assets/logos/logo-boa-gray.png
Disallow /assets/logos/logo-ecolab-gray.png
Disallow /assets/logos/logo-under-armour-gray.png
Disallow /assets/logos/logo-stanley-bnd-gray.png
Disallow /assets/logos/logo-deutsche-bank-gray.png
Disallow /assets/logos/logo-caterpillar-gray.png
Disallow /assets/logos/logo-vodafone-gray.png
Disallow /assets/logos/logo-ul-solutions-gray.png
Disallow /assets/logos/logo-honeywell-gray.png
Disallow /assets/logos/logo-barclays-gray.png
Disallow /assets/logos/logo-adp-gray.png
Disallow /assets/logos/logo-oracle-gold-partner-gray.png
Disallow /assets/logos/logo-workday-gray.png
Disallow /assets/logos/logo-icims-gray.png
Disallow /assets/logos/logo-cornerstone-gray.png
Disallow /assets/logos/logo-smartrecruiters-gray.png

Other Records

Field Value
crawl-delay 5

Other Records

Field Value
sitemap https://www.shl.com/sitemap.xml

Comments

  • Robots.txt
  • Enables robots.txt rules for all crawlers
  • Rate limit Yahoo!, Bing and Yandex (Google ignores this)
  • SilverStripe sitemap: