Log In Sign Up

testclear.com
robots.txt

Robots Exclusion Standard data for testclear.com

Resource Scan

Scan Details

Site Domain testclear.com
Base Domain testclear.com
Scan Status Ok
Last Scan2025-04-19T05:46:44+00:00
Next Scan 2025-05-19T05:46:44+00:00

Last Scan

Scanned2025-04-19T05:46:44+00:00
URL https://testclear.com/robots.txt
Redirect https://www.testclear.com/robots.txt
Redirect Domain www.testclear.com
Redirect Base testclear.com
Domain IPs 172.66.40.130, 172.66.43.126, 2606:4700:3108::ac42:2882, 2606:4700:3108::ac42:2b7e
Redirect IPs 172.66.40.130, 172.66.43.126, 2606:4700:3108::ac42:2882, 2606:4700:3108::ac42:2b7e
Response IP 172.66.40.130
Found Yes
Hash dda83ce6a4631a68d33f36c0ff35bdd962d153d96ae25bad636a48bb205df304
SimHash e501c5dee3d0

Groups

*

Rule Path
Disallow /membership/
Disallow /instructions/
Disallow /Admin/
Disallow /Assets/
Disallow /Checkout/
Disallow /ClientApi/
Disallow /ConLib/
Disallow /Install/
Disallow /Layouts/
Disallow /Members/
Disallow /webcharts/
Disallow /Basket.aspx
Disallow /BuyProduct.ashx
Disallow /*.axd$
Allow /dtcompanies/searchcompany.aspx
Allow /dtcompanies/companies-that-do-drug-test.aspx
Allow /dtcompanies/jobs-that-dont-drug-test.aspx
Allow /dtcompanies/most-common-drug-test-for-employment.aspx
Allow /false_positives_database/
Allow /false_positives_database/what-cause-false-positive-drug-test.aspx
Allow /false_positives_database/what-to-do-false-positive-drug-test.aspx
Allow /court_cases/
Allow /court_cases/child-custody-court-ordered-drug-testing-cases-list.aspx
Allow /court_cases/drug-testing-in-schools.aspx
Allow /court_cases/marijuana-court-cases-list.aspx
Allow /court_cases/welfare-drug-testing.aspx
Allow /court_cases/workplace-drug-testing-court-cases-list.aspx
Back to top

Other Records

Field Value
sitemap https://www.testclear.com/sitemap.xml
Back to top

Comments

  • Allow specific pages to be crawled
  • 101724 per Tarun
Back to top