care.com
robots.txt

Robots Exclusion Standard data for care.com

Resource Scan

Scan Details

Site Domain care.com
Base Domain care.com
Scan Status Ok
Last Scan2024-09-26T07:38:38+00:00
Next Scan 2024-10-03T07:38:38+00:00

Last Scan

Scanned2024-09-26T07:38:38+00:00
URL https://care.com/robots.txt
Redirect https://www.care.com/robots.txt
Redirect Domain www.care.com
Redirect Base care.com
Domain IPs 23.215.7.17, 23.215.7.25
Redirect IPs 23.215.7.17, 23.215.7.25
Response IP 23.59.168.107
Found Yes
Hash 7b6b83c12513140a78ba3144f1801c728533b736c63256597c12539c05ddb743
SimHash 4b145a32e221

Groups

*

Rule Path
Disallow /*/myaccount/
Disallow /ubertags/
Disallow /password
Disallow /trackVisit.trace
Disallow /verify-url
Disallow /us-marketplace/
Disallow /us-recruitment/
Disallow /us-senior-care/
Disallow /us-subscription/
Disallow /us-workplace-solutions/
Disallow /LitlePayPage
Disallow /visitor/popUpPhoto.do
Disallow /c/records/
Disallow /visitor/
Disallow /api/messages/*
Disallow *qs%3D1*
Disallow /seeker/
Disallow /app/enrollment/
Disallow /app/smb-profile/
Disallow /app/hp-enrollment/
Disallow */captureUnifiedSearch
Disallow */app/seo/*
Disallow */app/sem/*
Disallow */app/review/*
Allow /app/pn-business
Allow /app/enrollment/seeker/cc/tax-calculator
Allow /app/hp-vhp/

searchmetricsbot

Rule Path
Disallow /my-account/
Disallow /ubertags/
Disallow /login
Disallow /password

Other Records

Field Value
crawl-delay 3

adsbot-google

Rule Path
Allow *qs%3D1*
Allow */app/sem/*

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

gaisbot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

gptbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bytedance

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bytespider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://www.care.com/alpha-omega-jobs.xml
sitemap https://www.care.com/alpha-omega.xml
sitemap https://www.care.com/alpha-omega-theta-cost-guide.xml

Warnings

  • 2 invalid lines.