whodoyou.com
robots.txt
Robots Exclusion Standard data for whodoyou.com
Resource Scan
Scan Details
Site Domain | whodoyou.com |
Base Domain | whodoyou.com |
Scan Status | Ok |
Last Scan | 2024-10-05T15:14:29+00:00 |
Next Scan | 2024-10-12T15:14:29+00:00 |
Last Scan
Scanned | 2024-10-05T15:14:29+00:00 |
URL | https://whodoyou.com/robots.txt |
Redirect | https://www.whodoyou.com/robots.txt |
Redirect Domain | www.whodoyou.com |
Redirect Base | whodoyou.com |
Domain IPs | 18.211.231.78, 35.173.88.76 |
Redirect IPs | 18.211.231.78, 35.173.88.76 |
Response IP | 18.211.231.78 |
Found | Yes |
Hash | 03f23b498701a1a2acfeea0296588135efcbc8e6c8c724080b27637e3d31d58f |
SimHash | 465dd36c4412 |
Groups
*
Rule | Path |
---|---|
Disallow | /categories/ |
Disallow | /search |
Disallow | /nbn/search |
Disallow | /review-center |
Disallow | /create-review |
Disallow | /user/ |
Disallow | /my-recommendation-vault |
Disallow | /p/ |
Disallow | /s/ |
Disallow | /review/ |
Disallow | /recommended/ |
Disallow | /post/thread |
Disallow | /g/ |
Disallow | /claim-profile |
Disallow | /provider/build-claim-form |
Disallow | /biz/claim |
Disallow | /send/events |
Disallow | /js/categories |
Disallow | /prime |
Disallow | /prime/listingscanner/report |
Disallow | /app-biz/* |
mediapartners-google
Rule | Path |
---|---|
Disallow | /categories/ |
Disallow | /review-center |
Disallow | /create-review |
Disallow | /user/ |
Disallow | /my-recommendation-vault |
Disallow | /p/ |
Disallow | /s/ |
Disallow | /review/ |
Disallow | /recommended/ |
Disallow | /g/ |
Disallow | /claim-profile |
Disallow | /provider/build-claim-form |
Disallow | /biz/claim |
Disallow | /send/events |
Disallow | /js/categories |
Other Records
Field | Value |
---|---|
sitemap | https://www.whodoyou.com/sitemap_index.xml |