mydomain.com
robots.txt
Robots Exclusion Standard data for mydomain.com
Resource Scan
Scan Details
| Site Domain | mydomain.com |
| Base Domain | mydomain.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-06-09T16:06:07+00:00 |
| Next Scan | 2024-08-08T16:06:07+00:00 |
Last Successful Scan
| Scanned | 2024-03-19T15:28:32+00:00 |
| URL | https://mydomain.com/robots.txt |
| Redirect | https://www.mydomain.com/robots.txt |
| Redirect Domain | www.mydomain.com |
| Redirect Base | mydomain.com |
| Domain IPs | 65.254.242.180 |
| Redirect IPs | 18.155.68.15, 18.155.68.53, 18.155.68.68, 18.155.68.78 |
| Response IP | 18.155.68.53 |
| Found | Yes |
| Hash | 2bb21b13fafcad45235057bd1bc46bc19d56a34ef09cd93c293e856436332c3e |
| SimHash | 91c241444f26 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /aboutAppC/ |
| Disallow | /admin/ |
| Disallow | /affiliateAppC/ |
| Disallow | /affiliateControl/ |
| Disallow | /appinterface/ |
| Disallow | /appinterfaceAppC/ |
| Disallow | /articlesAppC/ |
| Disallow | /bandwidth/ |
| Disallow | /BizBuilder/ |
| Disallow | /build/ |
| Disallow | /categoryAppC/ |
| Disallow | /cgi-bin/ |
| Disallow | /cgi-fy/ |
| Disallow | /cgi-va/ |
| Disallow | /cobrand/ |
| Disallow | /cobrandAppC/ |
| Disallow | /communityAppC/ |
| Disallow | /cvsadmin/ |
| Disallow | /data/ |
| Disallow | /directMail/ |
| Disallow | /directoryAppC/ |
| Disallow | /directory/ |
| Disallow | /email/ |
| Disallow | /error/ |
| Disallow | /firetest/ |
| Disallow | /homeAppC/ |
| Disallow | /joinAppC/ |
| Disallow | /knowledgebase/allkeywords.cmp |
| Disallow | /knowledgebase/ |
| Disallow | /knowledgebase/beta |
| Disallow | /keepalive/ |
| Disallow | /knowledgebaseAppC/ |
| Disallow | /mail/ |
| Disallow | /marketplaceAppC/ |
| Disallow | /metatags/ |
| Disallow | /mining/ |
| Disallow | /monitoring/ |
| Disallow | /newsletterAppC/ |
| Disallow | /nortec/ |
| Disallow | /opentracker/ |
| Disallow | /panel/ |
| Disallow | /pdf/ |
| Disallow | /pluginControl/ |
| Disallow | /popupAppC/ |
| Disallow | /productAppC/ |
| Disallow | /referAppC/ |
| Disallow | /register/ |
| Disallow | /scripts/ |
| Disallow | /search/ |
| Disallow | /secure/ |
| Disallow | /secureControl/ |
| Disallow | /sitestudioAppC/ |
| Disallow | /supportAppC/ |
| Disallow | /tipsAppC/ |
| Disallow | /unsynced/ |
| Disallow | /utils/ |