domain.com
robots.txt
Robots Exclusion Standard data for domain.com
Resource Scan
Scan Details
| Site Domain | domain.com |
| Base Domain | domain.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-09-11T19:55:12+00:00 |
| Next Scan | 2024-12-10T19:55:12+00:00 |
Last Successful Scan
| Scanned | 2024-01-23T13:33:04+00:00 |
| URL | https://domain.com/robots.txt |
| Redirect | https://www.domain.com/robots.txt |
| Redirect Domain | www.domain.com |
| Redirect Base | domain.com |
| Domain IPs | 13.57.130.190, 18.221.195.49 |
| Redirect IPs | 13.57.130.190, 18.221.195.49 |
| Response IP | 13.57.130.190 |
| Found | Yes |
| Hash | 4f6afbfc457dc7366cedc5e58652fca3960b6f23db1fdb08db40e66d9568ffa9 |
| SimHash | 914240444d24 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /aboutAppC/ |
| Disallow | /admin/ |
| Disallow | /affiliateAppC/ |
| Disallow | /affiliateControl/ |
| Disallow | /appinterface/ |
| Disallow | /appinterfaceAppC/ |
| Disallow | /articlesAppC/ |
| Disallow | /bandwidth/ |
| Disallow | /BizBuilder/ |
| Disallow | /build/ |
| Disallow | /categoryAppC/ |
| Disallow | /cgi-bin/ |
| Disallow | /cgi-fy/ |
| Disallow | /cgi-va/ |
| Disallow | /cobrand/ |
| Disallow | /cobrandAppC/ |
| Disallow | /communityAppC/ |
| Disallow | /cvsadmin/ |
| Disallow | /data/ |
| Disallow | /directMail/ |
| Disallow | /directoryAppC/ |
| Disallow | /directory/ |
| Disallow | /error/ |
| Disallow | /firetest/ |
| Disallow | /homeAppC/ |
| Disallow | /joinAppC/ |
| Disallow | /knowledgebase/allkeywords.cmp |
| Disallow | /knowledgebase/ |
| Disallow | /knowledgebase/beta |
| Disallow | /keepalive/ |
| Disallow | /knowledgebaseAppC/ |
| Disallow | /mail/ |
| Disallow | /marketplaceAppC/ |
| Disallow | /metatags/ |
| Disallow | /mining/ |
| Disallow | /monitoring/ |
| Disallow | /newsletterAppC/ |
| Disallow | /nortec/ |
| Disallow | /opentracker/ |
| Disallow | /panel/ |
| Disallow | /pdf/ |
| Disallow | /pluginControl/ |
| Disallow | /popupAppC/ |
| Disallow | /productAppC/ |
| Disallow | /referAppC/ |
| Disallow | /register/ |
| Disallow | /scripts/ |
| Disallow | /search/ |
| Disallow | /secure/ |
| Disallow | /secureControl/ |
| Disallow | /sitestudioAppC/ |
| Disallow | /supportAppC/ |
| Disallow | /tipsAppC/ |
| Disallow | /unsynced/ |
| Disallow | /utils/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.domain.com/sitemap.xml |