firstcolleyville.com
robots.txt
Robots Exclusion Standard data for firstcolleyville.com
Resource Scan
Scan Details
Site Domain | firstcolleyville.com |
Base Domain | firstcolleyville.com |
Scan Status | Ok |
Last Scan | 2024-09-21T09:53:43+00:00 |
Next Scan | 2024-09-28T09:53:43+00:00 |
Last Scan
Scanned | 2024-09-21T09:53:43+00:00 |
URL | https://firstcolleyville.com/robots.txt |
Redirect | https://www.firstcolleyville.com/robots.txt |
Redirect Domain | www.firstcolleyville.com |
Redirect Base | firstcolleyville.com |
Domain IPs | 40.119.1.254 |
Redirect IPs | 40.119.1.254 |
Response IP | 40.119.1.254 |
Found | Yes |
Hash | 729f5be583c0ef7212a603a7cd2f1814a44fab3951158adebf9d9622bc3ac8e2 |
SimHash | 603448d7e155 |
Groups
googlebot
Rule | Path |
---|---|
Disallow | *.config$ |
Disallow | *.asmx$ |
Disallow | *.ascx$ |
Disallow | *.ashx$ |
Disallow | *.vb$ |
Disallow | *.cs$ |
Disallow | *.resx$ |
Disallow | *.master$ |
Disallow | *.dll$ |
Disallow | /profile/login.aspx |
Disallow | /Profile/login.aspx |
Disallow | /store_categoryproducts.aspx |
Disallow | /Store_CategoryProducts.aspx |
Disallow | /store_searchresults.aspx |
Disallow | /Store_SearchResults.aspx |
Disallow | /searchresults.aspx |
Disallow | /SearchResults.aspx |
Disallow | /app_services/ |
Disallow | /App_Services/ |
Disallow | /admin/ |
Disallow | /Admin/ |
Disallow | /bin/ |
Disallow | /Bin/ |
Other Records
Field | Value |
---|---|
crawl-delay | 30 |
googlebot-image
Rule | Path |
---|---|
Disallow | *.config$ |
Disallow | *.asmx$ |
Disallow | *.ascx$ |
Disallow | *.ashx$ |
Disallow | *.vb$ |
Disallow | *.cs$ |
Disallow | *.resx$ |
Disallow | *.master$ |
Disallow | *.dll$ |
Disallow | /profile/login.aspx |
Disallow | /Profile/login.aspx |
Disallow | /store_categoryproducts.aspx |
Disallow | /Store_CategoryProducts.aspx |
Disallow | /store_searchresults.aspx |
Disallow | /Store_SearchResults.aspx |
Disallow | /searchresults.aspx |
Disallow | /SearchResults.aspx |
Disallow | /app_services/ |
Disallow | /App_Services/ |
Disallow | /admin/ |
Disallow | /Admin/ |
Disallow | /bin/ |
Disallow | /Bin/ |
Other Records
Field | Value |
---|---|
crawl-delay | 30 |
twitterbot
Rule | Path |
---|---|
Disallow | *.config$ |
Disallow | *.asmx$ |
Disallow | *.ascx$ |
Disallow | *.ashx$ |
Disallow | *.vb$ |
Disallow | *.cs$ |
Disallow | *.resx$ |
Disallow | *.master$ |
Disallow | *.dll$ |
Disallow | /profile/login.aspx |
Disallow | /Profile/login.aspx |
Disallow | /store_categoryproducts.aspx |
Disallow | /Store_CategoryProducts.aspx |
Disallow | /store_searchresults.aspx |
Disallow | /Store_SearchResults.aspx |
Disallow | /searchresults.aspx |
Disallow | /SearchResults.aspx |
Disallow | /app_services/ |
Disallow | /App_Services/ |
Disallow | /admin/ |
Disallow | /Admin/ |
Disallow | /bin/ |
Disallow | /Bin/ |
Other Records
Field | Value |
---|---|
crawl-delay | 30 |
*
Rule | Path |
---|---|
Disallow | *.config$ |
Disallow | *.asmx$ |
Disallow | *.ascx$ |
Disallow | *.ashx$ |
Disallow | *.vb$ |
Disallow | *.cs$ |
Disallow | *.resx$ |
Disallow | *.master$ |
Disallow | *.dll$ |
Disallow | /profile/login.aspx |
Disallow | /Profile/login.aspx |
Disallow | /store_categoryproducts.aspx |
Disallow | /Store_CategoryProducts.aspx |
Disallow | /store_searchresults.aspx |
Disallow | /Store_SearchResults.aspx |
Disallow | /searchresults.aspx |
Disallow | /SearchResults.aspx |
Disallow | /app_services/ |
Disallow | /App_Services/ |
Disallow | /admin/ |
Disallow | /Admin/ |
Disallow | /bin/ |
Disallow | /Bin/ |
Disallow | /Sitefiles/ |
Disallow | /sitefiles/ |
Other Records
Field | Value |
---|---|
crawl-delay | 30 |