airflow.com
robots.txt
Robots Exclusion Standard data for airflow.com
Resource Scan
Scan Details
Site Domain | airflow.com |
Base Domain | airflow.com |
Scan Status | Ok |
Last Scan | 2024-09-24T03:41:55+00:00 |
Next Scan | 2024-10-24T03:41:55+00:00 |
Last Scan
Scanned | 2024-09-24T03:41:55+00:00 |
URL | https://airflow.com/robots.txt |
Redirect | https://www.airflow.com/robots.txt |
Redirect Domain | www.airflow.com |
Redirect Base | airflow.com |
Domain IPs | 20.13.109.145 |
Redirect IPs | 13.107.246.59, 2620:1ec:bdf::59 |
Response IP | 13.107.246.59 |
Found | Yes |
Hash | b9bd7b6bb4666d92ed35cedbf382a8ca78fbadc15430e0f76a93095f42a6b02a |
SimHash | 2614dd11fe78 |
Groups
googlebot
Rule | Path |
---|---|
Disallow | /cms/cms.jsp?menu_id=30152 |
Disallow | /Delivery-Address |
Disallow | /Delivery-Charges |
Disallow | /cms/cms.jsp?menu_id=30096&choice=YES |
Disallow | /Credit-Checks |
Disallow | /Checkout-Complete |
Disallow | /Login |
Disallow | /ForgottenPassword |
Disallow | /Basket |
Disallow | */cms.jsp* |
Disallow | /product?prodref=* |
googlebot-image
Rule | Path |
---|---|
Disallow | /cms/cms.jsp?menu_id=30152 |
Disallow | /Delivery-Address |
Disallow | /Delivery-Charges |
Disallow | /cms/cms.jsp?menu_id=30096&choice=YES |
Disallow | /Credit-Checks |
Disallow | /Checkout-Complete |
Disallow | /Login |
Disallow | /ForgottenPassword |
Disallow | /Basket |
Disallow | */cms.jsp* |
Disallow | /product?prodref=* |
bingbot
Rule | Path |
---|---|
Disallow | /cms/cms.jsp?menu_id=30152 |
Disallow | /Delivery-Address |
Disallow | /Delivery-Charges |
Disallow | /cms/cms.jsp?menu_id=30096&choice=YES |
Disallow | /Credit-Checks |
Disallow | /Checkout-Complete |
Disallow | /Login |
Disallow | /ForgottenPassword |
Disallow | /Basket |
Disallow | */cms.jsp* |
Disallow | /product?prodref=* |
Other Records
Field | Value |
---|---|
crawl-delay | 1 |
msnbot
Rule | Path |
---|---|
Disallow | /cms/cms.jsp?menu_id=30152 |
Disallow | /Delivery-Address |
Disallow | /Delivery-Charges |
Disallow | /cms/cms.jsp?menu_id=30096&choice=YES |
Disallow | /Credit-Checks |
Disallow | /Checkout-Complete |
Disallow | /Login |
Disallow | /ForgottenPassword |
Disallow | /Basket |
Disallow | */cms.jsp* |
Disallow | /product?prodref=* |
Warnings
- 2 invalid lines.
Comments