classifieds.edmontonjournal.com
robots.txt
Robots Exclusion Standard data for classifieds.edmontonjournal.com
Resource Scan
Scan Details
Site Domain | classifieds.edmontonjournal.com |
Base Domain | edmontonjournal.com |
Scan Status | Ok |
Last Scan | 2024-05-14T19:39:40+00:00 |
Next Scan | 2024-06-13T19:39:40+00:00 |
Last Scan
Scanned | 2024-05-14T19:39:40+00:00 |
URL | https://classifieds.edmontonjournal.com/robots.txt |
Domain IPs | 34.209.20.111, 34.212.153.218, 52.35.191.153 |
Response IP | 34.212.153.218 |
Found | Yes |
Hash | fda73e11b809729011a31128d426b031737e78037a0151e879bafcb4ffa738a7 |
SimHash | 6b0537730be7 |
Groups
*
No rules defined. All paths allowed.
Other Records
Field | Value |
---|---|
crawl-delay | 30 |
*
Rule | Path |
---|---|
Disallow | /*?*search_type* |
*
Rule | Path |
---|---|
Disallow | /search* |
*
Rule | Path |
---|---|
Disallow | /*?*ap_search_* |
*
Rule | Path |
---|---|
Disallow | /ajax/post_form/ |
*
Rule | Path |
---|---|
Disallow | /admin/ |
*
Rule | Path |
---|---|
Disallow | /*-admin/ |
*
Rule | Path |
---|---|
Disallow | /manage-*/ |
*
Rule | Path |
---|---|
Disallow | /create-*/ |
*
Rule | Path |
---|---|
Disallow | /edit-*/ |
*
Rule | Path |
---|---|
Disallow | /claim-*/ |
*
Rule | Path |
---|---|
Disallow | /wp-admin/ |
Allow | /wp-admin/admin-ajax.php |
Warnings
- 8 invalid lines.
Comments