fairtradetobacco.com
robots.txt
Robots Exclusion Standard data for fairtradetobacco.com
Resource Scan
Scan Details
| Site Domain | fairtradetobacco.com |
| Base Domain | fairtradetobacco.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2025-05-06T04:39:33+00:00 |
| Next Scan | 2025-05-20T04:39:33+00:00 |
Last Successful Scan
| Scanned | 2025-03-29T04:38:28+00:00 |
| URL | https://fairtradetobacco.com/robots.txt |
| Domain IPs | 104.36.62.101 |
| Response IP | 104.36.62.101 |
| Found | Yes |
| Hash | a8b24a20ebb75b5f40cb3cdd74d5ad86a7c756f71f428568cecdd6281466673c |
| SimHash | 2f4e5a208e87 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /helpfulanswers.php |
| Disallow | /calendar.php |
| Disallow | /editpost.php |
| Disallow | /misc.php |
| Disallow | /newreply.php |
| Disallow | /newthread.php |
| Disallow | /printthread.php |
| Disallow | /private.php |
| Disallow | /register.php |
| Disallow | /report.php |
| Disallow | /search.php |
| Disallow | /announcement.php |
| Disallow | /showgroups.php |
| Disallow | /online.php |
| Disallow | /subscription.php |
| Disallow | /threadrate.php |
| Disallow | /poll.php |
| Disallow | /album.php |
| Disallow | /faq.php |
| Disallow | /cron.php |
| Disallow | /usercp.php |
| Disallow | /profile.php |
| Disallow | /vbseocp.php |
| Disallow | /login.php |
| Disallow | /member.php |
| Disallow | /sitemap/ |
| Disallow | /archive/ |
| Disallow | /ajax.php |
| Disallow | /arcade.php |
| Disallow | /attachment.php |
| Disallow | /joinrequests.php |
| Disallow | /memberlist.php |
| Disallow | /global.php |
| Disallow | /usernote.php |
| Disallow | /reputation.php |
| Disallow | /includes/ |
| Disallow | /install/ |
| Disallow | /customavatars/ |
| Disallow | /customgroupicons/ |
| Disallow | /customprofilepics/ |
| Disallow | /attachments/ |
| Disallow | /memberlist/ |
| Disallow | /sitemap/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
Other Records
| Field | Value |
|---|---|
| sitemap | http://fairtradetobacco.com/sitemap_index.xml.gz |