interworldfreight.com
robots.txt
Robots Exclusion Standard data for interworldfreight.com
Resource Scan
Scan Details
| Site Domain | interworldfreight.com |
| Base Domain | interworldfreight.com |
| Scan Status | Ok |
| Last Scan | 2025-10-04T00:44:16+00:00 |
| Next Scan | 2025-11-03T00:44:16+00:00 |
Last Scan
| Scanned | 2025-10-04T00:44:16+00:00 |
| URL | https://interworldfreight.com/robots.txt |
| Redirect | https://interwf.com/robots.txt |
| Redirect Domain | interwf.com |
| Redirect Base | interwf.com |
| Domain IPs | 199.60.103.189, 199.60.103.89 |
| Redirect IPs | 199.60.103.189, 199.60.103.89 |
| Response IP | 199.60.103.189 |
| Found | Yes |
| Hash | fdfe44a498ee4262960107e97d5071a701570566ccfb44d444271671031d4bd4 |
| SimHash | 2218b071e493 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /_hcms/per |
| Disallow | /es/terminos-y-condiciones |
| Disallow | /terms-and-conditions |
| Disallow | /es/politica-de-cookies |
| Disallow | /cookies-policy |
| Disallow | /_hcms/livechat |
| Disallow | /_hcms/forms/ |
| Disallow | /_hcms/preview/ |
| Disallow | /hs/manage-preferences/ |
| Disallow | /*?*hs_preview=* |
| Disallow | /*?*hsCacheBuster=* |
| Disallow | /hs/preferences-center/ |
oai-searchbot
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /_hcms/preview/ |
| Disallow | /hs/manage-preferences/ |
| Disallow | /hs/preferences-center/ |
| Disallow | /*?*hs_preview=* |
| Disallow | /*?*hsCacheBuster=* |
chatgpt-user
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /_hcms/preview/ |
| Disallow | /hs/manage-preferences/ |
| Disallow | /hs/preferences-center/ |
| Disallow | /*?*hs_preview=* |
| Disallow | /*?*hsCacheBuster=* |
perplexitybot
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /_hcms/preview/ |
| Disallow | /hs/manage-preferences/ |
| Disallow | /hs/preferences-center/ |
| Disallow | /*?*hs_preview=* |
| Disallow | /*?*hsCacheBuster=* |
firecrawlagent
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /_hcms/preview/ |
| Disallow | /hs/manage-preferences/ |
| Disallow | /hs/preferences-center/ |
| Disallow | /*?*hs_preview=* |
| Disallow | /*?*hsCacheBuster=* |
andibot
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /_hcms/preview/ |
| Disallow | /hs/manage-preferences/ |
| Disallow | /hs/preferences-center/ |
| Disallow | /*?*hs_preview=* |
| Disallow | /*?*hsCacheBuster=* |
exabot
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /_hcms/preview/ |
| Disallow | /hs/manage-preferences/ |
| Disallow | /hs/preferences-center/ |
| Disallow | /*?*hs_preview=* |
| Disallow | /*?*hsCacheBuster=* |
phindbot
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /_hcms/preview/ |
| Disallow | /hs/manage-preferences/ |
| Disallow | /hs/preferences-center/ |
| Disallow | /*?*hs_preview=* |
| Disallow | /*?*hsCacheBuster=* |
youbot
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /_hcms/preview/ |
| Disallow | /hs/manage-preferences/ |
| Disallow | /hs/preferences-center/ |
| Disallow | /*?*hs_preview=* |
| Disallow | /*?*hsCacheBuster=* |
chatgpt-user-agent
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /_hcms/preview/ |
| Disallow | /hs/manage-preferences/ |
| Disallow | /hs/preferences-center/ |
| Disallow | /*?*hs_preview=* |
| Disallow | /*?*hsCacheBuster=* |
anthropic-ai
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /_hcms/preview/ |
| Disallow | /hs/manage-preferences/ |
| Disallow | /hs/preferences-center/ |
| Disallow | /*?*hs_preview=* |
| Disallow | /*?*hsCacheBuster=* |
gptbot
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /_hcms/preview/ |
| Disallow | /hs/manage-preferences/ |
| Disallow | /hs/preferences-center/ |
| Disallow | /*?*hs_preview=* |
| Disallow | /*?*hsCacheBuster=* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://interwf.com/sitemap.xml |
Comments