thetrucker.com
robots.txt

Robots Exclusion Standard data for thetrucker.com

Resource Scan

Scan Details

Site Domain thetrucker.com
Base Domain thetrucker.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-10-05T13:14:57+00:00
Next Scan 2026-01-03T13:14:57+00:00

Last Successful Scan

Scanned2025-06-07T17:47:39+00:00
URL https://thetrucker.com/robots.txt
Redirect https://www.thetrucker.com/robots.txt
Redirect Domain www.thetrucker.com
Redirect Base thetrucker.com
Domain IPs 104.26.0.223, 104.26.1.223, 172.67.75.175, 2606:4700:20::681a:1df, 2606:4700:20::681a:df, 2606:4700:20::ac43:4baf
Redirect IPs 104.26.0.223, 104.26.1.223, 172.67.75.175, 2606:4700:20::681a:1df, 2606:4700:20::681a:df, 2606:4700:20::ac43:4baf
Response IP 104.26.0.223
Found Yes
Hash 9a7255095c743f74d916fc434efde23448fdb20126836e02e0465db28219a949
SimHash 500448004fb2

Groups

twitterbot

Rule Path
Disallow *
Allow /wp-content/uploads/*

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Allow /wp-admin/edit-tags.php

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

colly

Rule Path
Disallow /

bingbot

Rule Path
Allow /

Other Records

Field Value
sitemap https://www.thetrucker.com/sitemap_index.xml