fordtruckzone.com
robots.txt

Robots Exclusion Standard data for fordtruckzone.com

Resource Scan

Scan Details

Site Domain fordtruckzone.com
Base Domain fordtruckzone.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-10-02T07:36:21+00:00
Next Scan 2024-10-09T07:36:21+00:00

Last Successful Scan

Scanned2024-06-04T06:44:21+00:00
URL https://fordtruckzone.com/robots.txt
Redirect https://www.fordtruckzone.com/robots.txt
Redirect Domain www.fordtruckzone.com
Redirect Base fordtruckzone.com
Domain IPs 104.21.34.218, 172.67.165.153, 2606:4700:3032::ac43:a599, 2606:4700:3037::6815:22da
Redirect IPs 104.21.34.218, 172.67.165.153, 2606:4700:3032::ac43:a599, 2606:4700:3037::6815:22da
Response IP 104.21.34.218
Found Yes
Hash 29798c28dbe270c8efb74b764b43d5369b453f310b6238ad603d9b46560a8a25
SimHash 30779b6144c5

Groups

*

Rule Path
Disallow /

googlebot

Rule Path
Disallow
Allow /

mediapartners-google

Rule Path
Disallow /hp/
Disallow /hp/separationdetective.php

msnbot-media

Rule Path
Disallow
Allow /

mediapartners-google

Rule Path
Disallow /hp/
Disallow /hp/separationdetective.php

bingbot

Rule Path
Disallow
Allow /

mediapartners-google

Rule Path
Disallow /hp/
Disallow /hp/separationdetective.php

slurp

Rule Path
Disallow
Allow /

Other Records

Field Value
crawl-delay 60

mediapartners-google

Rule Path
Disallow /hp/
Disallow /hp/separationdetective.php

msnbot

Rule Path
Disallow
Allow /

mediapartners-google

Rule Path
Disallow /hp/
Disallow /hp/separationdetective.php

msnbot-newsblogs

Rule Path
Disallow
Allow /

mediapartners-google

Rule Path
Disallow /hp/
Disallow /hp/separationdetective.php

msnbot

Rule Path
Disallow
Allow /

mediapartners-google

Rule Path
Disallow /hp/
Disallow /hp/separationdetective.php

applebot

Rule Path
Disallow /articles/authors
Disallow /*/account/
Disallow /account/
Disallow /admin.php
Disallow /ajax/
Disallow /blogs/
Disallow /calendar.php
Disallow /clientscript/
Disallow /conversations/
Disallow /converse.php
Disallow /cron.php
Disallow /cron.php*
Disallow /deferred.php
Disallow /emailsubscription/manage/
Disallow /events/birthdays/
Disallow /events/monthly
Disallow /events/weekly
Disallow /find-new/
Disallow /forums/-/
Disallow /forums/help/
Disallow /forums/search/
Disallow /forums/tweets/
Disallow /gallery/external.php
Disallow /*/help/
Disallow /help/
Disallow */index.rss
Disallow /list/author/
Disallow /*/login/
Disallow /login/
Disallow /login.php
Disallow /lost-password/
Disallow /media/help/
Disallow /*/members/%60
Disallow /*/misc/
Disallow /misc.php
Disallow /newreply.php
Disallow /newthread.php
Disallow /*/online/
Disallow /online/
Disallow /pages/conduct/
Disallow /pages/privacy/
Disallow /pages/private-or-public-computer/
Disallow /projhp/
Disallow /proxy.php?link
Disallow /recent-activity/
Disallow /*/register/
Disallow /register/
Disallow /*/reply?quote
Disallow /*/search/
Disallow /search/
Disallow /subscription.php
Disallow /threadloom
Disallow /threads/*/watch-confirm
Disallow /trapbot/
Disallow /visitme/
Disallow */watch
Disallow */watch-confirm
Disallow /hp/
Disallow /hp/separationdetective.php

mediapartners-google

Rule Path
Disallow /hp/
Disallow /hp/separationdetective.php

binglocalsearchbot

Rule Path
Disallow
Allow /

mediapartners-google

Rule Path
Disallow /hp/
Disallow /hp/separationdetective.php

binglocalsearch

Rule Path
Disallow
Allow /

mediapartners-google

Rule Path
Disallow /hp/
Disallow /hp/separationdetective.php

Comments

  • robots.txt for http://www.fordtruckzone.com/
  • generated by robots.php VERSION
  • All hits are logged. If your crawler isn't a search engine or other service
  • that will bring us traffic or benefit us, its not welcome here.
  • We do not allow non-authorized robots. Any unauthorized bot running on our
  • will have its IP(s) banned. Honey pots are running.
  • Showcontents 1
  • Mozilla/5.0 (compatible; WellKnownBot/0.1; +https://well-known.dev/about/#bot)
  • restrict all access 2