exporthub.com
robots.txt

Robots Exclusion Standard data for exporthub.com

Resource Scan

Scan Details

Site Domain exporthub.com
Base Domain exporthub.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-06-03T17:25:05+00:00
Next Scan 2024-07-03T17:25:05+00:00

Last Successful Scan

Scanned2024-05-05T13:47:04+00:00
URL https://exporthub.com/robots.txt
Redirect https://www.exporthub.com/robots.txt
Redirect Domain www.exporthub.com
Redirect Base exporthub.com
Domain IPs 172.66.40.184, 172.66.43.72, 2606:4700:3108::ac42:28b8, 2606:4700:3108::ac42:2b48
Redirect IPs 172.66.40.184, 172.66.43.72, 2606:4700:3108::ac42:28b8, 2606:4700:3108::ac42:2b48
Response IP 172.66.43.72
Found Yes
Hash 00ea13efe742ef9bb0d6f6cde58b4d17554a81ed96e568c41c050a928337a600
SimHash 6c545bf5c751

Groups

googlebot

Rule Path
Allow /

googlebot-news

Rule Path
Allow /

googlebot-image/1.0

Rule Path
Allow /

googlebot-video/1.0

Rule Path
Allow /

mediapartners-google

Rule Path
Allow /

adsbot-google-mobile-apps

Rule Path
Allow /

apis-google

Rule Path
Allow /

adsbot-google-mobile

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

googleproducer

Rule Path
Allow /

storebot-google

Rule Path
Allow /

bingbot

Rule Path
Allow /

slurp

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

baiduspider

Rule Path
Allow /

baiduspider-image

Rule Path
Allow /

baiduspider-video

Rule Path
Allow /

baiduspider-news

Rule Path
Allow /

baiduspider-favo

Rule Path
Allow /

baiduspider-cpro

Rule Path
Allow /

baiduspider-ads

Rule Path
Allow /

yandexbot

Rule Path
Allow /

exabot

Rule Path
Allow /

applebot

Rule Path
Allow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

adidxbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingpreview

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

*

Rule Path
Disallow
Disallow /eh-*
Disallow /prod/*
Disallow /latest/
Disallow /suppliers/*
Disallow /trade-lead/*

Other Records

Field Value
sitemap https://www.exporthub.com/all_sitemap_com.xml