currency.ha.com
robots.txt

Robots Exclusion Standard data for currency.ha.com

Resource Scan

Scan Details

Site Domain currency.ha.com
Base Domain ha.com
Scan Status Ok
Last Scan2024-11-11T17:31:40+00:00
Next Scan 2024-11-25T17:31:40+00:00

Last Scan

Scanned2024-11-11T17:31:40+00:00
URL https://currency.ha.com/robots.txt
Domain IPs 104.18.38.129, 172.64.149.127, 2606:4700:4400::6812:2681, 2606:4700:4400::ac40:957f
Response IP 104.18.38.129
Found Yes
Hash 9a77121258d88b342c53ce7e86022d682e56cb73d9ab7eb0633476e7da5e3d24
SimHash 13a4dec186ea

Groups

semrushbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

riddler

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

mozilla/5.0 (compatible; gluten free crawler/1.0; +http://glutenfreepleasure.com/)

Rule Path
Disallow /

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

adsbot-google

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

pinterest

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

*

Rule Path
Disallow /c/greetings.zx
Disallow /c/ha-rebuttal.zx
Disallow /c/cart/
Disallow /c/error/invalid-lot.zx
Disallow /c/my/collection/
Disallow /c/bid-review.zx
Disallow /c/bid.zx
Disallow /c/prlink.zx
Disallow /c/error/maintenance.zx
Disallow /c/print-prices-realized.zx
Disallow /c/catalog-print.zx
Disallow /c/s/d/frontmatter/
Disallow /c/ecatalog.zx
Disallow /c/auction-home.zx
Disallow /c/invoice/
Disallow /c/rank.zx
Disallow /c/moto/
Disallow /c/my/wantlist.zx
Disallow /c/webservices/ebay-banner.zx
Disallow /c/ebay-results.zx
Disallow /c/phone-bid.zx
Disallow /live.zx

Other Records

Field Value
crawl-delay 15

twiceler

Rule Path
Disallow /c/search-results.zx

baiduspider

Rule Path
Disallow /c/search-results.zx

megaindex

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

goodzer

Rule Path
Disallow /

twitterbot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

mozilla/5.0 (windows nt 6.1; wow64; rv:39.0) gecko/20100101 firefox/39.0 | collexion search engine crawler | contact: support@collexion.com

Rule Path
Disallow /

semrushbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5

semrushbot-sa

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5

semrushbot-si

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5

Other Records

Field Value
sitemap https://currency.ha.com/sitemap.zx