hotdeali.com
robots.txt

Robots Exclusion Standard data for hotdeali.com

Resource Scan

Scan Details

Site Domain hotdeali.com
Base Domain hotdeali.com
Scan Status Ok
Last Scan2024-11-14T08:09:12+00:00
Next Scan 2024-11-21T08:09:12+00:00

Last Scan

Scanned2024-11-14T08:09:12+00:00
URL https://hotdeali.com/robots.txt
Redirect https://www.hotdeali.com/robots.txt
Redirect Domain www.hotdeali.com
Redirect Base hotdeali.com
Domain IPs 162.0.229.111
Redirect IPs 162.0.229.111
Response IP 162.0.229.111
Found Yes
Hash e4d459a4bb4a861ebf938f6e18dd4b7eba1f87252fb7b983d1dead193cf11220
SimHash 00bc5141ef06

Groups

googlebot

Rule Path
Allow /
Disallow /adm/
Disallow /login/
Disallow /bbs/board.php?bo_table=proposal
Disallow /proposal
Disallow /shop/

yeti

Rule Path
Allow /
Disallow /adm/
Disallow /login/
Disallow /bbs/board.php?bo_table=proposal
Disallow /proposal
Disallow /shop/

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

amazonbot

Product Comment
amazonbot Amazon's user agent
Rule Path
Disallow /

amazon-kendra-web-crawler-*

Product Comment
amazon-kendra-web-crawler-* all customers of Amazon Kendra's web crawler
Rule Path Comment
Disallow / disallow everything

amazon cloudfront

Rule Path
Disallow /

ahrefs

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /