Log In Sign Up

bookfinder4u.com
robots.txt

Robots Exclusion Standard data for bookfinder4u.com

Resource Scan

Scan Details

Site Domain bookfinder4u.com
Base Domain bookfinder4u.com
Scan Status Ok
Last Scan2024-10-20T06:32:58+00:00
Next Scan 2024-11-19T06:32:58+00:00

Last Scan

Scanned2024-10-20T06:32:58+00:00
URL http://bookfinder4u.com/robots.txt
Domain IPs 45.32.85.136
Response IP 45.32.85.136
Found Yes
Hash e5e357f5cc79ac025147060fc41b200022e881b495252d64d292ba1a98e54a52
SimHash 0b5df050ad73

Groups

ccbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

voilabot

Rule Path
Disallow /

boitho.com-dc

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

exabot

Rule Path
Disallow /

voyager

Rule Path
Disallow /

becomebot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

msnbot

Rule Path
Disallow /compare_prices.bfu
Disallow /compare_prices.do
Disallow /customer_reviews.do
Disallow /TitleSearch.aspx
Disallow /compare/
Disallow /compare.aspx
Disallow /compare.bfu
Disallow /customer_reviews.aspx
Disallow /customer_reviews.bfu
Disallow /process_url.aspx
Disallow /process_url.bfu
Disallow /send_to_friends.aspx
Disallow /BookDetails.aspx
Disallow /Addwebsite_rewrite.aspx
Disallow /Addwebsite_rewrite.bfu
Disallow /author_adwords/
Disallow /title_adwords/
Disallow /redirect.aspx
Disallow /noindex/
Disallow /noindex/redirect.aspx
Disallow /noindex/show_images.aspx
Disallow /member/account/
Disallow /member/search_history/
Disallow /member/book_reviews/add_review.aspx
Disallow /member/store_reviews/add_review.aspx
Disallow /member/store_reviews/display_store.aspx
Disallow /wishlist/wishlist_add_book.aspx
Disallow /wishlist/add_success.aspx
Disallow /wishlist/add_note.aspx
Disallow /wishlist/price_alert.aspx
Disallow /wishlist/wishlist.aspx
Disallow /wishlist/update_price/
Disallow /wishlist/delete_item.aspx
Disallow /find_book_recommend_us.aspx
Disallow /links/

Other Records

Field Value
crawl-delay 2

teoma

Rule Path
Disallow /compare_prices.bfu
Disallow /compare_prices.do
Disallow /customer_reviews.do
Disallow /TitleSearch.aspx
Disallow /compare/
Disallow /compare.aspx
Disallow /compare.bfu
Disallow /customer_reviews.aspx
Disallow /customer_reviews.bfu
Disallow /process_url.aspx
Disallow /process_url.bfu
Disallow /send_to_friends.aspx
Disallow /BookDetails.aspx
Disallow /Addwebsite_rewrite.aspx
Disallow /Addwebsite_rewrite.bfu
Disallow /author_adwords/
Disallow /title_adwords/
Disallow /redirect.aspx
Disallow /noindex/
Disallow /noindex/redirect.aspx
Disallow /noindex/show_images.aspx
Disallow /member/account/
Disallow /member/search_history/
Disallow /member/book_reviews/add_review.aspx
Disallow /member/store_reviews/add_review.aspx
Disallow /member/store_reviews/display_store.aspx
Disallow /wishlist/wishlist_add_book.aspx
Disallow /wishlist/add_success.aspx
Disallow /wishlist/add_note.aspx
Disallow /wishlist/price_alert.aspx
Disallow /wishlist/wishlist.aspx
Disallow /wishlist/update_price/
Disallow /wishlist/delete_item.aspx
Disallow /find_book_recommend_us.aspx
Disallow /links/

Other Records

Field Value
crawl-delay 2

slurp

Rule Path
Disallow /compare_prices.bfu
Disallow /compare_prices.do
Disallow /customer_reviews.do
Disallow /TitleSearch.aspx
Disallow /compare/
Disallow /compare.aspx
Disallow /compare.bfu
Disallow /customer_reviews.aspx
Disallow /customer_reviews.bfu
Disallow /process_url.aspx
Disallow /process_url.bfu
Disallow /send_to_friends.aspx
Disallow /BookDetails.aspx
Disallow /Addwebsite_rewrite.aspx
Disallow /Addwebsite_rewrite.bfu
Disallow /author_adwords/
Disallow /title_adwords/
Disallow /redirect.aspx
Disallow /noindex/
Disallow /noindex/redirect.aspx
Disallow /noindex/show_images.aspx
Disallow /member/account/
Disallow /member/search_history/
Disallow /member/book_reviews/add_review.aspx
Disallow /member/store_reviews/add_review.aspx
Disallow /member/store_reviews/display_store.aspx
Disallow /wishlist/wishlist_add_book.aspx
Disallow /wishlist/add_success.aspx
Disallow /wishlist/add_note.aspx
Disallow /wishlist/price_alert.aspx
Disallow /wishlist/wishlist.aspx
Disallow /wishlist/update_price/
Disallow /wishlist/delete_item.aspx
Disallow /find_book_recommend_us.aspx
Disallow /links/

Other Records

Field Value
crawl-delay 2

googlebot

Rule Path
Disallow /compare_prices.bfu
Disallow /compare_prices.do
Disallow /customer_reviews.do
Disallow /TitleSearch.aspx
Disallow /compare.aspx
Disallow /compare.bfu
Disallow /customer_reviews.aspx
Disallow /customer_reviews.bfu
Disallow /process_url.aspx
Disallow /process_url.bfu
Disallow /send_to_friends.aspx
Disallow /BookDetails.aspx
Disallow /Addwebsite_rewrite.aspx
Disallow /Addwebsite_rewrite.bfu
Disallow /author_adwords/
Disallow /title_adwords/
Disallow /compare/
Disallow /amazon.aspx
Disallow /free_magazines.aspx
Disallow /redirect.aspx
Disallow /noindex/
Disallow /noindex/redirect.aspx
Disallow /noindex/show_images.aspx
Disallow /member/account/
Disallow /member/search_history/
Disallow /member/book_reviews/add_review.aspx
Disallow /member/store_reviews/add_review.aspx
Disallow /member/store_reviews/display_store.aspx
Disallow /wishlist/wishlist_add_book.aspx
Disallow /wishlist/add_success.aspx
Disallow /wishlist/add_note.aspx
Disallow /wishlist/price_alert.aspx
Disallow /wishlist/wishlist.aspx
Disallow /wishlist/update_price/
Disallow /wishlist/delete_item.aspx
Disallow /find_book_recommend_us.aspx
Disallow /links/

Other Records

Field Value
crawl-delay 2

*

Rule Path
Disallow /compare_prices.bfu
Disallow /compare_prices.do
Disallow /customer_reviews.do
Disallow /TitleSearch.aspx
Disallow /compare/
Disallow /compare.aspx
Disallow /compare.bfu
Disallow /customer_reviews.aspx
Disallow /customer_reviews.bfu
Disallow /process_url.aspx
Disallow /process_url.bfu
Disallow /send_to_friends.aspx
Disallow /BookDetails.aspx
Disallow /Addwebsite_rewrite.aspx
Disallow /Addwebsite_rewrite.bfu
Disallow /author_adwords/
Disallow /title_adwords/
Disallow /redirect.aspx
Disallow /noindex/
Disallow /noindex/redirect.aspx
Disallow /noindex/show_images.aspx
Disallow /member/account/
Disallow /member/search_history/
Disallow /member/book_reviews/add_review.aspx
Disallow /member/store_reviews/add_review.aspx
Disallow /member/store_reviews/display_store.aspx
Disallow /wishlist/wishlist_add_book.aspx
Disallow /wishlist/add_success.aspx
Disallow /wishlist/add_note.aspx
Disallow /wishlist/price_alert.aspx
Disallow /wishlist/wishlist.aspx
Disallow /wishlist/update_price/
Disallow /wishlist/delete_item.aspx
Disallow /find_book_recommend_us.aspx
Disallow /links/
Back to top