biblio.co.uk
robots.txt

Robots Exclusion Standard data for biblio.co.uk

Resource Scan

Scan Details

Site Domain biblio.co.uk
Base Domain biblio.co.uk
Scan Status Ok
Last Scan2024-11-10T21:41:20+00:00
Next Scan 2024-11-17T21:41:20+00:00

Last Scan

Scanned2024-11-10T21:41:20+00:00
URL https://biblio.co.uk/robots.txt
Domain IPs 104.26.6.171, 104.26.7.171, 172.67.70.124, 2606:4700:20::681a:6ab, 2606:4700:20::681a:7ab, 2606:4700:20::ac43:467c
Response IP 172.67.70.124
Found Yes
Hash 2ef4a9469703b88d388d7ca06145eb30938cb9ae2e6278aa79ca599d5c2dfa1a
SimHash 42995e615e59

Groups

*

Rule Path
Disallow /abrowse.php
Disallow /account_menu.php
Disallow /booksellers/
Disallow /app/booksellers/
Disallow /add_rental_to_cart.php
Disallow /app/jaxie/find_prices
Disallow /app/jaxie/meta_search
Disallow /app/offer/make
Disallow /b/
Disallow /banner.php
Disallow /bgsearch.php
Disallow /bookhounddownload/
Disallow /book_photos/
Disallow /browse_catalogs/
Disallow /cart.php
Disallow /create_account.php
Disallow /ebooks/
Disallow /email_listing.php
Disallow /favorite_booksellers.php
Disallow /info_request.php
Disallow /login.php
Disallow /login
Disallow /m/
Disallow /map.php
Disallow /o/
Disallow /orderdetails.php
Disallow /print-on-demand/
Disallow /remote-search*
Disallow /rental_search.php
Disallow /review.php
Disallow /search.php
Disallow /secure-checkout/
Disallow /selling_books/account_registration.php
Disallow /shipping_estimate.php
Disallow /submit_recommendation.php
Disallow /utility/
Disallow /w/
Disallow /wants.php
Disallow /z/

ahrefsbot

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

boitho.com-dc

Rule Path
Disallow /

charlotte

Rule Path
Disallow /

crawlbot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

riddler

Rule Path
Disallow /

seekbot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

speedy

Rule Path
Disallow /

the knowledge ai

Rule Path
Disallow /

trovitbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

voilabot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yeti

Rule Path
Disallow /