Log In Sign Up

bootselearning.net
robots.txt

Robots Exclusion Standard data for bootselearning.net

Resource Scan

Scan Details

Site Domain bootselearning.net
Base Domain bootselearning.net
Scan Status Ok
Last Scan2025-04-07T22:17:08+00:00
Next Scan 2025-04-14T22:17:08+00:00

Last Scan

Scanned2025-04-07T22:17:08+00:00
URL https://bootselearning.net/robots.txt
Domain IPs 104.21.76.42, 172.67.186.157, 2606:4700:3030::6815:4c2a, 2606:4700:3030::ac43:ba9d
Response IP 172.67.186.157
Found Yes
Hash a0e1efae9600b52bd060d2fbfe9d12ebe2f701c8f935b583cbd4ab3ee8a36c43
SimHash 26347391c218

Groups

*

Rule Path
Disallow /wp-json/
Disallow /?rest_route=

*

Rule Path
Disallow /search/
Disallow /?s=

*

Rule Path
Disallow *?s=*
Disallow *?p=*
Disallow *%26p%3D*
Disallow *%26preview%3D*

*

Rule Path
Disallow /feed/
Disallow /feed/$
Disallow /comments/feed
Disallow */feed
Disallow */feed$
Disallow /?feed=
Disallow /wp-feed

*

Rule Path
Disallow /trackback/
Disallow */comments$
Disallow */trackback
Disallow */trackback$
Disallow /wp-comments
Disallow /wp-trackback
Disallow */replytocom%3D

ia_archiver

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

ia_archiver-web.archive.org

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

*

Rule Path
Disallow /cart/

*

Rule Path
Disallow /checkout/

*

Rule Path
Disallow /my-account/

*

Rule Path
Disallow /login/

*

Rule Path
Disallow /*?orderby=price
Disallow /*?orderby=rating
Disallow /*?orderby=date
Disallow /*?orderby=price-desc
Disallow /*?orderby=popularity
Disallow /*?filter
Disallow /*?orderby=title
Disallow /*?orderby=desc
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /*?paged=&count=*
Disallow /*?count=*

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

xenu

Rule Path
Disallow /

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /wp-content/uploads/

mediapartners-google

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

adsbot-google-mobile

Rule Path
Allow /

bingbot

Rule Path
Allow /

applebot

Rule Path
Allow /

msnbot-media

Rule Path
Allow /wp-content/uploads/

msnbot

Rule Path
Allow /

yandex

Rule Path
Allow /

yandeximages

Rule Path
Allow /wp-content/uploads/

slurp

Rule Path
Allow /

qwantify

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

seznambot

Rule Path
Allow /

naverbot

Rule Path
Allow /

baiduspider

Rule Path
Allow /

baiduspider/2.0

Rule Path
Allow /

baiduspider-video

Rule Path
Allow /

baiduspider-image

Rule Path
Allow /

sogou spider

Rule Path
Allow /

sogou web spider

Rule Path
Allow /

sosospider

Rule Path
Allow /

sosospider+

Rule Path
Allow /

sosospider/2.0

Rule Path
Allow /

yodao

Rule Path
Allow /

youdao

Rule Path
Allow /

youdaobot

Rule Path
Allow /

youdaobot/1.0

Rule Path
Allow /

*

Rule Path
Allow /*.webp$

*

Rule Path
Allow /*.jpg$

*

Rule Path
Allow /*.png$

*

Rule Path
Allow /*.pdf$

*

Rule Path
Allow /*.gif$

discordbot

Rule Path
Allow /

pinterest bot

Rule Path
Allow /

pinterest/0.1

Rule Path
Allow /

pinterest/0.2

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

linkedinbot/1.0

Rule Path
Allow /

twitterbot

Rule Path
Allow /

telegrambot

Rule Path
Allow /

whatsapp bot

Rule Path
Allow /

instagrambot

Rule Path
Allow /

facebook

Rule Path
Allow /

facebookplatform/1.0

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

facebookexternalhit/1.0

Rule Path
Allow /

facebookexternalhit/1.1

Rule Path
Allow /

facebookscraper

Rule Path
Allow /

facebot/1.0

Rule Path
Allow /

visionutils/0.2

Rule Path
Allow /

datagnionbot/1.0

Rule Path
Allow /

*

Rule Path
Allow /*.php$

*

Rule Path
Allow /*.html$

*

Rule Path
Allow /*.docx$
Back to top

Other Records

Field Value
sitemap https://www.bootselearning.net/post-sitemap.xml
sitemap https://www.bootselearning.net/page-sitemap.xml
sitemap https://www.bootselearning.net/sitemap-news.xml
Back to top

Comments

  • Prevent Crawling of WordPress JSON API Endpoints
  • Block Search URLs /search/ and /?s=
  • Block Parameters
  • Block Feed
  • Block Spam Directories
  • Block archive.org bots
  • Block Chatgpt
  • Block Cart Page
  • Block Checkout Page
  • Block My Account Page
  • Block Login Page
  • Block Woocommerce Parameters
  • Rankmath Sitemap Link
  • News Sitemap Link
  • Block Semrush Crawler
  • Block Moz Crawler
  • Block Majestic Crawler
  • Block Xenu Crawler
  • Allow Google Bot
  • Allow Google Images Bot
  • Allow Google Media Partners Bot
  • Allow Google AdsBot Bot
  • Allow Google Mobile Bot
  • Allow Bing Bot
  • Allow Apple Bot
  • Allow MSNBot Media Bot
  • Allow MSN Bot
  • Allow Yandex Bot
  • Allow Yandex Images Bot
  • Allow Yahoo Search (Slurp bot)
  • Allow Qwant Bot
  • Allow DuckDuckGo Bot
  • Allow Seznam Bot
  • Allow Naver Bot
  • Allow Baidu/Sogou/Soso/Youdao Bot
  • Allow Webp Images
  • Allow Jpg Images
  • Allow Png Images
  • Allow PDF Files
  • Allow Gif Images
  • Allow Discord Bot
  • Allow Pinterest Bot
  • Allow Linkedin Bot
  • Allow Twitter Bot
  • Allow Telegram Bot
  • Allow Whatsapp Bot
  • Allow Instagram Bot
  • Allow Facebook Bot
  • Allow Php Files
  • Allow Html Files
  • Allow DOCX Files
Back to top