bevywise.com
robots.txt

Robots Exclusion Standard data for bevywise.com

Resource Scan

Scan Details

Site Domain bevywise.com
Base Domain bevywise.com
Scan Status Ok
Last Scan2024-09-19T23:13:52+00:00
Next Scan 2024-10-19T23:13:52+00:00

Last Scan

Scanned2024-09-19T23:13:52+00:00
URL https://bevywise.com/robots.txt
Redirect https://www.bevywise.com/robots.txt
Redirect Domain www.bevywise.com
Redirect Base bevywise.com
Domain IPs 104.21.48.161, 172.67.154.85, 2606:4700:3033::ac43:9a55, 2606:4700:3037::6815:30a1
Redirect IPs 104.21.48.161, 172.67.154.85, 2606:4700:3033::ac43:9a55, 2606:4700:3037::6815:30a1
Response IP 172.67.154.85
Found Yes
Hash b0ad334e3e046947fadd8be4ebd5c140f57d8c5867318840c96bbde27f414c15
SimHash 6e17d63377fb

Groups

*

Rule Path
Disallow /payments/wp-admin
Disallow /payments/wp-includes
Disallow /payments/wp-content/plugins
Disallow /payments/wp-content/cache
Disallow /payments/trackback
Disallow /payments/feed
Disallow /payments/wp-json/
Disallow /payments/readme.html
Disallow /payments/comments
Disallow /payments/category/*/*
Disallow */payments/trackback
Disallow */payments/feed
Disallow */payments/comments
Disallow /*.pdf$
Disallow /*?*
Disallow /*?
Disallow /payments/wp-login.php
Disallow /payments/wp-content/themes/V4/assets/js/optim/*
Disallow /cdn-cgi/*
Disallow /payments/author/*
Disallow /payments/product-category/*
Disallow /payments/product/*
Disallow /iot-solutions/*
Disallow /forum/*
Disallow /tcp-server/*
Disallow /?s=*
Disallow /search/*

googlebot

Rule Path
Disallow /*.php$
Disallow /*.inc$
Disallow /*.gz$
Disallow /*.pdf$
Disallow
Allow /*

nuclei
wikido
riddler
petalbot
zoominfobot
go-http-client
node/simplecrawler
cazoodlebot
dotbot/1.0
gigabot
barkrowler
blexbot
magpie-crawler

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.bevywise.com/sitemap.xml

Comments

  • Global rules
  • -----------------
  • Prevent crawling URLs
  • experimenting with blocking search results to prevent search result spam
  • Autoriser Google Image
  • Sitemap
  • -----------------
  • Ban bots that don't benefit us.
  • --------------------------------