Log In Sign Up

bitbytesoft.com
robots.txt

Robots Exclusion Standard data for bitbytesoft.com

Resource Scan

Scan Details

Site Domain bitbytesoft.com
Base Domain bitbytesoft.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-04-14T09:44:11+00:00
Next Scan 2025-04-21T09:44:11+00:00

Last Successful Scan

Scanned2025-04-06T09:43:35+00:00
URL https://bitbytesoft.com/robots.txt
Domain IPs 104.21.37.173, 172.67.210.199, 2606:4700:3037::6815:25ad, 2606:4700:3037::ac43:d2c7
Response IP 172.67.210.199
Found Yes
Hash dafe11230cc2ca3e3ced21a8a9173eb0c2812caf013a72d9aca7716d93d63cc3
SimHash 2e549513aff2

Groups

*

Rule Path
Disallow /wp-json/

ccbot

Rule Path
Disallow /
Disallow /?s=*
Disallow /search/*
Disallow /wp-admin
Disallow /wp-includes
Disallow /trackback
Disallow /*.php$
Disallow /*.inc$
Disallow /*.gz$
Disallow /wp-login.php
Back to top

Other Records

Field Value
sitemap https://bitbytesoft.com/sitemap_index.xml
Back to top

Comments

  • We're experimenting with blocking search results to prevent search result spam
  • Indexing of sensitive files is prevented
  • We de-index the login page (unnecessary content)
  • Sitemap
  • -----------------
Back to top