bobbiebush.info
robots.txt

Robots Exclusion Standard data for bobbiebush.info

Resource Scan

Scan Details

Site Domain bobbiebush.info
Base Domain bobbiebush.info
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-05-03T03:31:46+00:00
Next Scan 2024-07-02T03:31:46+00:00

Last Successful Scan

Scanned2021-09-30T20:17:50+00:00
URL http://bobbiebush.info/robots.txt
Redirect http://www.bobbiebush.com/robots.txt
Redirect Domain www.bobbiebush.com
Redirect Base bobbiebush.com
Found Yes
Hash bdba79e7178d3424615782b3d92c5b2996f6592809b0e9ff7efedb10608bcb4b
SimHash 114cc85cc3b3

Groups

googlebot

Rule Path
Allow *.css
Allow *.js

twitterbot

Rule Path
Disallow

facebookexternalhit

Rule Path
Disallow

archive.org_bot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

aboundex

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

deusu

Rule Path
Disallow /

*

Rule Path
Disallow /ajax/
Disallow /cart/
Disallow /cfdocs/
Disallow /CFIDE/
Disallow /global/
Disallow /image*/
Disallow /includes/
Disallow /js/
Disallow /controlpanel/
Disallow /corporate/
Disallow /emailsupport/
Disallow /photobizadmin/
Disallow /music/
Disallow /xml/
Disallow /portfolio/
Disallow /fckeditor/
Disallow /movies/
Disallow /qacontrol/
Disallow /billing/
Disallow /fonts/
Disallow /viewing/
Disallow /store/share.cfm

Other Records

Field Value
crawl-delay 60

Other Records

Field Value
sitemap https://www.bobbiebush.com/sitemap.xml