Log In Sign Up

businessgiftuk.com
robots.txt

Robots Exclusion Standard data for businessgiftuk.com

Resource Scan

Scan Details

Site Domain businessgiftuk.com
Base Domain businessgiftuk.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-08-21T01:15:29+00:00
Next Scan 2024-11-19T01:15:29+00:00

Last Successful Scan

Scanned2023-10-03T22:00:15+00:00
URL https://businessgiftuk.com/robots.txt
Domain IPs 185.53.175.42
Response IP 185.53.175.42
Found Yes
Hash 55e0eb00910556d52def4845654c157be28a86f993ae186c036af773edde784d
SimHash 352c8f70fe98

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

yandexbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

bl.uk_lddc_bot

Rule Path
Disallow /

adcore

Rule Path
Disallow /

ipvanish

Rule Path
Disallow /

requestedoffers.clu

Rule Path
Disallow /

notarius-russia.ru

Rule Path
Disallow /

bitminer.it

Rule Path
Disallow /

secretsharinghub.com

Rule Path
Disallow /

ibsconsultant.com

Rule Path
Disallow /

generositycool.com

Rule Path
Disallow /

ioflood.com

Rule Path
Disallow /

shootorder.com

Rule Path
Disallow /

visualpleasures.net

Rule Path
Disallow /

ninja-crawler

Rule Path
Disallow /

audit

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

opensiteexplorer.org

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

go-resty

Rule Path
Disallow /

crownpeak

Rule Path
Disallow /

clearscope.io

Rule Path
Disallow /

seopowersuite

Rule Path
Disallow /

webceo

Rule Path
Disallow /

cognitiveseo

Rule Path
Disallow /

netpeaksoftware

Rule Path
Disallow /

majestic.com

Rule Path
Disallow /

agencyanalytics

Rule Path
Disallow /

netpeaksoftware

Rule Path
Disallow /

brightedge

Rule Path
Disallow /

raventools

Rule Path
Disallow /

conductor.com

Rule Path
Disallow /

seranking

Rule Path
Disallow /

ispionage

Rule Path
Disallow /

similarweb

Rule Path
Disallow /

buzzsumo

Rule Path
Disallow /

sproutsocial

Rule Path
Disallow /

siteimprove.com

Rule Path
Disallow /

ahrefs.com

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

webmeup.com

Rule Path
Disallow /

webmeup-crawler.com

Rule Path
Disallow /

yandex

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

webmeup.com

Rule Path
Disallow /

webmeup-crawler.com

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://businessgiftuk.com/wp-sitemap.xml
Back to top

Comments

  • Block bots
Back to top

Warnings

  • 4 invalid lines.
Back to top