1baiser.com
robots.txt

Robots Exclusion Standard data for 1baiser.com

Resource Scan

Scan Details

Site Domain 1baiser.com
Base Domain 1baiser.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-11-12T16:36:07+00:00
Next Scan 2024-11-19T16:36:07+00:00

Last Successful Scan

Scanned2024-10-12T06:01:27+00:00
URL https://1baiser.com/robots.txt
Domain IPs 104.26.6.69, 104.26.7.69, 172.67.68.223, 2606:4700:20::681a:645, 2606:4700:20::681a:745, 2606:4700:20::ac43:44df
Response IP 104.26.6.69
Found Yes
Hash 0a5217f4fe3d69408d7f43b5934a486998d9c7608317b6397c010f6b44acf928
SimHash d0157240cc33

Groups

*

Rule Path
Allow /

mj12bot

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

ip-web-crawler.com

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

aboundexbot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

nutch

Rule Path
Disallow /

proximic

Rule Path
Disallow /

spbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

wesee

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

advbot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

ntentbot

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

deepcrawl

Rule Path
Disallow /

brightbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

buck/2.3.2

Rule Path
Disallow /

Warnings

  • 2 invalid lines.