Log In Sign Up

faturl.com
robots.txt

Robots Exclusion Standard data for faturl.com

Resource Scan

Scan Details

Site Domain faturl.com
Base Domain faturl.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-07-10T02:02:28+00:00
Next Scan 2024-10-08T02:02:28+00:00

Last Successful Scan

Scanned2024-03-19T01:36:47+00:00
URL http://faturl.com/robots.txt
Domain IPs 78.46.34.44
Response IP 78.46.34.44
Found Yes
Hash 549339524b5d1ccd3d6ab4fe7897653634f183da439ea4b14757320e073dfa96
SimHash 2a31e429ccf3

Groups

*

Rule Path
Disallow /~*
Disallow /?signup
Disallow /re.php*
Disallow *counter.php*
Disallow *wp-admin/*
Disallow *wp-login.php*
Disallow *wp-signup.php*
Disallow *wp-*.php*
Disallow *wp-*/*
Disallow /?new*

Other Records

Field Value
crawl-delay 10

ahrefsbot

Rule Path
Disallow /

aitcsrobot

Rule Path
Disallow /

alexibot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

arachnophilia

Rule Path
Disallow /

aspider

Rule Path
Disallow /

backdoorbot

Rule Path
Disallow /

bilbo

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bspider

Rule Path
Disallow /

bubing

Rule Path
Disallow /

cfnetwork

Rule Path
Disallow /

cyberpatrol

Rule Path
Disallow /

deusu

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

exabot

Rule Path
Disallow /

feedlybot

Rule Path
Disallow /

genieo

Rule Path
Disallow /

gluten\ free\ crawler

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

maxpointcrawler

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

pagesinventory

Rule Path
Disallow /

php

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

riddler

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

seokicks\-robot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

synapse

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /

wesee\_bot

Rule Path
Disallow /

xenu\ link\ sleuth

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

yahoo!\ slurp

Rule Path
Disallow /

zeus

Rule Path
Disallow /

hacker

Rule Path
Disallow /

human

Rule Path
Disallow /adult/
Disallow /crime/
Disallow /hack/
Disallow /porno/
Disallow /spam/
Disallow /violence/

spammer

Rule Path
Disallow /
Back to top

Comments

  • Welcome to Name.ly. Below are our basic home rules.
  • AhrefsBot, Applebot, Bilbo, MJ12bot, SemrushBot, Vagabondo et al please contact us if you think the ban is unfair, but come up with some extra sweet cherry on top of the cake
Back to top