Log In Sign Up

able.me
robots.txt

Robots Exclusion Standard data for able.me

Resource Scan

Scan Details

Site Domain able.me
Base Domain able.me
Scan Status Ok
Last Scan2024-11-14T03:49:36+00:00
Next Scan 2024-11-21T03:49:36+00:00

Last Scan

Scanned2024-11-14T03:49:36+00:00
URL http://able.me/robots.txt
Domain IPs 78.46.34.44
Response IP 78.46.34.44
Found Yes
Hash 549339524b5d1ccd3d6ab4fe7897653634f183da439ea4b14757320e073dfa96
SimHash 2a31e429ccf3

Groups

*

Rule Path
Disallow /~*
Disallow /?signup
Disallow /re.php*
Disallow *counter.php*
Disallow *wp-admin/*
Disallow *wp-login.php*
Disallow *wp-signup.php*
Disallow *wp-*.php*
Disallow *wp-*/*
Disallow /?new*

Other Records

Field Value
crawl-delay 10

ahrefsbot

Rule Path
Disallow /

aitcsrobot

Rule Path
Disallow /

alexibot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

arachnophilia

Rule Path
Disallow /

aspider

Rule Path
Disallow /

backdoorbot

Rule Path
Disallow /

bilbo

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bspider

Rule Path
Disallow /

bubing

Rule Path
Disallow /

cfnetwork

Rule Path
Disallow /

cyberpatrol

Rule Path
Disallow /

deusu

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

exabot

Rule Path
Disallow /

feedlybot

Rule Path
Disallow /

genieo

Rule Path
Disallow /

gluten\ free\ crawler

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

maxpointcrawler

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

pagesinventory

Rule Path
Disallow /

php

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

riddler

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

seokicks\-robot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

synapse

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /

wesee\_bot

Rule Path
Disallow /

xenu\ link\ sleuth

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

yahoo!\ slurp

Rule Path
Disallow /

zeus

Rule Path
Disallow /

hacker

Rule Path
Disallow /

human

Rule Path
Disallow /adult/
Disallow /crime/
Disallow /hack/
Disallow /porno/
Disallow /spam/
Disallow /violence/

spammer

Rule Path
Disallow /
Back to top

Comments

  • Welcome to Name.ly. Below are our basic home rules.
  • AhrefsBot, Applebot, Bilbo, MJ12bot, SemrushBot, Vagabondo et al please contact us if you think the ban is unfair, but come up with some extra sweet cherry on top of the cake
Back to top