chevelles.net
robots.txt
Robots Exclusion Standard data for chevelles.net
Resource Scan
Scan Details
| Site Domain | chevelles.net |
| Base Domain | chevelles.net |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-05-04T20:32:28+00:00 |
| Next Scan | 2024-08-02T20:32:28+00:00 |
Last Successful Scan
| Scanned | 2023-10-08T16:36:51+00:00 |
| URL | https://www.chevelles.net/robots.txt |
| Domain IPs | 151.101.1.91, 151.101.129.91, 151.101.193.91, 151.101.65.91 |
| Response IP | 199.232.45.91 |
| Found | Yes |
| Hash | 53f34438abf5329e714b314eb8db365d248548b79d5872eea9f1f38d73f3784d |
| SimHash | 72de46456c1d |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
bingbot
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /forum/install/ |
| Disallow | /forum/customavatars/ |
| Disallow | /forum/admincp/ |
| Disallow | /forum/clientscript/ |
| Disallow | /forum/cpstyles/ |
| Disallow | /forum/customprofilepics/ |
| Disallow | /forum/modcp/ |
| Disallow | /forum/ajax.php |
| Disallow | /forum/attachment.php |
| Disallow | /forum/calendar.php |
| Disallow | /forum/cron.php |
| Disallow | /forum/editpost.php |
| Disallow | /forum/global.php |
| Disallow | /forum/image.php |
| Disallow | /forum/inlinemod.php |
| Disallow | /forum/joinrequests.php |
| Disallow | /forum/login.php |
| Disallow | /forum/members/ |
| Disallow | /forum/members.php |
| Disallow | /forum/misc.php |
| Disallow | /forum/moderator.php |
| Disallow | /forum/newattachment.php |
| Disallow | /forum/newreply.php |
| Disallow | /forum/newthread.php |
| Disallow | /forum/online.php |
| Disallow | /forum/postings.php |
| Disallow | /forum/printthread.php |
| Disallow | /forum/private.php |
| Disallow | /forum/register.php |
| Disallow | /forum/report.php |
| Disallow | /forum/reputation.php |
| Disallow | /forum/search.php |
| Disallow | /forum/sendmessage.php |
| Disallow | /forum/showgroups.php |
| Disallow | /forum/showpost.php |
| Disallow | /forum/subscription.php |
| Disallow | /forum/threadrate.php |
| Disallow | /forum/usercp.php |
| Disallow | /forum/usernote.php |
| Disallow | /forum/customgroupicons/ |
| Disallow | /forum/signaturepics/ |
| Disallow | /forum/showthread.php |
| Disallow | /forum/member.php |
| Disallow | /forum/forumdisplay.php |
| Disallow | /forum/blog.php |
| Disallow | /forum/activity.php |
| Disallow | /forum/index.php |
| Disallow | /forum/itrader.php |
| Disallow | /forum/garage.php |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 4 |
slurp
yahoo! slurp
msnbot
msnbot-media
twitterbot
tweetedtimes bot
tweetmemebot
paperlibot
tineye-bot
qwantify
exabot
coccoc
blexbot
umbot-ln
queryseekerspider
yandex
yandexbot
yandexmobilebot
yandeximageresizer
yandeximages
sogou spider
sogou web spider
baiduspider
youdaobot
mj12bot
spbot
seznambot
ahrefsbot
linkdex
linkdexbot
semrushbot
semrushbot-sa
cfnetwork
sputnikbot
turnitinbot
proximic
grapeshot
viglink
clickagy
yak
deusu
taboolabot
pinterestbot
psbot
plukkie
getintent
facebot
linkdexbot
vegi
istellabot
leikibot
claritybot
piplbot
feedspotbot
moatbot
yeti
contxbot
seokicks-robot
rogerbot
discordbot
tapatalk
mail.ru_bot
coccocbot
hybridbot
slack-imgproxy
twitterbot
adbeat_bot
kraken
mojeekbot
calculon
mcsbot
avkzarabotok
searchmetricsbot
uptimebot
yacybot
applebot
gigabot
g2reader-bot
seoterritory
ahrefsbot
feedbot
findxbot
wotbox
msnbot
smtbot
ocarinabot
dragonbot
zombiebot
metabot.ru
rssmicro
_zbot
uptimerobot
uipbot
zoominfobot
flamingo_searchengine
bot-pge.chlooe.com
girafabot
multiviewbot
safednsbot
gnowitnewsbot
cliqzbot
dotbot
exabot
moatbot
tapatalk
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
Warnings
- 1 invalid line.