footballforum.com
robots.txt
Robots Exclusion Standard data for footballforum.com
Resource Scan
Scan Details
| Site Domain | footballforum.com |
| Base Domain | footballforum.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-04-26T02:09:38+00:00 |
| Next Scan | 2024-07-25T02:09:38+00:00 |
Last Successful Scan
| Scanned | 2023-09-29T01:05:10+00:00 |
| URL | https://www.footballforum.com/robots.txt |
| Domain IPs | 151.101.1.91, 151.101.129.91, 151.101.193.91, 151.101.65.91 |
| Response IP | 199.232.45.91 |
| Found | Yes |
| Hash | 163b85fa7fee3a96b8ca202036658107722f9c7ebd5c32b02db6f6d2d87681ee |
| SimHash | 738f47476e5c |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /admincp/ |
| Disallow | /cgi-bin/ |
| Disallow | /clientscript/ |
| Disallow | /includes/ |
| Disallow | /install/ |
| Disallow | /gallery/showmembers.php |
| Disallow | /gallery/misc.php |
| Disallow | /gallery/comments.php? |
| Disallow | /modcp/ |
| Disallow | /subscriptions/ |
| Disallow | /announcement.php |
| Disallow | /attachment.php |
| Disallow | /calendar.php |
| Disallow | /cron.php |
| Disallow | /editpost.php |
| Disallow | /faq.php |
| Disallow | /global.php |
| Disallow | /image.php |
| Disallow | /joinrequests.php |
| Disallow | /login.php |
| Disallow | /misc.php |
| Disallow | /moderator.php |
| Disallow | /newattachment.php |
| Disallow | /newreply.php |
| Disallow | /newthread.php |
| Disallow | /online.php |
| Disallow | /poll.php |
| Disallow | /postings.php |
| Disallow | /printthread.php |
| Disallow | /private.php |
| Disallow | /register.php |
| Disallow | /report.php |
| Disallow | /reputation.php |
| Disallow | /search.php |
| Disallow | /sendmessage.php |
| Disallow | /showgroups.php |
| Disallow | /showpost.php |
| Disallow | /showthread.php |
| Disallow | /subscription.php |
| Disallow | /subscriptions.php |
| Disallow | /threadrate.php |
| Disallow | /traderratings.php |
| Disallow | /usercp.php |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
bingbot
| Rule | Path |
|---|---|
| Disallow | /install/ |
| Disallow | /customavatars/ |
| Disallow | /admincp/ |
| Disallow | /clientscript/ |
| Disallow | /cpstyles/ |
| Disallow | /customprofilepics/ |
| Disallow | /modcp/ |
| Disallow | /ajax.php |
| Disallow | /attachment.php |
| Disallow | /calendar.php |
| Disallow | /cron.php |
| Disallow | /editpost.php |
| Disallow | /global.php |
| Disallow | /image.php |
| Disallow | /inlinemod.php |
| Disallow | /joinrequests.php |
| Disallow | /login.php |
| Disallow | /members/ |
| Disallow | /members.php |
| Disallow | /misc.php |
| Disallow | /moderator.php |
| Disallow | /newattachment.php |
| Disallow | /newreply.php |
| Disallow | /newthread.php |
| Disallow | /online.php |
| Disallow | /postings.php |
| Disallow | /printthread.php |
| Disallow | /private.php |
| Disallow | /register.php |
| Disallow | /report.php |
| Disallow | /reputation.php |
| Disallow | /search.php |
| Disallow | /sendmessage.php |
| Disallow | /showgroups.php |
| Disallow | /showpost.php |
| Disallow | /subscription.php |
| Disallow | /threadrate.php |
| Disallow | /usercp.php |
| Disallow | /usernote.php |
| Disallow | /customgroupicons/ |
| Disallow | /signaturepics/ |
| Disallow | /showthread.php |
| Disallow | /member.php |
| Disallow | /forumdisplay.php |
| Disallow | /blog.php |
| Disallow | /activity.php |
| Disallow | /index.php |
| Disallow | /itrader.php |
| Disallow | /garage.php |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 4 |
slurp
yahoo! slurp
msnbot
msnbot-media
twitterbot
baiduspider
tweetedtimes bot
tweetmemebot
paperlibot
bomborabot
wf search
heritrix
expo9
netseer
addthis.com
qwantify
feed
sogou
zumbot
feedspot
acebookexternalhit
facebookexternalhit
storygizebot
scrapy
germcrawler
netvibes
vagabondo
genieo
um-ic
typhoeus
newspaper
weborama-fetcher
scooperbot
byo-2
simplepie
grouphigh
feedparser
universalfeedparser
pulsepoint
steeler
garlikcrawler
flipboard
flipboardrss
moreover
mercury
feedburner
yahoo
omgili
bloglovin
yahoocachesystem
mtalk
everyonesocialbot
ezinearticleslinkscanner
dlvr.it
pfblogs
aol
toutiaospider
photon
picofeed
um-fc
digg
synaptive
yisouspider
byo-4
feedbin
twingly
tiny
www-mechanize
commafeed
tkbot
tineye-bot
qwantify
exabot
coccoc
blexbot
umbot-ln
queryseekerspider
yandex
yandexbot
yandexmobilebot
yandeximageresizer
yandeximages
sogou spider
sogou web spider
baiduspider
youdaobot
mj12bot
spbot
seznambot
ahrefsbot
linkdex
linkdexbot
semrushbot
semrushbot-sa
cfnetwork
sputnikbot
turnitinbot
proximic
grapeshot
viglink
clickagy
yak
deusu
taboolabot
pinterestbot
psbot
plukkie
getintent
facebot
linkdexbot
vegi
istellabot
leikibot
claritybot
piplbot
feedspotbot
moatbot
yeti
contxbot
seokicks-robot
rogerbot
discordbot
tapatalk
mail.ru_bot
coccocbot
hybridbot
slack-imgproxy
twitterbot
adbeat_bot
kraken
mojeekbot
calculon
mcsbot
avkzarabotok
searchmetricsbot
uptimebot
yacybot
applebot
gigabot
g2reader-bot
seoterritory
ahrefsbot
feedbot
findxbot
wotbox
msnbot
smtbot
ocarinabot
dragonbot
zombiebot
metabot.ru
rssmicro
_zbot
uptimerobot
uipbot
zoominfobot
flamingo_searchengine
bot-pge.chlooe.com
girafabot
multiviewbot
safednsbot
gnowitnewsbot
treato-bot
pvblcbot
laserlikebot
trendiction
mediatoolkitbot
charliebot
bubing
superfeedr
focusbot
sottopop
maxpointcrawler
nutch-1.10
wonderbot
zarabotok--doma
treato-bot
pvblcbot
laserlikebot
trendiction
mediatoolkitbot
charliebot
bubing
superfeedr
focusbot
sottopop
maxpointcrawler
nutch-1.10
wonderbot
zarabotok--doma
uptimerobot
uipbot
zoominfobot
flamingo_searchengine
bot-pge.chlooe.com
girafabot
multiviewbot
safednsbot
gnowitnewsbot
treato-bot
pvblcbot
laserlikebot
trendiction
mediatoolkitbot
charliebot
bubing
superfeedr
focusbot
sottopop
maxpointcrawler
nutch-1.10
wonderbot
zarabotok--doma
treato-bot
pvblcbot
laserlikebot
trendiction
mediatoolkitbot
charliebot
bubing
superfeedr
focusbot
sottopop
maxpointcrawler
nutch-1.10
wonderbot
zarabotok--doma
cliqzbot
dotbot
exabot
moatbot
tapatalk
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
Warnings
- 1 invalid line.