Log In Sign Up

thefinalfantasy.net
robots.txt

Robots Exclusion Standard data for thefinalfantasy.net

Resource Scan

Scan Details

Site Domain thefinalfantasy.net
Base Domain thefinalfantasy.net
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-11-12T12:52:12+00:00
Next Scan 2025-02-10T12:52:12+00:00

Last Successful Scan

Scanned2024-01-18T12:35:55+00:00
URL https://thefinalfantasy.net/robots.txt
Domain IPs 104.21.56.250, 172.67.157.60, 2606:4700:3031::6815:38fa, 2606:4700:3036::ac43:9d3c
Response IP 104.21.56.250
Found Yes
Hash cc3d6529aed9f12d673747cd7dc27174d89660d8b179228ca0556c9ff123c732
SimHash 2a5dd3d0ec80

Groups

*

Rule Path
Disallow /*?act=
Disallow /*?f=
Disallow /*search.php
Disallow /polls
Disallow /generator
Disallow /forums/ajax.php
Disallow /forums/calendar.php
Disallow /forums/cron.php
Disallow /forums/editpost.php
Disallow /forums/inlinemod.php
Disallow /forums/joinrequests.php
Disallow /forums/login.php
Disallow /forums/misc.php
Disallow /forums/moderator.php
Disallow /forums/newreply.php
Disallow /forums/newthread.php
Disallow /forums/online.php
Disallow /forums/postings.php
Disallow /forums/printthread.php
Disallow /forums/private.php
Disallow /forums/profile.php
Disallow /forums/register.php
Disallow /forums/report.php
Disallow /forums/reputation.php
Disallow /forums/search.php
Disallow /forums/sendmessage.php
Disallow /forums/showgroups.php
Disallow /forums/subscription.php
Disallow /forums/threadrate.php
Disallow /forums/usercp.php
Disallow /forums/usernote.php

aboundexbot

Rule Path
Disallow /

acapbot

Rule Path
Disallow /

admantx

Rule Path
Disallow /

adnormcrawler

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

aolbot/4.0

Rule Path
Disallow /

attribot

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

bot.araturka.com

Rule Path
Disallow /

butterfly/1.0

Rule Path
Disallow /

buzzsumo

Rule Path
Disallow /

cis455crawler

Rule Path
Disallow /

claritybot

Rule Path
Disallow /

claritydailybot

Rule Path
Disallow /

clickagy intelligence bot

Rule Path
Disallow /

cms crawler

Rule Path
Disallow /

commoncrawler

Rule Path
Disallow /

crawler4j (http://code.google.com/p/crawler4j/)

Rule Path
Disallow /

crazywebcrawler

Rule Path
Disallow /

crazywebcrawler-spider

Rule Path
Disallow /

dataparksearch

Rule Path
Disallow /

deepcrawl

Rule Path
Disallow /

deusu

Rule Path
Disallow /

dialogsearch.com

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

dj-research

Rule Path
Disallow /

domainappender

Rule Path
Disallow /

domainmacrocrawler

Rule Path
Disallow /

domainsigmacrawler

Rule Path
Disallow /

domain re-animator bot

Rule Path
Disallow /

ezooms robot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

experibot

Rule Path
Disallow /

favorg

Rule Path
Disallow /

findxbot

Rule Path
Disallow /

flipboardproxy

Rule Path
Disallow /

friendica

Rule Path
Disallow /

getproxi.es-bot

Rule Path
Disallow /

gigablastopensource/1.0

Rule Path
Disallow /

gluten free crawler

Rule Path
Disallow /

google-http-java-client

Rule Path
Disallow /

gocrawl

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

guzzlehttp

Rule Path
Disallow /

hatena star

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

httrack 3

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

inagist url resolver

Rule Path
Disallow /

insitesbot

Rule Path
Disallow /

iskanie

Rule Path
Disallow /

jack

Rule Path
Disallow /

james bot

Rule Path
Disallow /

java

Rule Path
Disallow /

js-kit url resolver

Rule Path
Disallow /

komodiabot

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

livelapbot

Rule Path
Disallow /

ls session

Rule Path
Disallow /

lssrocketcrawler

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

madaali.de

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

memorybot

Rule Path
Disallow /

metauri api/2.0 +metauri.com

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mozilla/4.0 (cms crawler: http://www.cmscrawler.com)

Rule Path
Disallow /

mozilla/4.0 (compatible; msie 4.01; windows nt; ms search 4.0 robot)

Rule Path
Disallow /

mozilla/4.0 (compatible; msie 6.0; windows nt; ms search 4.0 robot)

Rule Path
Disallow /

mozilla/4.0 (compatible; msie 4.01; windows nt; ms search 5.0 robot)

Rule Path
Disallow /

mozilla/4.0 (compatible; msie 6.0; windows nt; ms search 5.0 robot)

Rule Path
Disallow /

mozilla/4.0 (compatible; msie 8.0; windows nt 6.0 ; claritydailybot)

Rule Path
Disallow /

mozilla/5.0 (compatible; 200pleasebot/1.0; +http://www.200please.com/bot)

Rule Path
Disallow /

mozilla/5.0 (compatible;acapbot/0.1;treat like googlebot)

Rule Path
Disallow /

mozilla/5.0 (compatible; ahrefsbot/5.0; +http://ahrefs.com/robot/)

Rule Path
Disallow /

mozilla/5.0 (compatible; aolbot/4.0; +http://www.aol-soft.com/)

Rule Path
Disallow /

mozilla/5.0 (compatible; domainappender /1.0; +http://www.profound.net/domainappender)

Rule Path
Disallow /

mozilla/5.0 (compatible; domainsigmacrawler/0.1; +http://domainsigma.com/robot)

Rule Path
Disallow /

mozilla/5.0 (compatible; exabot/3.0; +http://www.exabot.com/go/robot)

Rule Path
Disallow /

mozilla/5.0 (compatible; findxbot/1.0; +http://www.findxbot.com)

Rule Path
Disallow /

mozilla/5.0 (compatible; genieo/1.0 http://www.genieo.com/webfilter.html)

Rule Path
Disallow /

mozilla/5.0 (compatible; linkdexbot/2.0; +http://www.linkdex.com/bots/)

Rule Path
Disallow /

mozilla/5.0 (compatible; meanpathbot/1.0; +http://www.meanpath.com/meanpathbot.html)

Rule Path
Disallow /

mozilla/5.0 (compatible; mj12bot/v1.4.5; http://www.majestic12.co.uk/bot.php?+)

Rule Path
Disallow /

mozilla/5.0 (compatible; openhosebot/2.1; +http://www.openhose.org/bot.html)

Rule Path
Disallow /

mozilla/5.0 (compatible; pad-bot/9.0; +http://www.descargarprogramagratis.com/)

Rule Path
Disallow /

mozilla/5.0 (compatible; paperlibot/2.1; http://support.paper.li/entries/20023257-what-is-paper-li)

Rule Path
Disallow /

mozilla/5.0 (compatible; semrushbot-si/0.97; +http://www.semrush.com/bot.html)

Rule Path
Disallow /

mozilla/5.0 (compatible; softlistbot/2.2; +http://www.softlist.us/)

Rule Path
Disallow /

mozilla/5.0 (compatible; tweetedtimes bot/1.0; +http://tweetedtimes.com)

Rule Path
Disallow /

mozilla/5.0 (compatible; tweetmemebot/3.0; +http://tweetmeme.com/)

Rule Path
Disallow /

mozilla/5.0 (compatible; umbot-ln/1.0; mailto: crawling@ubermetrics-technologies.com)

Rule Path
Disallow /

mozilla/5.0 (compatible; xovibot/2.0; +http://www.xovibot.net/)

Rule Path
Disallow /

mozilla/5.0 (macintosh; intel mac os x 10.9; rv:28.0) gecko/20100101 firefox/28.0 (flipboardproxy/1.1; +http://flipboard.com/browserproxy)

Rule Path
Disallow /

mozilla/5.0 (windows; u; windows nt 5.1; en-us; rv:1.8.1.6) gecko/20070725 firefox/2.0.0.6 - james bot - webcrawler http://cognitiveseo.com/bot.html

Rule Path
Disallow /

mozilla/5.0 (windows; u; windows nt 5.1; en-us; rv:1.9.1.2) gecko/20090729 firefox/3.5.2 (.net clr 3.5.30729; diffbot/0.1; +http://www.diffbot.com)

Rule Path
Disallow /

mozilla/5.0 (windows nt 6.2) insitesbot/1.0

Rule Path
Disallow /

ms search 4.0 robot

Rule Path
Disallow /

ms search 5.0 robot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

nativehost

Rule Path
Disallow /

nerdybot

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

netseer crawler

Rule Path
Disallow /

netshelter contentscan

Rule Path
Disallow /

newsme/1.0; feedback@news.me

Rule Path
Disallow /

niki-bot

Rule Path
Disallow /

ning/1.0

Rule Path
Disallow /

node/simplecrawler 0.3.9 (http://github.com/cgiffard/node-simplecrawler.git)

Rule Path
Disallow /

openhosebot

Rule Path
Disallow /

pad-bot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

peerindex

Rule Path
Disallow /

perl lwp

Rule Path
Disallow /

photon

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

postano

Rule Path
Disallow /

proximic

Rule Path
Disallow /

pulsecrawler/1.1

Rule Path
Disallow /

pulsepoint xt3 web scraper

Rule Path
Disallow /

queryseekerspider

Rule Path
Disallow /

quipu

Rule Path
Disallow /

raven

Rule Path
Disallow /

ravencrawler

Rule Path
Disallow /

rbot

Rule Path
Disallow /

recorded future

Rule Path
Disallow /

riddler

Rule Path
Disallow /

ruby

Rule Path
Disallow /

safednsbot

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

searchie

Rule Path
Disallow /

semantic-visions.com crawler

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

seolyticscrawler

Rule Path
Disallow /

showyoubot (http://showyou.com/crawler)

Rule Path
Disallow /

simplecrawler

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

siteluxbot

Rule Path
Disallow /

smeshbot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

srmse/nutch

Rule Path
Disallow /

softlistbot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

spbot

Rule Path
Disallow /

squirrly

Rule Path
Disallow /

ssearch_bot

Rule Path
Disallow /

stratagems kumo

Rule Path
Disallow /

tbot-nutch

Rule Path
Disallow /

telegrambot

Rule Path
Disallow /

tweetmemebot

Rule Path
Disallow /

twmbot

Rule Path
Disallow /

typhoeus - https://github.com/typhoeus/typhoeus

Rule Path
Disallow /

umbot-ln

Rule Path
Disallow /

uptimerobot/2.0

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

webindex

Rule Path
Disallow /

vebot

Rule Path
Disallow /

wesee_bot

Rule Path
Disallow /

wiseguys robot

Rule Path
Disallow /

voltron

Rule Path
Disallow /

woobot

Rule Path
Disallow /

woriobot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

x-cad-se

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

yahoo pipes 1.0

Rule Path
Disallow /

yioopbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://thefinalfantasy.net/sitemap.xml
Back to top

Comments

  • Blocklist
Back to top

Warnings

  • 6 invalid lines.
Back to top