animecharactersdatabase.com
robots.txt

Robots Exclusion Standard data for animecharactersdatabase.com

Resource Scan

Scan Details

Site Domain animecharactersdatabase.com
Base Domain animecharactersdatabase.com
Scan Status Ok
Last Scan2024-06-01T07:01:06+00:00
Next Scan 2024-06-08T07:01:06+00:00

Last Scan

Scanned2024-06-01T07:01:06+00:00
URL https://animecharactersdatabase.com/robots.txt
Domain IPs 167.114.1.56, 2607:5300:60:6438::acdb
Response IP 167.114.1.56
Found Yes
Hash b00e302d28ebbb1c8df52c07502fd1e768958919970b7df0f5b40e4598ca5750
SimHash 01191a26c931

Groups

*

Rule Path
Disallow /who.php
Disallow /contact.php
Disallow /includes
Disallow /private
Disallow /inc
Disallow /af/af/
Disallow /af/cn/
Disallow /af/en/
Disallow /af/fi/
Disallow /af/fr/
Disallow /af/ge/
Disallow /af/in/
Disallow /af/jp/
Disallow /af/kr/
Disallow /af/sp/
Disallow /af/si/
Disallow /cn/af/
Disallow /cn/cn/
Disallow /cn/en/
Disallow /cn/fi/
Disallow /cn/fr/
Disallow /cn/ge/
Disallow /cn/in/
Disallow /cn/jp/
Disallow /cn/kr/
Disallow /cn/sp/
Disallow /cn/si/
Disallow /en/af/
Disallow /en/cn/
Disallow /en/en/
Disallow /en/fi/
Disallow /en/fr/
Disallow /en/ge/
Disallow /en/in/
Disallow /en/jp/
Disallow /en/kr/
Disallow /en/sp/
Disallow /en/si/
Disallow /fi/af/
Disallow /fi/cn/
Disallow /fi/en/
Disallow /fi/fi/
Disallow /fi/fr/
Disallow /fi/ge/
Disallow /fi/in/
Disallow /fi/jp/
Disallow /fi/kr/
Disallow /fi/sp/
Disallow /fi/si/
Disallow /fr/af/
Disallow /fr/cn/
Disallow /fr/en/
Disallow /fr/fi/
Disallow /fr/fr/
Disallow /fr/ge/
Disallow /fr/in/
Disallow /fr/jp/
Disallow /fr/kr/
Disallow /fr/sp/
Disallow /fr/si/
Disallow /ge/af/
Disallow /ge/cn/
Disallow /ge/en/
Disallow /ge/fi/
Disallow /ge/fr/
Disallow /ge/ge/
Disallow /ge/in/
Disallow /ge/jp/
Disallow /ge/kr/
Disallow /ge/sp/
Disallow /ge/si/
Disallow /in/af/
Disallow /in/cn/
Disallow /in/en/
Disallow /in/fi/
Disallow /in/fr/
Disallow /in/ge/
Disallow /in/in/
Disallow /in/jp/
Disallow /in/kr/
Disallow /in/sp/
Disallow /in/si/
Disallow /jp/af/
Disallow /jp/cn/
Disallow /jp/en/
Disallow /jp/fi/
Disallow /jp/fr/
Disallow /jp/ge/
Disallow /jp/in/
Disallow /jp/jp/
Disallow /jp/kr/
Disallow /jp/sp/
Disallow /jp/si/
Disallow /kr/af/
Disallow /kr/cn/
Disallow /kr/en/
Disallow /kr/fi/
Disallow /kr/fr/
Disallow /kr/ge/
Disallow /kr/in/
Disallow /kr/jp/
Disallow /kr/kr/
Disallow /kr/sp/
Disallow /kr/si/
Disallow /sp/af/
Disallow /sp/cn/
Disallow /sp/en/
Disallow /sp/fi/
Disallow /sp/fr/
Disallow /sp/ge/
Disallow /sp/in/
Disallow /sp/jp/
Disallow /sp/kr/
Disallow /sp/sp/
Disallow /sp/si/
Disallow /si/af/
Disallow /si/cn/
Disallow /si/en/
Disallow /si/fi/
Disallow /si/fr/
Disallow /si/ge/
Disallow /si/in/
Disallow /si/jp/
Disallow /si/kr/
Disallow /si/sp/
Disallow /si/si/

Other Records

Field Value
crawl-delay 1

linkchecker/7.4

Rule Path
Disallow /

linkchecker

Rule Path
Disallow /

wget

Rule Path
Disallow /

wget

Rule Path
Disallow /

webzip

Rule Path
Disallow /

webzip/3.6

Rule Path
Disallow /

plucker/py-1.0

Rule Path
Disallow /

webcapture

Rule Path
Disallow /

webcopy

Rule Path
Disallow /

webcopy/0.98b7

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webcopier v2.8

Rule Path
Disallow /

webcopier v3.0

Rule Path
Disallow /

mass downloader

Rule Path
Disallow /

mass downloader/2.2

Rule Path
Disallow /

web downloader

Rule Path
Disallow /

web downloader/2.9

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

offline explorer/2.5

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webstripper/2.14

Rule Path
Disallow /

pagenest free edition/3.30

Rule Path
Disallow /

sqlmap/1.0-dev

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.animecharactersdatabase.com/sm_index.php
sitemap https://thenaturedb.com/sm_tndb.php