thenaturedb.com
robots.txt
Robots Exclusion Standard data for thenaturedb.com
Resource Scan
Scan Details
| Site Domain | thenaturedb.com |
| Base Domain | thenaturedb.com |
| Scan Status | Ok |
| Last Scan | 2024-11-14T01:46:10+00:00 |
| Next Scan | 2024-11-21T01:46:10+00:00 |
Last Scan
| Scanned | 2024-11-14T01:46:10+00:00 |
| URL | https://thenaturedb.com/robots.txt |
| Domain IPs | 167.114.1.56, 2607:5300:60:6438::acdb |
| Response IP | 167.114.1.56 |
| Found | Yes |
| Hash | b00e302d28ebbb1c8df52c07502fd1e768958919970b7df0f5b40e4598ca5750 |
| SimHash | 01191a26c931 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /who.php |
| Disallow | /contact.php |
| Disallow | /includes |
| Disallow | /private |
| Disallow | /inc |
| Disallow | /af/af/ |
| Disallow | /af/cn/ |
| Disallow | /af/en/ |
| Disallow | /af/fi/ |
| Disallow | /af/fr/ |
| Disallow | /af/ge/ |
| Disallow | /af/in/ |
| Disallow | /af/jp/ |
| Disallow | /af/kr/ |
| Disallow | /af/sp/ |
| Disallow | /af/si/ |
| Disallow | /cn/af/ |
| Disallow | /cn/cn/ |
| Disallow | /cn/en/ |
| Disallow | /cn/fi/ |
| Disallow | /cn/fr/ |
| Disallow | /cn/ge/ |
| Disallow | /cn/in/ |
| Disallow | /cn/jp/ |
| Disallow | /cn/kr/ |
| Disallow | /cn/sp/ |
| Disallow | /cn/si/ |
| Disallow | /en/af/ |
| Disallow | /en/cn/ |
| Disallow | /en/en/ |
| Disallow | /en/fi/ |
| Disallow | /en/fr/ |
| Disallow | /en/ge/ |
| Disallow | /en/in/ |
| Disallow | /en/jp/ |
| Disallow | /en/kr/ |
| Disallow | /en/sp/ |
| Disallow | /en/si/ |
| Disallow | /fi/af/ |
| Disallow | /fi/cn/ |
| Disallow | /fi/en/ |
| Disallow | /fi/fi/ |
| Disallow | /fi/fr/ |
| Disallow | /fi/ge/ |
| Disallow | /fi/in/ |
| Disallow | /fi/jp/ |
| Disallow | /fi/kr/ |
| Disallow | /fi/sp/ |
| Disallow | /fi/si/ |
| Disallow | /fr/af/ |
| Disallow | /fr/cn/ |
| Disallow | /fr/en/ |
| Disallow | /fr/fi/ |
| Disallow | /fr/fr/ |
| Disallow | /fr/ge/ |
| Disallow | /fr/in/ |
| Disallow | /fr/jp/ |
| Disallow | /fr/kr/ |
| Disallow | /fr/sp/ |
| Disallow | /fr/si/ |
| Disallow | /ge/af/ |
| Disallow | /ge/cn/ |
| Disallow | /ge/en/ |
| Disallow | /ge/fi/ |
| Disallow | /ge/fr/ |
| Disallow | /ge/ge/ |
| Disallow | /ge/in/ |
| Disallow | /ge/jp/ |
| Disallow | /ge/kr/ |
| Disallow | /ge/sp/ |
| Disallow | /ge/si/ |
| Disallow | /in/af/ |
| Disallow | /in/cn/ |
| Disallow | /in/en/ |
| Disallow | /in/fi/ |
| Disallow | /in/fr/ |
| Disallow | /in/ge/ |
| Disallow | /in/in/ |
| Disallow | /in/jp/ |
| Disallow | /in/kr/ |
| Disallow | /in/sp/ |
| Disallow | /in/si/ |
| Disallow | /jp/af/ |
| Disallow | /jp/cn/ |
| Disallow | /jp/en/ |
| Disallow | /jp/fi/ |
| Disallow | /jp/fr/ |
| Disallow | /jp/ge/ |
| Disallow | /jp/in/ |
| Disallow | /jp/jp/ |
| Disallow | /jp/kr/ |
| Disallow | /jp/sp/ |
| Disallow | /jp/si/ |
| Disallow | /kr/af/ |
| Disallow | /kr/cn/ |
| Disallow | /kr/en/ |
| Disallow | /kr/fi/ |
| Disallow | /kr/fr/ |
| Disallow | /kr/ge/ |
| Disallow | /kr/in/ |
| Disallow | /kr/jp/ |
| Disallow | /kr/kr/ |
| Disallow | /kr/sp/ |
| Disallow | /kr/si/ |
| Disallow | /sp/af/ |
| Disallow | /sp/cn/ |
| Disallow | /sp/en/ |
| Disallow | /sp/fi/ |
| Disallow | /sp/fr/ |
| Disallow | /sp/ge/ |
| Disallow | /sp/in/ |
| Disallow | /sp/jp/ |
| Disallow | /sp/kr/ |
| Disallow | /sp/sp/ |
| Disallow | /sp/si/ |
| Disallow | /si/af/ |
| Disallow | /si/cn/ |
| Disallow | /si/en/ |
| Disallow | /si/fi/ |
| Disallow | /si/fr/ |
| Disallow | /si/ge/ |
| Disallow | /si/in/ |
| Disallow | /si/jp/ |
| Disallow | /si/kr/ |
| Disallow | /si/sp/ |
| Disallow | /si/si/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.animecharactersdatabase.com/sm_index.php |
| sitemap | https://thenaturedb.com/sm_tndb.php |