Log In Sign Up

x-files.wikia.com
robots.txt

Robots Exclusion Standard data for x-files.wikia.com

Resource Scan

Scan Details

Site Domain x-files.wikia.com
Base Domain wikia.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-09-19T18:17:04+00:00
Next Scan 2025-10-03T18:17:04+00:00

Last Successful Scan

Scanned2025-07-17T11:53:46+00:00
URL https://x-files.wikia.com/robots.txt
Redirect https://x-files.fandom.com/robots.txt
Redirect Domain x-files.fandom.com
Redirect Base fandom.com
Domain IPs 151.101.0.194, 151.101.128.194, 151.101.192.194, 151.101.64.194, 2a04:4e42:200::194, 2a04:4e42:400::194, 2a04:4e42:600::194, 2a04:4e42::194
Redirect IPs 199.232.208.194, 199.232.212.194
Response IP 199.232.212.194
Found Yes
Hash b455b3657648040413c65ad8e3216142806ad32c81849dffc94b1f51da4765c6
SimHash 7c4619d0f1c5

Groups

semrushbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

*

Rule Path
Allow /api.php?
Allow /api.php?action=
Allow /api.php?*&action=
Allow /wiki/Special%3ACreateNewWiki
Allow /wiki/Special%3AAllMaps
Allow /bg/api.php?
Allow /bg/api.php?action=
Allow /bg/api.php?*&action=
Allow /bg/wiki/%D0%A1%D0%BF%D0%B5%D1%86%D0%B8%D0%B0%D0%BB%D0%BD%D0%B8%3ACreateNewWiki
Allow /bg/wiki/Special%3ACreateNewWiki
Allow /bg/wiki/%D0%A1%D0%BF%D0%B5%D1%86%D0%B8%D0%B0%D0%BB%D0%BD%D0%B8%3AAllMaps
Allow /bg/wiki/Special%3AAllMaps
Allow /es/api.php?
Allow /es/api.php?action=
Allow /es/api.php?*&action=
Allow /es/wiki/Especial%3ACreateNewWiki
Allow /es/wiki/Special%3ACreateNewWiki
Allow /es/wiki/Especial%3AAllMaps
Allow /es/wiki/Special%3AAllMaps
Allow /fr/api.php?
Allow /fr/api.php?action=
Allow /fr/api.php?*&action=
Allow /fr/wiki/Sp%C3%A9cial%3ACreateNewWiki
Allow /fr/wiki/Special%3ACreateNewWiki
Allow /fr/wiki/Sp%C3%A9cial%3AAllMaps
Allow /fr/wiki/Special%3AAllMaps
Allow /ru/api.php?
Allow /ru/api.php?action=
Allow /ru/api.php?*&action=
Allow /ru/wiki/%D0%A1%D0%BB%D1%83%D0%B6%D0%B5%D0%B1%D0%BD%D0%B0%D1%8F%3ACreateNewWiki
Allow /ru/wiki/Special%3ACreateNewWiki
Allow /ru/wiki/%D0%A1%D0%BB%D1%83%D0%B6%D0%B5%D0%B1%D0%BD%D0%B0%D1%8F%3AAllMaps
Allow /ru/wiki/Special%3AAllMaps
Disallow /wiki/Special%3A
Disallow /wiki/User_talk%3A
Disallow /wiki/Template%3A
Disallow /wiki/Template_talk%3A
Disallow /wiki/Help%3A
Disallow /wiki/User%3A
Disallow /wiki/UserProfile%3A
Disallow /bg/wiki/%D0%A1%D0%BF%D0%B5%D1%86%D0%B8%D0%B0%D0%BB%D0%BD%D0%B8%3A
Disallow /bg/wiki/Special%3A
Disallow /bg/wiki/%D0%9F%D0%BE%D1%82%D1%80%D0%B5%D0%B1%D0%B8%D1%82%D0%B5%D0%BB_%D0%B1%D0%B5%D1%81%D0%B5%D0%B4%D0%B0%3A
Disallow /bg/wiki/User_talk%3A
Disallow /bg/wiki/%D0%A8%D0%B0%D0%B1%D0%BB%D0%BE%D0%BD%3A
Disallow /bg/wiki/Template%3A
Disallow /bg/wiki/%D0%A8%D0%B0%D0%B1%D0%BB%D0%BE%D0%BD_%D0%B1%D0%B5%D1%81%D0%B5%D0%B4%D0%B0%3A
Disallow /bg/wiki/Template_talk%3A
Disallow /bg/wiki/%D0%9F%D0%BE%D0%BC%D0%BE%D1%89%3A
Disallow /bg/wiki/Help%3A
Disallow /bg/wiki/%D0%9F%D0%BE%D1%82%D1%80%D0%B5%D0%B1%D0%B8%D1%82%D0%B5%D0%BB%3A
Disallow /bg/wiki/User%3A
Disallow /bg/wiki/UserProfile%3A
Disallow /es/wiki/Especial%3A
Disallow /es/wiki/Special%3A
Disallow /es/wiki/Usuario_discusi%C3%B3n%3A
Disallow /es/wiki/User_talk%3A
Disallow /es/wiki/Plantilla%3A
Disallow /es/wiki/Template%3A
Disallow /es/wiki/Plantilla_discusi%C3%B3n%3A
Disallow /es/wiki/Template_talk%3A
Disallow /es/wiki/Ayuda%3A
Disallow /es/wiki/Help%3A
Disallow /es/wiki/Usuario%3A
Disallow /es/wiki/User%3A
Disallow /es/wiki/UserProfile%3A
Disallow /fr/wiki/Sp%C3%A9cial%3A
Disallow /fr/wiki/Special%3A
Disallow /fr/wiki/Discussion_utilisateur%3A
Disallow /fr/wiki/User_talk%3A
Disallow /fr/wiki/Mod%C3%A8le%3A
Disallow /fr/wiki/Template%3A
Disallow /fr/wiki/Discussion_mod%C3%A8le%3A
Disallow /fr/wiki/Template_talk%3A
Disallow /fr/wiki/Aide%3A
Disallow /fr/wiki/Help%3A
Disallow /fr/wiki/Utilisateur%3A
Disallow /fr/wiki/User%3A
Disallow /fr/wiki/UserProfile%3A
Disallow /ru/wiki/%D0%A1%D0%BB%D1%83%D0%B6%D0%B5%D0%B1%D0%BD%D0%B0%D1%8F%3A
Disallow /ru/wiki/Special%3A
Disallow /ru/wiki/%D0%9E%D0%B1%D1%81%D1%83%D0%B6%D0%B4%D0%B5%D0%BD%D0%B8%D0%B5_%D1%83%D1%87%D0%B0%D1%81%D1%82%D0%BD%D0%B8%D0%BA%D0%B0%3A
Disallow /ru/wiki/User_talk%3A
Disallow /ru/wiki/%D0%A8%D0%B0%D0%B1%D0%BB%D0%BE%D0%BD%3A
Disallow /ru/wiki/Template%3A
Disallow /ru/wiki/%D0%9E%D0%B1%D1%81%D1%83%D0%B6%D0%B4%D0%B5%D0%BD%D0%B8%D0%B5_%D1%88%D0%B0%D0%B1%D0%BB%D0%BE%D0%BD%D0%B0%3A
Disallow /ru/wiki/Template_talk%3A
Disallow /ru/wiki/%D0%A1%D0%BF%D1%80%D0%B0%D0%B2%D0%BA%D0%B0%3A
Disallow /ru/wiki/Help%3A
Disallow /ru/wiki/%D0%A3%D1%87%D0%B0%D1%81%D1%82%D0%BD%D0%B8%D0%BA%3A
Disallow /ru/wiki/User%3A
Disallow /ru/wiki/UserProfile%3A

ias_crawler

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1
Back to top

Other Records

Field Value
sitemap https://x-files.fandom.com/sitemap-newsitemapxml-index.xml
sitemap https://x-files.fandom.com/bg/sitemap-newsitemapxml-index.xml
sitemap https://x-files.fandom.com/es/sitemap-newsitemapxml-index.xml
sitemap https://x-files.fandom.com/fr/sitemap-newsitemapxml-index.xml
sitemap https://x-files.fandom.com/ru/sitemap-newsitemapxml-index.xml
Back to top

Warnings

  • `noindex` is not a known field.
Back to top