spain.info
robots.txt

Robots Exclusion Standard data for spain.info

Resource Scan

Scan Details

Site Domain spain.info
Base Domain spain.info
Scan Status Ok
Last Scan2024-04-27T22:40:26+00:00
Next Scan 2024-05-27T22:40:26+00:00

Last Scan

Scanned2024-04-27T22:40:26+00:00
URL https://spain.info/robots.txt
Redirect https://www.spain.info/robots.txt
Redirect Domain www.spain.info
Redirect Base spain.info
Domain IPs 34.78.108.191
Redirect IPs 138.199.8.193, 143.244.35.226, 2a00:de40:10:545::a, 2a00:de40:10:545::c, 2a02:6ea0:c77e::4, 2a02:6ea0:d513::4, 51.81.243.73
Response IP 79.127.211.129
Found Yes
Hash 52ad82381500aab9eaa15cdf2458ca436604c5c84370e9d830e35d04c9671744
SimHash 0a6061704415

Groups

*

Rule Path
Allow /system/*.css$
Allow /system/*.js$
Allow /system/*jpg$
Allow /system/*gif$
Allow /system/*png$
Allow /system/*svg$
Disallow /admin/
Disallow /system/
Disallow /app/
Disallow /*/app/
Disallow /search-results/
Disallow /resultados-busqueda/
Disallow /resultats-recherche/
Disallow /suchergebnisse/
Disallow /risultati-ricerca/
Disallow /resultados-busca/
Disallow /rezultaty-poiska/
Disallow /wyniki-wyszukiwania/
Disallow /kensaku-kekka/

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 250

Other Records

Field Value
sitemap https://www.spain.info/sitemap_index.xml
sitemap https://www.spain.info/export/sitemap_pt_BR.xml
sitemap https://www.spain.info/export/sitemap_es.xml
sitemap https://www.spain.info/export/sitemap_de.xml
sitemap https://www.spain.info/export/sitemap_en.xml
sitemap https://www.spain.info/export/sitemap_fr.xml
sitemap https://www.spain.info/export/sitemap_it.xml
sitemap https://www.spain.info/export/sitemap_ja.xml
sitemap https://www.spain.info/export/sitemap_pl.xml
sitemap https://www.spain.info/export/sitemap_ru.xml
sitemap https://www.spain.info/export/sitemap_exp.xml
sitemap https://www.spain.info/export/sitemap_eventos.xml
sitemap https://www.spain.info/gcc/sitemap.xml

Comments

  • GENERAL SETTINGS
  • SITEMAPS
  • Sitemap Index
  • Sitemaps languages
  • Allow Directives
  • Private pages
  • Directories
  • Blocking bots

Warnings

  • 2 invalid lines.