Log In Sign Up

pregunta-a-christel.es
robots.txt

Robots Exclusion Standard data for pregunta-a-christel.es

Resource Scan

Scan Details

Site Domain pregunta-a-christel.es
Base Domain pregunta-a-christel.es
Scan Status Ok
Last Scan2025-10-12T18:19:54+00:00
Next Scan 2025-10-19T18:19:54+00:00

Last Scan

Scanned2025-10-12T18:19:54+00:00
URL http://pregunta-a-christel.es/robots.txt
Domain IPs 2a01:138:9002:2::150, 80.190.156.150
Response IP 80.190.156.150
Found Yes
Hash ce0f8c296dff84461d72653814721d411d5d253f2dfb96f585624097dd8d80ee
SimHash 325cd754c8d8

Groups

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

dotbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

semrushbot-coub

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

mbcrawler

Rule Path
Disallow /

censysinspect

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

zoomrank/2.0

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /
Back to top

Comments

  • slow down Bingbot
  • Block Dotbot
  • https://moz.com/help/moz-procedures/crawlers/dotbot
  • Block Rogerbot
  • https://moz.com/help/moz-procedures/crawlers/rogerbot
  • Block Semrush
  • https://www.semrush.com/bot/
  • Block Seekport
  • https://bot.seekport.com/
  • Block Ahrefs
  • https://ahrefs.com/robot
  • Block SEOkicks
  • https://www.seokicks.de/robot.html
  • Block MegaIndex.ru
  • http://megaindex.ru/2.0
  • Block MJ12bot
  • https://mj12bot.com/
  • Block BLEXBot
  • http://webmeup-crawler.com/
  • Block CCBot
  • https://commoncrawl.org/big-picture/frequently-asked-questions/
  • Block VelenPublicWebCrawler
  • https://velen.io/
  • Block MBCrawler
  • https://monitorbacklinks.com
  • Block CensysInspect
  • https://about.censys.io/
  • Block GrapeshotCrawler
  • https://www.oracle.com/corporate/acquisitions/grapeshot/crawler.html
  • Block DataForSeoBot
  • https://dataforseo.com/dataforseo-bot
  • Block GPTBot
  • https://platform.openai.com/docs/gptbot
  • Block Google Bard, Vertex, etc.
  • https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers?hl=de
Back to top