blogdego.com
robots.txt

Robots Exclusion Standard data for blogdego.com

Resource Scan

Scan Details

Site Domain blogdego.com
Base Domain blogdego.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-10-01T10:39:45+00:00
Next Scan 2024-12-30T10:39:45+00:00

Last Successful Scan

Scanned2023-06-05T05:29:09+00:00
URL https://blogdego.com/robots.txt
Domain IPs 183.181.81.26
Response IP 183.181.81.26
Found Yes
Hash 2506b27c6ff0773320d2ee4471a8201447f7ec2f02e68dafc42bde1e03a9721d
SimHash 92cd1440bb98

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-login.php
Allow /wp-admin/admin-ajax.php
Disallow /feed
Disallow /feeds
Disallow /rss

megalodon

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

daumoa

Rule Path
Disallow /

newspaper/0.2.8

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ias_crawler

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.com

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

accompanybot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

pinterestbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

archive.org_bot
ia_archiver

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

researchscan

Rule Path
Disallow /

netcraftsurveyagent

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

builtwith

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

steeler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

proximic

Rule Path
Disallow /

pinterestbot

Rule Path
Disallow /

*

Rule Path
Disallow /wp-content/uploads/wpo-plugins-tables-list.json

Other Records

Field Value
sitemap https://blogdego.com/sitemap.xml

Comments

  • archivebot-block
  • Common Crawl
  • NICT
  • COMSYS
  • Netcraft
  • Grapeshot
  • BuiltWith
  • AhrefsBot
  • SEMrush
  • Steeler
  • Dotbot
  • Majestic
  • Serpstat
  • SEOkicks
  • Barkrowler
  • BLEXBot
  • MegaIndex
  • proximic
  • Pinterestbot