Log In Sign Up

blog.feedspot.com
robots.txt

Robots Exclusion Standard data for blog.feedspot.com

Resource Scan

Scan Details

Site Domain blog.feedspot.com
Base Domain feedspot.com
Scan Status Ok
Last Scan2025-08-04T10:21:10+00:00
Next Scan 2025-09-03T10:21:10+00:00

Last Scan

Scanned2025-08-04T10:21:10+00:00
URL https://blog.feedspot.com/robots.txt
Domain IPs 13.35.238.107, 13.35.238.12, 13.35.238.127, 13.35.238.99, 2600:9000:2085:1800:14:8cf0:e1c0:93a1, 2600:9000:2085:3000:14:8cf0:e1c0:93a1, 2600:9000:2085:6000:14:8cf0:e1c0:93a1, 2600:9000:2085:7600:14:8cf0:e1c0:93a1, 2600:9000:2085:e600:14:8cf0:e1c0:93a1, 2600:9000:2085:e800:14:8cf0:e1c0:93a1, 2600:9000:2085:fa00:14:8cf0:e1c0:93a1, 2600:9000:2085:fe00:14:8cf0:e1c0:93a1
Response IP 13.35.238.107
Found Yes
Hash 0978d989bb02e7d32e7669d0a0be5ab260880b3f38cebad55988619d95eb9c80
SimHash 07574608f2d2

Groups

*

Rule Path
Disallow /suggest/?*

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bytespider

Rule Path
Disallow /

mediapartners-google

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

mj12bot

Rule Path
Disallow /

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /folder/
Disallow /u/
Disallow /rsscombiner/
Disallow /brandmonitoring/
Disallow /widgets/
Disallow /cgi-bin/
Disallow /wp-admin
Disallow /wp-config*
Disallow /wp-content/themes/*.php
Disallow /wp-login.php
Disallow /*.asp*
Disallow /*.cfm
Disallow /*.cgi
Disallow /*.%7Cjsp*
Disallow /*.pl
Disallow /*.py
Back to top

Comments

  • This space intentionally left blank
  • If you want to learn about why our robots.txt looks like this, read this post: https://yoa.st/robots-txt
Back to top