investingnote.com
robots.txt

Robots Exclusion Standard data for investingnote.com

Resource Scan

Scan Details

Site Domain investingnote.com
Base Domain investingnote.com
Scan Status Ok
Last Scan2024-06-27T02:18:06+00:00
Next Scan 2024-07-04T02:18:06+00:00

Last Scan

Scanned2024-06-27T02:18:06+00:00
URL https://investingnote.com/robots.txt
Redirect https://www.investingnote.com/robots.txt
Redirect Domain www.investingnote.com
Redirect Base investingnote.com
Domain IPs 139.59.192.132
Redirect IPs 139.59.192.132
Response IP 139.59.192.132
Found Yes
Hash 52d127cd91d623a4970e0b44f75c115894ec000c1a1d4da209bbbcf3434e87f2
SimHash 26dc0d3f3040

Groups

*
grapeshot

Rule Path
Disallow /admin
Disallow /checkout
Disallow /cart
Disallow /orders
Disallow /countries
Disallow /line_items
Disallow /password_resets
Disallow /states
Disallow /user_sessions
Disallow /user_registrations
Disallow /account
Disallow /api
Disallow /rapi
Disallow /packages
Disallow /liked_posts.json
Disallow /favorite_posts.json
Disallow /stocks/undefined/symbol_info

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-Agent: *
  • Disallow: /