Log In Sign Up

business.equalsmoney.com
robots.txt

Robots Exclusion Standard data for business.equalsmoney.com

Resource Scan

Scan Details

Site Domain business.equalsmoney.com
Base Domain equalsmoney.com
Scan Status Ok
Last Scan2024-11-03T07:39:01+00:00
Next Scan 2024-12-03T07:39:01+00:00

Last Scan

Scanned2024-11-03T07:39:01+00:00
URL https://business.equalsmoney.com/robots.txt
Domain IPs 108.156.133.10, 108.156.133.101, 108.156.133.104, 108.156.133.8
Response IP 108.156.133.10
Found Yes
Hash c77023febc61614010e32303c3d0a285c672b4390e9af7352e1ae93e1c078d60
SimHash 99364c17ce74

Groups

*

Rule Path
Disallow /password/
Disallow /c

Other Records

Field Value
crawl-delay 10

webreaper

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /
Back to top

Comments

  • Robots.txt for https://business.fairfx.com/
  • 28/11/2018
  • Site areas where indexing makes no sense
  • States that crawlers must give a 10 second delay between requests to the same server
  • ----------------------------------------------
  • This section covers bots that we don't like, for whatever reason.
  • If you have an objection to the inclusion of your bot, email _enquiries@fairfx.com_
Back to top