Log In Sign Up

fareastflora.com
robots.txt

Robots Exclusion Standard data for fareastflora.com

Resource Scan

Scan Details

Site Domain fareastflora.com
Base Domain fareastflora.com
Scan Status Ok
Last Scan2024-05-21T15:04:17+00:00
Next Scan 2024-06-20T15:04:17+00:00

Last Scan

Scanned2024-05-21T15:04:17+00:00
URL https://fareastflora.com/robots.txt
Domain IPs 104.26.12.127, 104.26.13.127, 172.67.68.54, 2606:4700:20::681a:c7f, 2606:4700:20::681a:d7f, 2606:4700:20::ac43:4436
Response IP 104.26.12.127
Found Yes
Hash 66c305a1ef527d57fc8dae700e5284e456379882a89035bf7be2bf95019a948d
SimHash 705599816eb5

Groups

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

googlebot
google

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

deepcrawl
ahrefsbot
mj12bot
dotbot
rogerbot
screaming frog seo spider
scoutjet
linkdex
moatbot
adbeat_bot
xenu

Rule Path
Disallow /catalogsearch/
Disallow /*_escaped_fragment_
Disallow /*morefilters%3D
Disallow /*price%3D

googlebot-image

Rule Path
Disallow

googlebot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 20
Back to top

Comments

  • User-agent: bingbot
  • User-agent: bing
  • User-agent: SemrushBot
  • Disallow: /
  • User-agent: Googlebot
Back to top