mylessimeonmusic.com.au
robots.txt

Robots Exclusion Standard data for mylessimeonmusic.com.au

Resource Scan

Scan Details

Site Domain mylessimeonmusic.com.au
Base Domain mylessimeonmusic.com.au
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-09-22T05:56:07+00:00
Next Scan 2024-12-21T05:56:07+00:00

Last Successful Scan

Scanned2023-05-19T08:58:34+00:00
URL https://mylessimeonmusic.com.au/robots.txt
Domain IPs 35.241.62.186
Response IP 35.241.62.186
Found Yes
Hash 69386b641a9406f6f451227f53bf9498f7fea05e012ab7d93f330d56d816bf9c
SimHash 0238fa41df73

Groups

*

Rule Path
Disallow /tools
Disallow /checkout
Disallow /download_check
Disallow /cart/
Disallow /corpbanner/
Disallow /stream
Disallow /api/
Allow /api/currency_data/
Disallow /*_cb$

nextgensearchbot

Rule Path
Disallow /

edisterbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

swebot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

Comments

  • the currency data endpoint is required to render pages
  • pattern matching known to work only with Google and Yahoo
  • badly-behaving bots
  • unwanted bots