I managed to remove all horizontale lines/rules in my chart using this:

scales: {
       xAxes: [{
        gridLines: {
            display: false
        }  
    }]
}

But I also want to get rid of the rule/bar that represents the Y-axis as well. But I want to keep the labels:

Unfortunately I can't find any option for that. I can only remove the whole axis including labels.

I'm using Chart.js 2.3.

I found a way to remove this line. It's actually called the border of the axis and there's an option for it, see "Grid Line Configuration":

scales: {
    yAxes: [{
        gridLines: {
            drawBorder: false,
        }
    }]
}