I used to feel that way. Still do sometimes. Then one day when I was feeling down and wanted music to match I listened to the album “Blood on the Tracks.”
Fit my mood. Then I started to get it: "Ah. He’s not a singer. He’s a storyteller, and a poet, and an artist. And his voice is important only so that you know these stories are coming from a real person (not an “entertainer”), but the voice is fairly unimportant as an instrument to be admired. Almost as if the talentless everyman quality of the voice is broadcasting an unspoken message: “This is not entertainment, this is my message.”