I am getting braver and braver shooting people. They had no idea I did it :p

I think what helps me is being fast with it, not looking at them too often or pointing the camera at them for a long time. Notice the upcoming composition, and then quickly take it. And in the rare event that someone just asks about it, you can always just tell them and offer to show them the picture and explain it.

And with time as you grow more comfortable, the less you even think of this stuff, you're so in that zone, the flow of noticing and capturing, that you have no problems photographing people and thinking of whether they notice or not. Most of the time people don't care, unless you're getting up in their faces or really invading their space with the camera.

Now that I think about it, nobody ever seems bothered by it. Maybe I notice the composition I want to take and look more at the surrounds enough that it seems like my interest was never in the people to begin with.