I have a larger DSLR and am still working on being able to take it everywhere. I have it easily accessible around the house and if I am going to an event or even just to someone's house, I try to remember to put my camera backpack in the car. (You never know when the moment might pop up to use it).
My husband has a smaller point and shoot which is a lot easier to grab spur of the moment, and sometimes I find myself grabbing that for quick moments. It is smaller so it is a lot less obtrusive and can easily be put into a large pocket. I also don't have to worry about the lens getting knocked around and such.
My uncle lives in New York and just got a new small point and shoot type camera that he keeps in his coat pocket. He has gotten wonderful photos with it and it is very portable. It is small so he can keep it in his pocket while on the subway or walking around the city and it is not noticable or in the way.
I think if you really want to be able to take it everywhere and want to be comfortable with it, I would start with the smallest camera that will suit your needs. That way it will fit in the smaller spaces and you won't be self-conscious about it. Then as you gain confidence with carrying the camera around and if the need ever arises, you could move to a larger camera.
Like I said, most times I am still more comfortable carrying my husband's smaller camera than my larger one. I also find people are less aware and more natural when I have the smaller camera. When I pull out the bigger one, it usually screams "pose for me" or "run from the lens"!
