I can totally relate, although I must give you kudos for at least shooting from the hip. I just neglect to take the camera out at all.
I stopped recently to take pictures of a huge tree that had been hit by lightning and split in half near my son's daycare. I was totally self conscious about being out in the road as cars drove by and people walked down the street, assuming they are thinking to themselves, "who the heck does she think she is?!"
