John, it's definitely a problem, although for me it's my own voice inside, not my wife's, worrying about the intrustion aspect. Asking permission in this particular shot (certainly not always!) would spoil it because then they'd pose themselves. I use a long lens so I don't have to be close, I try to be quick, and I look through the camera in many directions so I find all possible shots and so I don't appear to be "picking on" my subject. I also tend toward crowded locations so it's not just me and the subject.
Nice "people capture!" I like the assortment of things you have in your portfolio, but this one caught my eye for some reason. I love to shoot people shots like this when traveling, but my wife always thinks I'm intruding on the subjects to capture their photos, which is somewhat inhibiting, to say the least! But this shot is made more interesting by the inclusion of the dolls to add to the old faces of your subjects. Nice job!