Well you do have to respect their privacy don't you. Couldn't bare the thought of being 'out of the office' as it were, and have people pestering you. I know I wouldn't like it. I think you should treat people with respect and in the same way as you would want to be treated - its called being polite.