I've been in Japan too long, I'm probably jaded (just a little) when it comes to people who try to pull this shit with me.
When I first got here, same exact stuff that Patty and Ches describe... even got chastised for using my earbuds when clearly other people were allowed to or were ignored. (When I'm writing reports, music makes things go faster...)
Those days are long gone for me, but I developed my own brand of defense which usually meant pulling the foreigner card and not bowing to the culture. Probably not the best way, but the jackasses that thought they could harass me, were sorely mistaken. My first 2 years in Japan, were rough to say the least. Things are much better now... got promoted a couple of times, pretty much work autonomously now.
@patty diaz -- He probably thinks he can push you around because you're A) a woman (no offense) and B) shy = easy prey.
@Ches -- I'm surprised at you. I thought you were always a fireball who didn't take anyone's crap. That dude you're talking about, just sounds like he's jealous of you or maybe was scorned by some girl and he's taking it out on you, 'cause you're an easy target in this particular case.
Japanese bullies completely get deflated as soon as someone stands up to them. In particular, a foreigner...
I had my fair share of bullying days, learned real fast how to shut them down.
That's just my 2 yen, shooting from the hip... apologies for any offensive remarks to the ladies.