I can think of several issues here. In the order that you wrote in your post:
1. You're not sure if you want to try
2. You're still holding your v-card
3. You don't drink
4. You're only in Tokyo for a short duration
If you wanted to add an additional complication, I'd say lack of command of...