Did you talk to people who were married with kids to Japanese women (in Japan) before you did this? I'm sorry if this is not much consolation, but your situation sounds like every other 5yo marriage in Japan that I have ever heard about.
All of my friends who are married with kids to Japanese...