Last I was in Japan for about a month, and I was able to withdraw over $10,000 throughout my stay (albeit $500 here, $1000 there each time), so I don't think there should be an issue. It happens to be the limit at which you've got to declare it (which may have been OP's point), but I guess that only applies to lump-sum cash being brought into the country, and not to ATM transactions which are clearly documented.
I leave my daily limit to $1000, and just withdraw as needed, as I don't like keeping so much cash on me. I use a bank that refunds all ATM fees worldwide, so the number of times doesn't matter to me.
As an aside: it used to be such a pain in the ass to find the one ATM 5 subway stops away that would accept your card, but these days 7-11 are ubiquitous and I use SevenBank ATMs exclusively... I used to be able to use Citi (which has moved out), and MUFJ (which is often hit-or-miss for foreign cards). Also keep in mind that many more places (hotels, restaurants, shinkansen, gift shops) are OK with card purchases than say 5 years ago, so you may not need all that cash on hand.