Okay, buddy, since it's now New Year's, you asked for an academic lecture.
<Reason 1> The slang is offensive.
Shortening "Gaikoku-jin" (Foreigner) to "Gaijin" may sound hip and cool and not offensive, but the meaning of the entire word changes when you do so. Gaikoku-jin literally means "person from a foreign country," which is accurate when you use it to refer to a foreigner. "Gaijin," however, literally means "outsider," which has a lot of nuances attached to it.
By "outsider," the person using it is implying that foreigners in Japan can never assimilate into Japanese society and will always be outsiders no matter how hard they try to fit in. Many foreigners in Japan can speak fluent Japanese and/or have learned proper Japanese customers/etiquette. Despite this, by referring to such people as "gaijin," it's the same as saying that your langue skills or actions do not matter, you will always be an outsider due to your race, appearance, citizenship, etc.
This is racist, xenophobic, and just plain rude and offensive. So much so that I have had Japanese people use it in front of me and then apologize for it by saying that they weren't using it to refer to me, because I am "more Japanese" than most Japanese people are in their minds. Because they know me personally, they can't use it without some feeling of guilt.
You may not get this because maybe you never thought about what the word means or have never considered the situation of foreigners besides yourself and your circle of mates, but many people in the past and present are offended by it. It's a forbidden word on TV and Radio for goodness sake.
<Reason 2> Japanese people often use it as a slur
If you can understand Japanese and the connotation of when people often use it, you'll understand that they are not always using it as a cute, loving pet term. In many cases, they are using it to express their displease/distaste/reproach, etc. towards foreigners. There are two common situations when people use the terms:
1. As a common phrase with no ill malice: This is fine on the surface, but it shows ignorance and disrespect for the implications of using it, which should not be easily forgiven. People should be educated about it and if the circle of people decides to use it with no offense taken, that's just dandy.
2. The person is looking down on foreigners and is expressing their negative views by using this word (related to Reason 1). This is not acceptable because it's the same kind of slur as the N-word and other similar racial or bigoted slurs. You may say that words like the J-word have a different history so it's comparing apples to oranges, but the context or purpose for why they are using it are basically the same -- to put a different group of people down and show disrespect/hated, so please don't you or anyone else bring up that BS argument again.
Again, maybe your Japanese skills prevent you from understanding why many people are actually using this term, but having heard it used up close and personal for over 20 years, I think I know what I am talking about.
In the end, you may say that I'm harping too much on a pointless silly thing, which is fine. I'm just saying that it'd be nice if people stopped using the term so willy-nilly and thought about how it is offensive to many people.