What do you mean that it looks odd?
Well, it just looks weird. It depends on the man, but most people can't pull this off. Both genders dress differently, women clothing is designed with a female body in mind, both to compliment it and to accentuate this type of body. Those features don't often translate to a male body, not to mention that female clothing is more expressive than male clothing, which tends to be bland and with far less variants.
it may be cultural, the way have evolved to see each other, or simple bias. But that's the way I feel, and unless you live in a very progressive area, I can guarantee you that most people will think similarly. This is something out of the ordinary, so they'll think that you look odd and then move on, some may ask why you dress that way.
Most people will think positively off you regardless of how you dress or look like if you are close enough to them anyway. This would only be an issue in an extremely conservative place.
I'm not saying that there's anything wrong with this, I just know how people think about you if you dress or look different.
