It's the biggest perceived "issue" with what people generally call art games -- they're counter-intuitive or inscrutable, players get frustrated, and then they don't buy that artist's line that the emotions they're feeling are part of the intended experience.
The sensation that a designer has intentionally withheld his or her intention from a player's reach often makes them feel tricked, excluded and frustrated. I'm the sort of player who likes to analyze what the designer is trying to get me to think and feel -- and even I feel annoyed by games that punish me.
As it turns out, the problem with some of those games isn't that they made me feel bad. It's that I didn't understand why they did. I learned this by talking to Douglas Wilson from the Copenhagen Game Collective about the group's surprisingly fascinating philosophy of "abusive" game design.
The designers in the collective work in the discomfort zone because it's a way of starting a conversation between the player and the designer. Ultimately, their work seems to see games partially as frameworks for interaction between people, not as the interaction themselves. It's really thought-provoking: Read the interview!