I think this has nothing to do with the "wait almost three minutes to play a card" (which is hardly provable but if it should become provable somehow is a behaviour which should be strongly sanctioned, imho).
The thing seems to be merely with challenges. While it is annoying to have 10 challenges in a row not refused but "your opponent did not answer your challenge", I too found a huge discrepancy in the time it takes for that message to show up. Usual seems to be around 30 seconds (fine). Sometimes, however, around 1 in 20 times, it takes 3 minutes or so to show up - which is really annoying. Also, I sometimes had battles timeout that did not even start; the challenge was simply not accepted.
I suspect there is a bug or a script that cannot deal with too many players at the same time or something like that... a look at that might be a good idea
There are 2 cases:
a) they don't receive the challenge because they are no longer connected: 30 seconds
b) they receive the challenge but they don't answer it (because they might be away from keyboard): 1min30 (because we need to allow time to type a feedback if the player is playing)
I think that typing a feedback is not necessary... i mean im from brasil and man i receive some answers in hungaryan... lol... it is really anoying for u to not be answered there must be a rule that if the guy doesnt answers 3 challenges then he gets booted from the room(since hes AFK he shouldent make volume there...)