I understand how it seems like it is rape the way Kanou treats Ayase, but I firmly believe that that is part of the author's/mangaka's style. As in, they rarely show Ayase experiencing pleasure, it usually is Kanou saying things like, "Look you're hard again so fast even though I just..." or "I haven't done anything horrible, didn't I do everything the way you like it?" We only get snippets of evidence that Ayase actually enjoys the encounters however I know that he actually does.
Whilst it is true that the first time Kanou had sex with Ayase it was rape, the same cannot be said for every single encounter. In fact after the first few times, Ayase, through unable to accept relations between two men, is incapable of not enjoying himself. And whilst he does feel ashamed by his arousal, nevertheless, he is aroused.
Par example, although the sex scenes we witness usually has Ayase writhing around, mewling "No..." and people criticise Ayase for being weak, the truth is he is enjoying himself. This can be seen in Volume 3 I believe, when a recording is sent to Ayase's apartment. It's the sounds of him and Kanou-san having sex, and the first thing we hear is Kanou's voice saying, "Come on, lift up your hips. I'll do it the way you like." Before Ayase's voice drowns his out with a loud, "Ahh! Kanou-san!! It's great. More- more. Ahh!"
If that doesn't convince you, then what about the things Kanou says about Ayase? Such as, "You always get really into it during the middle." and "Why is it that when a guy just teases you a little through your clothes, both top and bottom get hard so easily?" I believe that being Ayase's only bed partner, he would know best.
And those who think it's non-con or rape obviously don't have enough understanding for the unique relationship between the two. It is quite evident that Ayase treats Kanou differently. Frequently preaching about how kind he is and what a special person he is to him. VERY early on within the manga, the perverted politician made sexual advances towards Ayase whilst forcing Gion to film their interactions, but from that point already, Ayase established a clear distinction between the gross feeling he was experiencing at the hands of the politician to the warm, loving one he feels when Kanou touches him, asserting, "It's not the same!" And even slaps the man. Proving that one cannot easily attest to the veracity of the assertions made by those who advocate how Ayase is spineless and defenseless.
Not only that, Ayase reacts very easily under Kanou's touch. When Kanou found him dressed in a nurse's costume, he decided to 'check' to see if Ayase was touched by someone other than him. All Kanou did was grope his ass a little but Ayase was completely aroused, causing Kanou to say, "How could I let this lustful body work in a place like that?!" This is a very different reaction had by one who was about to be raped, and Ayase has had people try to rape him before, and his expression is vastly different, usually filled with immense fear. With Kanou, it's more embarrassment that he's so aroused.
If you're still not convinced, think about when Ayase is drunk and uninhibited. He tends to start telling the truth more, as seen VERY early on yet again (to those who think Ayase was raped over and over again until he was forced to like it) when Ayase became drunk and when Kanou was touching him, Ayase moaned out, "Ahh! There!" and confessed he liked it, making Kanou pause a beat with surprise because, "he usually makes such a show of not wanting it." But he actually does want it. Why else would he always be so eager to have sex whenever he was drunk? It's because he becomes less inhibited, and loses his embarrassment and shame. I doubt Ayase would just jump into bed with whoever he was with when drunk.
Someone who was being repeatedly rape who not be so eager to have sex with their rapist even under the influence of alcohol.
'Nuff said.