That confession is soooooo cheesy. Ayumu has the best end, 3 MILFS! He is going to become a woman killer when he grows up, even Ui stripped for him just for him to draw LooL.
If we put aside the random encounter with kiss at the beginning and some hanging around in the first episodes, nothing that was shown gave away particular hints of a building relationship. It just had to be Ui, that's all of the motivations I've got.
So, a really poor execution. Truth to be told, it's not that Kazuma's interactions with the other girls were much better, except two. With Madoka there was a very poor mixture of shared feelings, emotions, memories and interests. Koyori has been the heroine for 1 episode/1 episode and a half, thus mining her believability as a contender. Tsumugi, who's Tsumugi in Kazuma's eyes other than a smart and mature senpai?
Things went better with Hina, towards whom our protagonist has at least shown some understanding, and white knight attitude. Although in my opinion the best choice for him would have been Ibuki, better chemistry, relevant interactions, mutual understanding, similar characters (both on the active/adventurous side, but also both capable of being thoughtful and well mannered). In facts Ibuki seemed the most struck by the rejection; sure there was also Madoka, but I saw her more as resigned. Ibuki probably also knew she stood little chances towards Ui, but yet she tried because she believed in it, and this is perfectly consistent with her character.
Speaking of consistencies, the last episode lacked some. The whole Megami being replaced at the last moment was pretty lame to say the least, so they missed their chanced to give a proper closure to the series.