My theory is that the people who were watching the moon wax towards full, in order to know when Easter was to arrive (with Easter day coming on the first Sunday after the first full moon after the vernal equinox... approximately).
Easter spotters will have been observing the moon during the evenings of the waxing gibbous phase - when the Rabbit is the most obvious appearance; and the Man's face is not visible at all.
By contrast, the waning gibbous moon is best visible early in the mornings.Note: The above photos do not illustrate the Rabbit or Man at all obviously, compared with what I see with the naked eye - indeed I cannot find any decent photos online. It is apparently one of those things you just need to See For Yourself.
The appearances are most evident when the moon is nearest the horizon - the rabbit when nearest moon-rise and the man nearish moon-set. This is due to the (at my latitude) c1/3 clockwise rotation of the moon from rising to setting.
Thus the rabbit begins with ears horizontal, but by moon-set the rabbit is on his side, with ears pointing down - and not obviously a rabbit.
Similarly, for the Man's face to have eyes at the top is most evident getting towards moonset.