The efficacy of prayer is not something which can be established empirically, because there is always another explanation for anything - in extremis anything, no matter how apparently miraculous, can be explained-away by mental illness, mass delusion or lying.
But the validity of prayer does need to be confirmed inwardly and personally by each person - and this is typically by a prayer being grated in such a way that we are convinced (even if, as is usual, we would not be able to convince others - that does not matter, because the prayer was of the nature of a private communication between the prayer and God).
However, not all prayers are granted. Why is this?
In a bigger picture, prayer is one way that we are helped in this life - by the interventions of divine powers. And asking for, and getting, such divine help is one of the important rocks of faith. But why do we not receive more help, since life is so difficult?
Such a question can only be answered if we understand the basic purposes of mortal living, and in particular, why life is some kind of struggle for most people most of the time. If we are able to appreciate that life is a kind of educational process, then we may realise why help is provided, and not provided, in the way it is.
When someone is learning there is a time for help - that is what teachers are supposed to do - and there is a time for solitary struggle - for practise, for grappling with problems, for try, try, try again.
So, one important reason that prayers are not answered will certainly be that we are being required to do something from our own resources, because that is the only way we can learn. In effect, in the unanswered prayer, we were asking the divine powers to 'do our homework' - when this homework was vital to our learning.
Now, this is not the only reason, nor is it always the reason, prayers are unanswered - but it is surely one of the reasons, and perhaps a neglected reason: prayers are not answered, help is not given, when it is important that we do something without help, for our own good: when we have asked for help instead of participating in the learning process that is life.
This does not mean we should not have asked for help in the first place, because we can't always know the nature of the situation (if in doubt: ask); but it does mean that when we have asked for help and received a negative answer, we then acknowledge the validity of that reply, and act accordingly.