From reading, especially, Owen Barfield, and the selections-from and commentary of Goethe published by Jeremy Naydler, I have considerably enriched my understanding of the essence of science from the state it had reached when I published Not Even Trying: the corruption of real science.
THE big problem for those trying to understand science has been the question of: Where do correct hypotheses come-from - given that there are an unbounded number of incorrect hypotheses. This does not entail assuming that correct hypotheses are either certain or complete (any hypotheses is almost-inevitably a shortened and selective model of reality) - merely that there are so many more ways of being wrong than right; and science could never get going if vast numbers of wrong hypotheses had to be eliminated with every step.
Upon this matter hinges almost the whole of science; because once good hypotheses are available, science becomes more-or-less a matter of applied routine (research and development, as it is called). But if god hypotheses are lacking, hen we either have nothing at all; or, as nowadays, we have fake science, pseudo-science, science that is not-even-trying to be truthful: a species of bureaucratic careerism which apes the superficial appearances of science but actually does nothing towards enhancing human understanding; except to expend resources, generate publications, and mislead in a thousand ways.
(By the standards of modern research evaluations, the more resources expended - e.g. grant income; and the more words generated - e.g. publications; and the more people who are misled by such lying profligacy - e.g the 'impact; the better will be the research evaluations, the higher the status and power. In these evaluations, 'truth' is never a variable.)
The origin of true hypotheses is not a question which can be answered by science itself - it is a meta-scientific question: that is, a philosophical question.
The uncannily prescience insight of Goethe was that true hypotheses come from the imagination of an individual scientist - constrained by a direct apprehension of phenomena.
Thus, a real scientist gives himself over to a consideration of the phenomenon in question; and after achieving a correct orientation, by means of this honest devotion, he may be rewarded by a direct understanding of reality.
This primary understanding is 'present' in the scientists own imagination - and is initially utterly private. The point at which it becomes public is when it is formulated as a general statement such as a theory or an hypothesis concerning the nature of reality.
But the public statement of the scientist's inner imagination understanding is inevitably a partial and biased summary of what is in his own imagination - therefore he may need (typically will need) to modify, and re-modify, that public statement - in light of his experience of how it is being understood, and in light of linking it to other phenomena (i.e. the 'evidence') in order to make it more validly representative of what is in his imagination.
(Furthermore, any specific hypothesis lies in relationship to many other hypotheses, which me be of various validities - and further modifications are likely to be necessary as these adjacent, and gradually more distant, hypothesis are clarified - thus even real science is usually in a dynamic state - but one that is utterly different from the fashionable whims of modern mainstream pseudo-science.)
Once this point of a public statement has been reached, then normal, routine, R&D type science can commence.
But this begs the question of how it is that the science may be able directly to 'intuit' directly the nature of reality? How is it that the scientist can apprehend phenomena directly?
The answer to this can only be metaphysical - and is usually religious. It presupposes that direct communication is possible to at-least a significant extent - i.e. communication which is not-merely a consequence of the set-up of the scientist's own sensory apparatus, brain apparatus, and in general 'subjectivity'. A communication that comes from the phenomena as it really is, and not 'merely' from the way that phenomena are framed by the dominant explanatory mode of a particular time and place.
In sum, the above explanation relies upon the possibility of objective knowledge - not merely in the weak sense of objectivity as 'publicly-agreed knowledge'; but in a strong sense of objectivity as 'corresponding with reality'.
By this account, we get a very different understanding of the nature of science, and the way that the 'success' of science contributes to our understanding of reality. This is not to be understood in terms of the 'power' of science (e.g. to generate transformative technologies, such as Western medicine and engineering). The real evidence of science is in terms of confirming the possibility of direct knowledge of reality via the imagination.
Remarkably, science - real science - emerges as being primarily an imaginative apprehension of reality - and evidence that such imagination is potentially truthful: objective.
It brings us back to the individual scientist and his devotion to his subject as the basis of that vast (and, now, mainly corrupt) superstructure which is the public, measurable, appearance of science.
It aligns science with poetry, and the other creative arts; and it breaks down the barrier between our conceptualizations of these aspects of creativity.
It also provides indirect evidence of order, design and purpose in the universe - in reality itself. And, even more significant, a role for Man - that Man is enabled to know this reality, is in a sense is made to know reality.
The scientist - that is, the primary and real scientist - is a man of intuition and imagination whose love of that which he contemplates is (or may be - this is not an algorithm, not a manageable 'protocol'!) rewarded by a direct and true apprehension concerning its reality.
NOTE: It should be emphasized that my understanding of these matters came mainly via Barfield and Naydler - but that the origin of these ideas is with Goethe and especially Rudolf Steiner. I have read probably some hundreds of thousands of words of Steiner, and can confidently confirm that he is indeed the true originator and developer of this perspective, building upon a relatively brief and preliminary account in Goethe - but I have seldom been able to discover this from Steiner directly, but rather by intermediary scholarship.