A few months ago, two of my colleagues and I submitted a NSF proposal to create principled estimates of tornado probability. It was rejected. The reviewers wrote that the idea had intellectual merit, was high-risk and high payoff (the kinds of things the NSF normally should fund) and had substantial "broader" impacts (NSF-speak for societal benefits of research as opposed to simply academic projects). They loved the fact that we'd done a pilot study and shown that the idea was feasible. (The NSF no longer funds off-the-wall ideas: it has to be incremental but not too much so). They also appreciated our reputation and previous accomplishments in the field despite being a young team overall. The one problem? We had too few graduate students budgeted -- the NSF likes to fund graduate students.
The upshot was that two reviewers rated the proposal excellent and one rated it very good. In spite of the positive individual reviews, the one caveat about graduate students was enough to make the final review panel rate the proposal "Fund if possible" and in today's funding climate, that is as good as a "reject". Science budgets are so depleted these days that a "if possible" category doesn't exist.
One of my co-investigators on the proposal -- she is a faculty member and absolutely needs NSF-type funding -- has not been beaten down yet. She wants to resubmit. I'll go along, but I've come to see the proposal-writing process as a futile exercise. The really good ideas that we've proven over the years have been done with very little external funding, or with after-the-fact funding where we write a proposal, then start doing the work. By the time the proposal gets approved and we get the money, we've already done it, so we end up using the money for the next neat idea. This works for agencies with less appetite for risk who tend to fund incremental projects, but when we have a truly innovative idea that requires several years of work, there is no way to find the money to do it.