One of my favorites genres and it seem's this particular niche movie type really excelled in the 70's, was it all thanks to Nixon, why was the 1970's so damm good for this type of movie and the surrounding decades so bad.
The Conservation Three days of the condor The Parallax View Rollerball The Andromeda Strain Marathon Man Capricorn One
If you think about it, most every critically acclaimed movie in the 1970's was dark, gloomy, and pessimistic, unless it was directed by Mel Brooks, had Gene Wilder in it, or both.
You forgot not only all the Irwin Allen disaster movies but also
Soylent Green Logan's Run The Omega Man A Boy and His Dog Mad Max (the BEST and original) THX 1138
It was..... The end of the Vietnam war. Distrust in the government and Corporate entities was growing. The fear that nuclear war was a button push away. Drugs. The idea that we (I was in high school and college during the 70'S) could "tune in-turn on and drop out". Pollution. The feeling that the world was going to hell and we would be better off if we all just joined a commune, took off our clothes, smoked weed, and grew tomatoes...Everything would be OK.
There was a real doom and gloom feeling in Hollywood too.
What's funny is that as much as those movies predicted the future by reflecting those bad times.... little did we know that in our now... vegan, milk toast, victim filled, Starbucks latt'a, bubble wrapped, participation trophy future ...... most of those 70's movies couldn't be made because of the fear of offending someone.
It was social upheaval that put everything into question. Or rather, the aftereffects of it. Hunter Thompson puts it best in 'Fear & Loathing in Las Vegas': the promise of social revolution led to unbridled idealism and the actual power to make it happen. But this ignores the baser aspects of humanity, which results in a startling drop from imagining the perfect world to a startling, harsh, and very solid reality. A lot had changed for the good, but a lot of people felt disconnected from it and realized the true nature of life in America. In a real way people mistook this realization for signs that the world was collapsing around them.
The power of cinema and literature is to externalize this discomfort and come to terms with them. Hence we see a lot of science fiction/fantasy taking on issues and trying to sound them out, recognize them, and then find solutions to them. The near future aspects were to keep the semblance of realism so the solutions were easily placed back in reality. That's why in a few cases you see these movies having really loose arguments, such as Rollerball's entire plot: the movie wants to be about individuality but veers into the nature of the world and how media presence creates a reality outside of the real one which creates issues with many characters.
More so to due with the Watergate recordings. Well, if one is talking about paranoia. Pessimism and violence was due to the social climate, also contributed by the growing knowledge that government leaders were distrustful and that anyone who spoke out was "silenced."
As others have stated the Cold War, Vietnam, Watergate, "The Pentagon Papers", the civil rights movement and so on are some of the reasons. But by themselves, that wasn't enough. Personally, I think some of it or a lot of it was on the baby boomers coming of age after being brought up on a censored, whitewashed view of the the world. A simplistic post WW2 view of good vs evil.
As baby boomers progressed through life, they had to come to terms with real reality, i.e. selfishness, corruption in the name of self interest. After growing up with kiddie fare of the 50's and 60's, it must have been disappointing to see the world act in a less than an idealistic way. Hollywood tapped into that feeling, "don't trust anybody over 30", nefarious forces controlling world politics and so on. And there you have it, a market for dystopian sci-fi. Although, I have seen examples of this genre going back further, like "Panic in the Year Zero". Then, that budding genre really manifested itself in the 70's with the likes of "THX1138", "Omega Man", "Soylent Green" and their ilk. The jaded youth culture and the older opportunistic directors that catered to them are who brought you the 70's dystopian films, IMHO.
These movies were based on best selling books so the story was already known to be excellent. By the 80's the books/stories had dried up except for Stephen King who was admittedly coked out of his mind and he wrote 10-20 books that became movies(mostly just so-so). After Hollywood ran out of Stephen King stories they could only think of going back to the Star Wars and along the way they found out that animation was an absolute gold mine.