I generally agree that the US could have taken relations with Russia in an entirely different direction following the end of the Soviet Union. Just compare the US treatment of Germany and Japan following WWII to US relations with Russia following the end of the cold war. For the former, there was the Marshall Plan, which included both grants and loans and allowed Germany and Japan to rapidly rebuild both the private economy and public institutions. For Russia, it was shock therapy. Russia was forced to auction off its public industries in fire sales to Russian oligarchs who quickly moved their wealth offshore. Russia immediately went into a deep depression due to the shock therapy imposed by the West. Had the West provided aid and not conditioned loans on the complete privatization of all state run industries (or at least a gradual privatization that allowed state run industries to benefit from capital investment instead of being auctioned off), Russia would have emerged from communism as a country that could readily be integrated into Europe and pacified to the point of nuclear disarmament.
The big question surrounding Russia back in the 1990s was whether it would emerge as a democracy or would revert back to being a communist country. Following the first Chechen war, much of Eastern Europe became afraid that Russia was going to try to reconstitute the Soviet Union as a new Russian republic. From that point on, it has been a chicken and the egg problem of whether Russian aggression spurred further NATO expansion or vice a versa.
But Russia's intervention in Syria made it clear that Russia's main concern about NATO wasn't that it was a threat to Russia's sovereignty. It was because NATO was a threat to Russia's desire to project its power in the region, especially in Ukraine where Russia had already annexed Crimea.
Russia could have a thousand missiles pointed at it from the Ukraine border and 500,000 NATO troops amassed on the Ukraine border and not a single shot would be fired because any military action would be met with a nuclear response. Even in Syria, the US and its allies had no choice but to let Putin prop up Assad because of the threat of direct conflict with Russia and a nuclear confrontation.
So we are left with another chicken and the egg problem. Ukraine wants to join NATO because it is afraid of a Russian invasion and Russia wants to invade Ukraine because it is afraid of NATO expansion. But Russia's real fear is that it would not be able to invade Ukraine once it became a NATO member.
