High resolution sun spectra from the Kitt Peak observatory are used to estimate the column-averaged mixing ratio of carbon dioxide in the atmosphere. Solar absorption lines are apparent on the measured spectra together with the telluric lines. Their discrimination is easy as they are spectrally shifted according to the Earth-Sun relative speed. For the objective of CO$_2$ mixing ratio estimate, a spectral window of size 15 cm$^{-1}$, with approximately 10 absorption lines, contains most of the necessary information. The window selection is based on the absence of contamination by other gases and solar absorption lines. Favorable spectral windows are found near 1.6 and 2.0 mm. A radiative transfer model is used to reproduce the data and make the inversion. An excellent fit is obtained. The retrieved CO$_2$ mixing ratio agrees with the expected values within a few percents and shows the expected growth rate of 1.5 ppm per year. On the other hand, the observed short-term variability of several ppm is much larger than what is expected from simulations by an atmospheric transport model using optimized fluxes. It appears necessary to correct the CO$_2$ mixing ratio estimates using concomitant measurements of the O-2 mixing ratio, which provides a proxy for the dry air optical path. After the correction, the annual cycle of the CO2 content above Kitt Peak is apparent on the retrieved values. The comparison of the inverted mixing ratios with the simulation results indicates an error on the order of 1.5 ppm RMS for the 1.6- $\mu$m band and 2.5 ppm RMS for the 2.0-$\mu$m band. The uncertainty on the effective airmass due to the long acquisition time of the spectra, with varying solar zenith angle, may be responsible for a large fraction of the error.