The random arithmetic which is the basis of the CESTAC Method consists in randomly perturbing the last bit of the mantissa of the result of each arithmetic operation. Let us consider a computer using such a random arithmetic. N parallel runs of a program on this computer, provide for each result a set of N samples. It has been shown that under some hypothesis, which generally hold in real life problems, these N samples belong to a Gaussian distribution centered on the exact mathematical result. The stochastic arithmetic, presented in this paper, is a modelization of the above described implementation of the random arithmetic. It is shown in this paper that, on the contrary to the usual floating-point arithmetic, most of the algebraic properties of the exact arithmetic are recovered in stochastic arithmetic. In the fields of round-off error analysis and validity of numerical software, stochastic arithmetic makes possible to develop theoretical studies. In fact the stochastic arithmetic is for the stochastic approach of round-off error analysis, what interval arithmetic is for its deterministic approach. The use of stochastic arithmetic in computing, makes possible: to estimate the accuracy of any result provided by a computer, to detect the numerical instabilities during the run of a program, to improve numerical algorithms. Briefly, the stochastic arithmetic is a powerfull tool for scientific computing. As illustration, its use in linear algebra will be proposed in a next paper.