Sexually reproducing populations with small number of individuals may go extinct by stochastic fluctuations in sex determination, causing all their members to become male or female in a generation. In this work we calculate the time to extinction of isolated populations with fixed number $N$ of individuals that are updated according to the Moran birth and death process. At each time step, one individual is randomly selected and replaced by its offspring resulting from mating with another individual of opposite sex; the offspring can be male or female with equal probabilities. A set of $N$ time steps is called a generation, the average time it takes for the entire population to be replaced. The number $k$ of females fluctuates in time, similarly to a random walk, and extinction, which is the only asymptotic possibility, occurs when $k=0$ or $k=N$. Contrary to the classic random walk, we show that the time to extinction is not proportional to $N$ generations but to $\tau_e = 2^N/N$. The dynamics stabilizes the populations despite its random nature, resulting in small extinction probabilities. The system has another important time scale $\tau_r=1$: it takes only one generation for an arbitrary initial distribution of males and females to approach the binomial distribution. This distribution, however, is unstable and the population eventually goes extinct in $\tau_e$ generations. Finally, we discuss the robustness of these results against bias in the determination of the sex of the offspring, a characteristic promoted by infection by the bacteria Wolbachia in some arthropod species or by temperature in reptiles.
