big bang theory - When was visible light for the first time created?

To expand a bit on James Kilfiger's great answer:

The peak wavelength of the cosmic microwave background

The wavelength of the cosmic microwave background (as well as any other photon) is stretched proportionally to the scale factor $a(t) = 1/(1+z)$ ("the size") of the Universe, where $t$ is the age of the Universe and $z$ is the redshift of light emitted at $t$ and observed today.
Thus, at redshift $z$ the peak of the CMB spectrum was lying at a wavelength
$$
lambda_mathrm{max}(z) = frac{lambda_mathrm{max,0}}{1+z}.
$$

Today, the CMB spectrum peaks at $lambda_mathrm{max,0} simeq 0.2,mathrm{cm}$. If we take visible light to lie in the range 400-700 nm, the maximum of the CMB spectrum must have been in the visible range at redshifts
$$
z ,,=,, frac{lambda_mathrm{max,0}}{lambda_mathrm{max}(z)} - 1 ,,=,, frac{0.2,mathrm{nm}}{{4,verb+-+,7}times10^{-5},mathrm{nm}} - 1 ,,sim,, {2700,verb+-+,4700}.
$$
This can be converted to an age of the Universe by integrating (numerically) the Friedmann equation. Assuming a Planck 2015 cosmology, I get that the peak of the CMB spectrum was in the visible range when the Universe was between roughly 50,000 and 125,000 years old.

The color of the Universe

However, as James Kilfiger also mentions, even though the peak lies outside the visible range, a fraction of the spectrum is still inside. Its color depends on the response of the cones of the human eye and follows approximately the so-called Planckian locus in the CIE 1931 color space:

In the plot above, the locus traces the color of the Universe as a function of temperature (thin numbers), and age of the Universe in kiloyears (kyr; bold numbers).

That is, the Universe started out as bluish (when $Tgtrsim10^4,mathrm{K}$), became whitish at an age of around 200,000 years (when $Tsim5verb+-+6000,mathrm{K}$), and then gradually went over orange and red at $tsim1,mathrm{Myr}$ before fading into the infrared. This evolution is completely analogous to the colors of stars of a given temperature.

The ages have been calculated with Python's CosmoloPy.distances.age(), which I think is not very accurate at such small ages, but it does give the approximate age. Also, whether or not the CMB was visible to the naked eye — i.e. without a telescope to enlarge the light-collecting area — depends on the intensity at a given point.