What would the night sky look like if the interstellar medium didn't exist to absorb or block light?

The main effect would be that the Milky Way would become much more prominent and asymmetric.

At the moment, our view into the Galactic plane is limited to around 1000-3000 parsecs by dust. If you look at the Galactic latitude distribution of naked eye (Aren't there more naked-eye-visible stars in the Milky Way plane? ) you see that most naked eye stars are much closer than this and there is only a modest concentration towards the Galactic plane.

What I think this means is that there would not be a big increase in the numbers of resolved naked eye stars. It is the sensitivity of our eyes, rather than dust that limits how far we can see them. However, the numbers of unresolved stars in the Milky Way would be greatly increased. The number of stars in the resolution element in your eye would increase as distance squared, but the light received decreases as distance squared. This means each "shell" you add contributes equally to the observed brightness.

The Sun is in a 30000pc diameter Galactic disc, about 8000pc from the centre. Thus I estimate that going from being able to see stars to 2000pc, seeing them to anywhere from around 7000pc (away from the Galactic centre) to 23000pc (through the Galactic centre) will increase the brightness of the Milky Way by factors of 3-10, strongly concentrated towards the Galactic plane (more so than now), depending on which way you look. This asymmetry will be increased by the increasing stellar density towards the Galactic centre.

In addition, the "bulge" would become more prominent. This is a pseudo-spherical region of diameter 4000pc, with a greater density of stars, centred on the Galactic centre. The bulge is mainly hidden by dust now, but without dust we would see a brighter, roughly circular blob towards the Galactic centre (Saggitarius), with an angular diameter of 30 degrees.