Table of absolute magnitudes of stars by spectral type and photometric band

There are a couple of standard papers containing the table you want.

Kenyon & Hartmann (1995) http://adsabs.harvard.edu/abs/1995ApJS..101..117K

Table A5 contains many colours for stars as a function of spectral type. You need to combine this with something that gives absolute V magnitude along the main sequence, like that of Schmidt-Kaler (1982), which can be found online at
http://xoomer.virgilio.it/hrtrace/Sk.htm

An alternative, that has colours for both main sequence and pre main sequence stars (though the absolute magnitude of a PMS star is age-dependent) is found in Pecaut & Mamajek (2013).
http://arxiv.org/abs/1307.2657

Online table at
http://vizier.cfa.harvard.edu/viz-bin/VizieR-3?-source=J/ApJS/208/9/table5