did you manage a full rgb picture before? this somehow seems like two unrelated issues.
about the values going crazy, a few simple things to check before really going in:
-all ribbons ok/plugged the right way? also in the console/fader bus cards?
-console setup is all correct in the computer - right amount of channels, vca groups asigned correctly etc?
-input/output test in the !tes menu comes out clean?
i've seen similar symptoms on an e series before, and it was a fried resistor on the tr buffer (in the keyboard).