You have to get wavelength from x position
You can not compute wavelength from color as the two are not the same thing. So first you should calibrate your spectroscope with known wavelengths. From that infer function:
wavelength = f(x)
either by LUT and interpolation, or by approximation polynomial. For more info see:
You can use Sun light as a reference and calibrate based on the known spectral (Fraunhofer) lines. Here first example I found on google:
So take/plot a sunlight shot here mine:
cross match the Fraunhofer lines (the darker lines, beware of over exponated images they can screw things up, also the Intensity is R+G+B without weight we do not want human perception like conversion) and make a table of Known wavelengths x positions in your image. From that interpolate your wavelength = f(x).
As you can see my shot of Sun's spectra more or less matches the reference one (discrepances are due to grating material, bayer filter, camera properties, clouds and atmosphere,etc...). How ever the Fraunhofer lines are not easily detectable by local minima so may be some user assistance GUI style will be a better idea to start with.
But beware that most spectra images on the WEB is wrong or nonlinear or shifted !!! So to be sure I created a reference spectra from linearized spectral data like this and here the result 400-700 [nm]:
And here the plot:
the gray lines are grid from 400-700 nm with 10 nm step.
Here is how your setup should look like:
Here is image from mine spectro-scope (looking at White area on my LCD):
I am using grating grid made from DVD hence the circular arc shapes. Now if your camera is in fixed position relative to your prism then for selected horizontal line the x position of pixel directly corresponds to a specific wavelength.
If you do not see any Fraunhofer lines then you are missing aperture before the prism/grating. I usually use 2 razor blades distanced by 0.1 mm set by thin paper. If your image is out of focus you need to add lens(es) before your camera/sensor and or add more shielding from outside light.
As I mentioned before You cannot get wavelength from color because there is "infinite" combinations of input spectra creating the same RGB response. For example take white color ... it can be composed form 3 or more distinct wavelengths or even continuous white noise. So from RGB you can not tell which one it is... If you add also the x positon with combination of prism/grating then you can get the wavelength but it would be much more complicated and less precise then direct conversion from just x position...
compute intensity from RGB
this may be a bit tricky as your sensor may have different sensitivity for different wavelengths. You can normalize intensity similarly to #1. Just take shot of known intensity light source and approximate for the missing wavelengths. Also this can be done with Sunlight as a source
From normalized color you just compute gray-scale intensity and that is it.
To improve accuracy you can average all the pixels for the same x.
Also to boost accuracy and sensitivity usually non color sensor is used (mostly linear cameras) either by design or by removing the Bayer filter so it does not mess up the data.
