Raphael uses SVG. Paper use Canvas.
That's the major difference.
In terms of what you can do with them, Canvas and SVG each have their own place, and are good for different things (although they are both capable of doing each other's thing as well if you ask them to). From a purely functional point of view, you need to consider what you want to do with the library before you decide which one you go with.
Browser compatibility is going to be a big issue, whichever one you use. This will possibly be a bigger issue than functionality, in fact.
Raphael has an advantage on the desktop because it detects older versions of IE (as far back as IE6) and falls back to using VML instead of SVG. This means it has excellent compatibility on virtualyl all desktop browsers. Canvas simply isn't supported on older IEs, and the Paper.js people don't really seem too worried about it.
But on mobiles, Paper.js may be better, because Canvas has much better support on mobiles than SVG. SVG isn't supported on most Android devices at all. This is changing: Android 3.0 introduced SVG support, but most Android devices being sold even now come with v2.x, so it'll be a while before you can rely on SVG working on a mobile.
For more info about browser support, see the CanIUse site:
Hope that helps.
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…