webgl - Outline object (normal scale + stencil mask) three.js

Question

For some time, I've been trying to figure out how to do an object selection outline in my game. (So the player can see the object over everything else, on mouse-over)

This is how the result should look:

The solution I would like to use goes like this:

1. Layer 1: Draw model in regular shading.
2. Layer 2: Draw a copy in red color, scaled along normals using vertex shader.
3. Mask: Draw a black/white flat color of the model to use it as a stencil mask for the second layer, to hide insides and show layer 1.

And here comes the problem. I can't really find any good learning materials about masks. Can I subtract the insides from the outline shape? What am I doing wrong?

I can't figure out how to stack my render passes to make the mask work. :(

Here's a jsfiddle demo

renderTarget = new THREE.WebGLRenderTarget(window.innerWidth, window.innerHeight, renderTargetParameters)

composer    = new THREE.EffectComposer(renderer, renderTarget)
// composer   = new THREE.EffectComposer(renderer)

normal      = new THREE.RenderPass(scene, camera)
outline     = new THREE.RenderPass(outScene, camera)
mask        = new THREE.MaskPass(maskScene, camera)
// mask.inverse = true
clearMask   = new THREE.ClearMaskPass
copyPass    = new THREE.ShaderPass(THREE.CopyShader)
copyPass.renderToScreen = true

composer.addPass(normal)
composer.addPass(outline)
composer.addPass(mask)
composer.addPass(clearMask)
composer.addPass(copyPass)

Also I have no idea whether to use render target or renderer for the source of the composer. :( Should I have the first pass in the composer at all? Why do I need the copy pass? So many questions, I know. But there are just not enough resources to learn from, I've been googling for days.

Thanks for any advice!

Answer 1

Here's a js fiddle with working solution. You're welcome. :)

http://jsfiddle.net/Eskel/g593q/6/

Update with only two render passes (credit to WestLangley): http://jsfiddle.net/Eskel/g593q/9/

The pieces missing were these:

composer.renderTarget1.stencilBuffer = true
composer.renderTarget2.stencilBuffer = true
outline.clear = false