> the filter only seems to alter the alpha component of the frames

Thanks, that worked!

    ffmpeg -loop 1 -i background.jpg -i /dev/video0 \
        -filter_complex "
            [1:0]frei0r=hqdn3d:.15|.15,
                 frei0r=bgsubtract0r:.1|y,
            [0:0]scale2ref,overlay" \
        -pix_fmt yuv420p -f xv - | ffplay -

Unfortunately, even with high quality denoising and other magic, I was
unable to achieve satisfactory results for my variegated setting. :<