V4L/DVB (10261): em28xx: fix kernel panic on audio shutdown

Revert a change made in change 9743 which resulted in a kernel panic in some
cases on shutdown of the audio stream.

First discovered when working on the Pinnacle 880e support, and later
reproduced by a user on the mailing list with the HVR-900 as well.

Signed-off-by: Devin Heitmueller <dheitmueller@linuxtv.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab@redhat.com>