lib/digsig: additional sanity checks against badly formated key payload

Added sanity checks for possible wrongly formatted key payload data:
- minimum key payload size
- zero modulus length
- corrected upper key payload boundary.

Signed-off-by: Dmitry Kasatkin <dmitry.kasatkin@intel.com>
Reviewed-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Signed-off-by: James Morris <jmorris@namei.org>