Not bypassing when the user is expanding quick settings

Previously we would also bypass when the user would
look at QS.

Fixes: 134094877
Test: pull down QS, wait on bypass, nothing happens!
Change-Id: I0b2ec98f3c361307845c16cc27866c6f3b3d4136