Remove side effects of contact affinity.

Let's not make things noiser, especially if the app
or user doesn't want them to be.

Also clean up a bunch of warnings in the test class.
the real change is at the bottom.

Test: runtest systemui-notification
Change-Id: I73bc41d3784a94e85daa2225c0582f1d900a3ef4
Fixes: 117406083