Only layout surfaces if applicable when app is going away

Otherwise we defer/continue layout which causes at least one
layout unnecessarily when an app without any visible activities
gets killed.

Test: Boots
Test: Kill app while in foreground

Change-Id: Id07885f57aafe63dc8fd82810d1d9118144ca938
Fixes: 117751071