Make sure the removeObservers be called correctly

Handle the surface destroyed case and add a timeout for finish call.

Bug: 196171638
Test: adb shell dumpsys SurfaceFlinger
Test: check the offscreen layers
Change-Id: Id1c0e3a0aea1e27a5581a9c7bdcbe6562f8e2ab6