Make sure the removeObservers be called correctly

Handle the surface destroyed case and add a timeout for finish call.

Bug: 196171638
Bug: 197555842
Test: adb shell dumpsys SurfaceFlinger
Test: check the offscreen layers
Test: see 198714745#comment6
Change-Id: Ic6d2521d788c50e10b81b556849194a97f240376
Merged-In: Iec1eb82d1a4756a7b85c8fbdd05f4132a74c613c