Delete latency tracking information in the Connection.

The information gathered here will no longer be valid once we
start dispatching multiple events at a time to the same connection.
Moreover, we are more concerned with end-to-end latency, which we
can measure with sufficiently high accuracy in other ways.

This is part of a series of changes to improve input system pipelining.

Bug: 5963420

Change-Id: I49a0c9876b64af56b40e96e0d98c45f325da2a73