Reland "Simplify our build file now we don't support non-neon."

This reverts commit 136b23cf.

The build system should no longer mistakenly believe that it needs
non-neon for the NDK.

Change-Id: I20dd2c43abf03f4cce39c1f7e558972c1a1cabc9
Test: treehugger