From 20c343b89ee729577427cad55db6875bb0307415 Mon Sep 17 00:00:00 2001 From: Dan Willemsen Date: Fri, 2 Mar 2018 15:03:24 -0800 Subject: [PATCH] Cap the number of cpus for Go compiles When passing "-c" to the Go compiler, any time this value changes, we'd force all of the Go compiles to rebuild. This could trigger a substantial portion of the tree to rebuild (anything that transitive depends on a Go helper tool). We're running into issues when moving output directories between multiple GCE machines with different core counts, but are otherwise identical. This could also hit users moving/mounting disks between machines, though changes to other host tools can make an impact too. On my 48-core machine, I get a ~15% benefit from going from -c 1 to -c 48, but also ~12% benefit from going from -c 1 to -c 8. So this will still let us scale somewhat, but prevent rebuilds when transitioning between machines that are more likely building Android. --- bootstrap/bootstrap.go | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/bootstrap/bootstrap.go b/bootstrap/bootstrap.go index 4f7d916..e9a8f01 100644 --- a/bootstrap/bootstrap.go +++ b/bootstrap/bootstrap.go @@ -39,7 +39,15 @@ var ( // Parallel compilation is only supported on >= go1.9 for _, r := range build.Default.ReleaseTags { if r == "go1.9" { - return fmt.Sprintf("-c %d", runtime.NumCPU()) + numCpu := runtime.NumCPU() + // This will cause us to recompile all go programs if the + // number of cpus changes. We don't get a lot of benefit from + // higher values, so cap this to make it cheaper to move trees + // between machines. + if numCpu > 8 { + numCpu = 8 + } + return fmt.Sprintf("-c %d", numCpu) } } return "" -- GitLab