Support for early mutators

"Variant" has been used to mean two slightly concepts in Blueprint.
The first and most obvious is a variant of a module, meaning a list
of all the ways that on version of a module differs from other
versions of the same modules, for example "arch:arm, link:static".
The other is for the specific way that a version of a module differs,
"arch:arm".

Rename all of uses of "variant" as the second meaning to "variation",
and update the documentation to clarify "variant" vs. "variation".
This modifies the build logic api to convert CreateVariants to
CreateVariations.

Change-Id: I789ef209ae6ddd39ec12e0c032f9f47f86698fe6

The mutators that run after dependencies are resolved can be too late
to support build logic that needs to vary the dependencies based on
the mutated axis, for example architecture.  This patch provides an
EarlyMutator interface that can be used to mutate modules before
any dependencies have been resolved.

In order for dependencies to be satisifed in a later pass, all
dependencies of a module must either have an identical variant,
must have a single variant, or must be inserted using
DynamicDependencyModuleContext.AddVariantDependency.

Change-Id: Ic6ae57e98edfd6c8c09a7788983128d3e4e992f0

Move actionDefs from moduleGroup to moduleInfo.  This will result
in multiple build.ninja headers for a single Blueprint file module
if it has multiple variants, each one specifying the variant
used to generate the following rules.

Change-Id: I414cd4d3266da8c2e92118b295569627ddf48cdd

Instead of iterating through groups and then iterating through the
group's modules, iterate through the modules directly.  This will
allow circular dependencies between groups as long as the individual
modules don't have circular dependencies.  In order to maintain the
ordering of modules in a group, each module is considered to have
an implicit dependency on any earlier modules in the same group.

Change-Id: Ibce67167c7234db70ede0a6b5c2b43fb8e0bb05d

Replace the array of mutator name/variant name pairs with
a map of mutator name to variant name, and store the string
variant name separately.

Change-Id: I181c2fcb05724c8755b90aaf866fdce9ef720b01

Move most of the contents of moduleGroup into moduleInfo.  This will
eventually reduce moduleGroup to a simple list of variants of the
same module.

Change-Id: I4289eb9953509751d7637558cd6db73373ccdf78

Forcing module names to be valid ninja names is an unnecessary
restraint on the project build logic.  Allow any string as a
module name, and sanitize and uniquify the module name for use
in module-scoped variables.

Also move the module scope to be per-module instead of per-group
so that modules can use the same local variable name for each variant.

Change-Id: If44cca20712305e2c0b6d6b39daa5eace335c148

Bugfixes

The cycle check can panic if the first call to check happens to land
on the first module in a cycle.  Print the cycle instead of panicking.

Change-Id: I6fc1c66dcc37b1eb6b11b9e65343452af3c8538d

If proptools.CopyProperties is passed two values that point same
slice then setting the destination slice to a new slice will
overwrite the source slice, and the properties struct that is both
the source and destination will have an empty slice.  Copy into
the new slice using a new reflect.Value, and then update the
destination.

Change-Id: I1bfcdc51e4278ea7c7ed81dafc928a5471219f05

Change-Id: I75875cbf60efc9aaf7c2df5709533c2c04b6fba4

New features

Miscellaneous bugfixes

Add overall Blueprint documentation

Variant errors can be introduced by Blueprints files, so print
a module error instead of panicing.

Switch back to:
moduleType {
    name: value,
    arch: {
        x86: {
            name: value,
        },
    },
}

This provides better consistency between properties defined at the
top level of a module and properties defined inside a map.

The parser will continue to support the other format for now, but the
printer will only produce the original format.

Change-Id: Ia9c5408b1b159843dbc57e2e480ca2972598186d

Add support for + operator on maps.  The semantics are that keys that
exist in both maps are added with the + operator, and keys that exist
in one map are copied to the resulting map.

Change-Id: Iba9a6f886477a1eb7311272d07944800c806e368

Change-Id: I271db2120aba1a5c99f21198f1eff6370e6e2389

Support += assignments to variables.  Variables are now mutable
up until they are referenced, then they become immutable.  This
will allow variables to be modified in a conditional, or allow
better commenting on why parts of a variable are set.

Change-Id: Iad964da7206b493365fe3686eedd7954e6eaf9a2

The printer was allowing extra newlines in odd places, for example
between a property name and the '=' in a module definition.  Make
newline location stricter by explicitly allowing newlines or
line comments.

generateModuleBuildActions was continuing to call GenerateBuildActions
on all modules even after one returned an error.  This could cause a
later module to panic because it was missing information that was
supposed to be generated by the module that returned an error, hiding
the earlier error message.

If a module returns an error, stop triggering any new goroutines to
call GenerateBuildActions on later modules.

If .ninja_log contains an entry that used to be a file but is now a
non-empty directory the os.Remove call will fail.  Silently ignore
this error, if the files in the directory are abandoned the directory
will be cleaned up when the files are removed.

Change-Id: Ibfc1190f2b5ac3c3445d40f1b5dd0cd782e63dfd

Change-Id: I27f0f010c32b3abe0107e69af830716ab728485a

Updates the Blueprints file to the new format and standard layout.

Change-Id: I1bfb67ab106c2a61fbe6f8201195efc3f8ff5566

Change-Id: I6f7c7374093c0745ee4aa677480376a06648b358

Make integrating with go tools easier by putting the blueprint package
files in the top level directory of the git project instead of in a
subdirectory called blueprint.

Change-Id: I35c144c5fe7ddf34e478d0c47c50b2f6c92c2a03

Updates the standard format from:
module_type {
    property: "value",
}
to:
module_type(
    property = "value",
}

The parser will accept both formats for now, so all files can be
updated to the new format using bpfmt -w .

Change-Id: I9566cf1bd3cd66c6cbf3d4bd9dac1f04b9112927

Don't sort the dependencies, keep them in the order they were in the
Blueprints file and use a set to prevent adding duplicates to the list.

Change-Id: Icade91d035ccfb12c26ba0584144c1723607019c

The final build.ninja can't contain a rules to build the primary
builder or it will create a circular dependency, so rules for all
bootstrap go packages and binaries are replaced with phony targets
that trigger a re-bootstrap to rebuild.  Add all bootstrap go
binaries to the dependencies that trigger a rebuild so that they
all get rebuilt.

Also fix the phony rule generation to include rules for the
intermediate files of bootstrap go binaries so that they don't
get deleted by the cleanup phase when the primary builder runs.

Change-Id: Id0cc287789f74cc4b21b062086723712b57ee051

bpmodify can be used to add or remove dependencies or other strings
from selected modules in Blueprint files.

Change-Id: I0df3762976e74bf46fd2922bbd48b46e526b7951

bpfmt is based off gofmt, and formats a blueprint file to a standard
format.

Change-Id: I060c1b6030bc937a8db217eaed237d8792c29565

Return comment info from parser, which will be used later for
pretty printing a parsed Blueprint file.  Also add more position
information to the AST.

Change-Id: I3cf0ba8ea4ba5b931d032098ae5fdc350806ed98

Modify the Generate phase to call GenerateBuildActions inside a
goroutine to allow parallization.  All modules from a single
moduleGroup are called from a single goroutine, and the goroutines
that processed all dependencies must have finished, so the only
change to the semantics of GenerateBuildActions implementations
is around access to globals.  Locks are added to the lazily
created package context variables and to the live tracker.

Reduces run time by ~33%.

Change-Id: I62a515acf86f1dcecb093ded83444b920ff603be

Parse multiple Blueprints files simultaneously by using a goroutine
to read the file, call the module factory, parse the properties,
and look for subdirectories.  Requires module factories to be
reentrant, but otherwise does not change any build logic semantics.

Change-Id: Ia07579917a2b6fd064cedbfff655c23d706da718

PreGenerateBuildActions is no longer used by the build logic, remove it.

Change-Id: I621c26e796d2bf6cdc587770a8f9572a769bc705