Bazel Configurability 2018 Roadmap

Last verified: 2019-01-28

Point of contact: gregestren

Discuss: Configurability roadmap: discussion

This roadmap has been retired. See here for the current roadmap.

Goal

Configurability’s goal is to make Bazel a graceful multiplatform build tool. It also focuses on how users decide how their projects are built.

This translates into the following high-level goals:

1. $ bazel {build|test} :all just works
   1. All targets “know” how to build for the right platforms with the right toolchains and desired options
   2. Users only have to fiddle with options they care about
2. “Platforms” is a first-class concept
   1. “Platforms” and “toolchains” are well-defined, map well to reality, and are easy to create
   2. Builds, build rules, remote executors, etc. naturally integrate platforms

3. Users decide what to configure

   $ bazel build //myapp:fancy_edition automatically builds my app with “fancy” features

4. Users decide what rules to configure
   1. e.g. “all foo rules use the foo toolchain”
   2. All rules can have multiplatform dependencies
   3. All configuration can be encoded in Starlark or BUILD files
5. Builds stay fast and efficient
   1. Multiplatform builds don’t duplicate platform-agnostic work
   2. Building “foo” in distinct configurations produces distinct output paths
   3. Builds remain cross platform-cacheable and remote execution-friendly
   4. Users have robust tools to understand multiplatform effects

Roadmap

Dates are approximate based on our best understanding of problem complexity and developer availability. ETAs will change, but we’ll keep them refreshed and current.

Platforms

There’s a more detailed Platforms Roadmap available with more details on ongoing subprojects.

Rules can declare what kinds of machines they can build on DONE (#5217)

• Because of remote execution, this might not be the same machine Bazel runs on

C++ rules fully support platforms and toolchains PARTIALLY DONE (#6516)

• This gives them first-class Starlark support, select() on platforms, and configuration via --platforms
• These set best practice templates for adoption by other rules

“Toolchain modes” documentation explains how to use flags and configuration to select between multiple toolchains for the same platform NOT STARTED (#6517)

• Examples: debug vs. opt, C++ correctness sanitizers

There’s one, simple way to choose platforms at the command line NOT STARTED (#6518)

• $ bazel build //a:foo_lang_rule --platforms=//platforms:mac
• --cpu, --host_cpu, --crosstool_top, --javabase, --apple_crosstool_top, etc. are obsolete and trigger deprecation warnings

Flagless multiplatform builds (unoptimized) NOT STARTED (#6519)

•       $ cat a/BUILD
        cc_binary(name = "app_for_linux", platforms = ["//platforms:linux"])
        cc_binary(name = "app_for_mac", platforms = ["//platforms:mac"])
  
        $ bazel build //a:all # No command line flags!
  

• Platform-independent deps (e.g. Java libraries) may be built twice: see Correctness and Speed below

Java, Android, Apple rules fully support platforms and toolchains SCHEDULED (#6521)

• These depend on Java and C++, so need to happen after those rules
• --android_sdk, --ios_sdk_version, etc. are deprecated and obsolete

User-Defined Configuration

Starlark supports platform transitions PARTIALLY DONE (#5574)

• Rule designers can decide which rules target which platforms
• Rule designers can declare default target platforms
• Rule designers can have dependencies target different platforms than their parents

Starlark supports multi-architecture (“fat”) binaries DONE (#5575)

• Rule designers can write rules that bundle deps configured for multiple platforms

Starlark supports user-defined configuration settings PARTIALLY DONE (#5577)

• A standard API defines how to declare custom settings (consolidating command line flags, “secret” flags, –define, –features, and feature flags)

• All hard-coded Bazel flags can be migrated to this API. Actual migration may not have begun.
• End users (i.e. non-rule designers) can’t customize settings. We want to start by seeing how far we can get with --platforms and feature flags.

All native Bazel rules can be implemented in Starlark PARTIALLY DONE (#5578)

Correctness and Speed

Users can manually tag rules to not duplicate under feature flags DONE (#6523)

• This makes “feature customization” under Android binaries more efficient
• Non-Android dependencies won’t duplicate due to Android-only changes

Bazel updates fast on --test_timeout, etc. changes DONE (#5579)

• Oct 2018: Must be enabled with --trim_test_configurarion

An experimental Bazel mode automatically minimizes build graphs PARTIALLY DONE (#6524)

• No rule builds twice due to unrelated flag changes

User documentation provides clear guidance on “safe” Starlark transitions use NOT STARTED (#6525)

• Explains the risks of performance and memory regressions
• Explains how to minimize these risks and make informed use of the feature
• Explains how to track improvements and report problemns

Java compilation doesn’t include cpu in its output paths ON HOLD (#6527)

• This improves multiplatform build times and cross-build cacheability
• This is conditional on the impact of generated sources, selects(), etc.

Distinct actions can’t write to the same output path IN PROGRESS (#6526)

• This prevents “output clobbering” when the same command is invoked twice with different inputs, producing different versions of the same output
• This is especially important for Starlark rules

Bazel automatically minimizes graphs over feature flag changes NOT STARTED (#6524)

Bazel automatically minimizes graphs over all configuration changes NOT STARTED (#6524)

• This productionizes the experimental minimization mode

Build actions cache efficiently IN PROGRESS (#6526)

• Content-identical outputs have the same file name (as much as possible)
• Output paths don’t include cache-poisoning segments.: bazel-out/ppc-fastbuild/PlatformIndependentModule.class