Proposal: Features / Feature packages (Feb 23 2017)

Note: this is the feature as it was initially specified and does not necessarily reflect the current behavior.

Up-to-date documentation is available at Selecting Library Features.

1. Motivation

A. OpenCV + CUDA

OpenCV is a computer vision library that can optionally be built with CUDA support to massively accelerate certain tasks when using computers with NVidia GPUs. For users without NVidia GPUs, building with CUDA support provides no benefit. CUDA is provided only via a 1.3 GB installer (at the time of this authoring), which requires administrator access to install and modifies the global system state.

Therefore, there is significant value in enabling users to choose whether they find CUDA support valuable for their particular scenario.

B. OpenCV + OpenCV_contrib

The community around OpenCV has built up a library of extensions called OpenCV_contrib. However, these extensions are a source-level patch onto the main OpenCV codebase and therefore must be applied during the core OpenCV build. Further confounding the problem, it is the author's understanding that these community extensions have only been developed with CUDA enabled and cannot be built without that dependency.

Therefore, if CUDA is disabled, OpenCV_contrib must also be disabled. Likewise, when a user requests OpenCV_contrib, CUDA must be enabled. It would be convenient, but not a requirement, to enable CUDA without enabling the community extensions.

Finally, these extensions add additional exports and headers which could be depended upon by other libraries. For maintainers, there must be a way to specify this requirement such that vcpkg install mylib-depends-ocv-contrib will verify/build/rebuild OpenCV with the community extensions enabled.

C. C++ REST SDK + SignalR

The C++ REST SDK is a networking library that provides (among other features) HTTP and Websockets clients. To implement the HTTP client functionality on Windows Desktop, only the core Win32 platform APIs are needed (zlib is optional).

However, the websockets client is based on Websockets++, which adds mandatory dependencies on boost, openssl, and zlib. Many users of the C++ REST SDK do not use the websockets component, so to minimize their overall dependency footprint it can be disabled at build time. Ideally, these kinds of options would be easily accessible to users in Vcpkg who are concerned about the final size or licensing of their deployment.

SignalR-Client-Cpp depends on the websockets functionality provided by the C++ REST SDK. Therefore, the maintainers of the signalrclient port would ideally like to express this dependency such that cpprestsdk will be automatically correctly built for their needs. Note that signalrclient does not inherently care about boost, websocketspp or openssl -- it depends only on the public websocket client APIs provided by cpprestsdk. It would be much more maintainable to declare dependencies based on the public APIs rather than the dependencies themselves.

2. Other design concerns

3. Proposed solution

A key summary of the above motivations is that they are all scenarios surrounding APIs that are not independently buildable from each other. We have an existing solution for APIs that are independently buildable: separate packages. Therefore, we seek to extend the user-facing notion of "packages" to include capabilities and contracts that cannot be made into independent builds.

This document proposes "features" (also called feature packages). These features are intended to model semi-independently toggleable API sets/contracts such that they can be sanely depended upon by other packages. It is not a goal to model exclusive alternatives (such as implementation choices that are not directly user-observable) through this mechanism.

From a user experience perspective (i.e. from vcpkg install) feature packages act as much as possible like completely independent packages. However, internally, any change to a package's features will result in a rebuild of the associated "parent" package. This will invoke a package rebuild experience similar to upgrading.

When using vcpkg install <package>, some features will be enabled by default. These default features can be avoided by referring to the packages as <package>[core] and features can be added by supplying them on the same installation line.

A. Proposed User experience

i. User with no preference about options

Install of a library with default features:

> vcpkg install cpprestsdk
// -- omitted build information -- //
Package cpprestsdk[core]:x86-windows is installed.
Package cpprestsdk[compression]:x86-windows is installed.
Package cpprestsdk[ws-client]:x86-windows is installed.

Removal of that library:

> vcpkg remove cpprestsdk
The following packages will be removed:
Removing package cpprestsdk:x86-windows...
Removing package cpprestsdk:x86-windows... done
Purging package cpprestsdk:x86-windows...
Cleaned up D:\src\vcpkg\packages\cpprestsdk_x64-windows
Purging package cpprestsdk:x86-windows... done

Installation of a library with optional features:

> vcpkg install opencv
// -- omitted build information -- //
Package opencv[core]:x86-windows is installed.

ii. User desires CUDA support for OpenCV directly, and is unfamiliar with feature packages

Developer Bob knows he wants OpenCV, so he guesses what the package is called

> vcpkg install opencv
// -- omitted build information -- //
Package opencv[core]:x86-windows is installed.

Bob attempts to build his application against OpenCV (assuming CUDA), which fails at runtime or compile time indicating that OpenCV wasn't built with CUDA. Bob comes back to vcpkg, not knowing about the "feature packages" feature. The primary inquiry tools for Vcpkg are search and list, so he runs vcpkg search:

> vcpkg search opencv
opencv               3.2.0            computer vision library
opencv[cuda]                          support for NVidia CUDA
opencv[contrib]                       community supported extensions for OpenCV

If your library is not listed, please open an issue at:

He isn't immediately sure what the lack of a version number means, but anything in vcpkg search can be applied to vcpkg install, so he runs:

> vcpkg install opencv[cuda]
The following packages will be rebuilt:

To rebuild with this feature, use:
    vcpkg remove opencv:x86-windows
    vcpkg install opencv[core,cuda]:x86-windows

Bob follows the instructions...

> vcpkg remove opencv:x86-windows
// -- omitted results as above -- //
> vcpkg install opencv[core,cuda]:x86-windows
// -- omitted build information -- //
Package opencv[core]:x86-windows is installed.
Package opencv[cuda]:x86-windows is installed.

and he can now use OpenCV's CUDA support in his application.

iii. User is familiar with feature packages, and wants to opt-out of a feature

Developer Alice has used cpprestsdk, built it from source, and she knows about the option to disable websockets. She uses search to find the complete list of features:

> vcpkg search cpprestsdk
cpprestsdk                  2.9.0-2       C++11 JSON, REST, and OAuth library The C++ RES...
cpprestsdk[compression]                   Gzip compression support in the HTTP client.
cpprestsdk[ws-client]                     Websocket client support based on websocketspp.

If your library is not listed, please open an issue at:

She decided she only wants cpprestsdk[compression], so she installs only that feature:

> vcpkg install cpprestsdk[compression]
// -- omitted build information -- //
Package cpprestsdk[core]:x86-windows is installed.
Package cpprestsdk[compression]:x86-windows is installed.

She receives a quick recursive build that only depends on zlib.

She's now interested in some additional libraries built on top of cpprestsdk: azure-storage-cpp and signalrclient.

> vcpkg install azure-storage-cpp
// -- omitted build information -- //
Package azure-storage-cpp[core]:x86-windows is installed.

> vcpkg install signalrclient
Package signalrclient:x86-windows depends on cpprestsdk[ws-client]:x86-windows.

The following packages will be rebuilt:
  * azure-storage-cpp:x86-windows
  * cpprestsdk:x86-windows

To rebuild the current package graph with this feature, use:
    vcpkg remove cpprestsdk:x86-windows azure-storage-cpp:x86-windows
    vcpkg install cpprestsdk[core,compression,ws-client]:x86-windows
    vcpkg install azure-storage-cpp[core]:x86-windows
    vcpkg install signalrclient[core]:x86-windows

She follows the above script and can use both azure-storage-cpp and signalrclient in her code.

Some time has passed, she decided not to use signalrclient, and she's interested in shipping her application. She wants to minimize her final install size, so she'd like to remove all unneeded packages like boost and openssl.

> vcpkg remove boost openssl
The following packages and features will be removed:
  * signalrclient[core]:x86-windows
  * cpprestsdk[ws-client]:x86-windows

The following packages will be rebuilt:
  * azure-storage-cpp:x86-windows
  * cpprestsdk:x86-windows

Removing features requires rebuilding packages.
To rebuild the current package graph without these features, use:
    vcpkg remove cpprestsdk:x86-windows azure-storage-cpp:x86-windows signalrclient:x86-windows openssl:x86-windows boost:x86-windows
    vcpkg install cpprestsdk[core,compression]:x86-windows
    vcpkg install azure-storage-cpp[core]:x86-windows

In the end, her final vcpkg list outputs:

> vcpkg list
zlib[core]:x86-windows              1.2.11          A compression library
azure-storage-cpp[core]:x86-windows 2.6.0           Microsoft Azure Storage Client SDK for ...
cpprestsdk[core]:x86-windows        2.9.0-2         C++11 JSON, REST, and OAuth library
cpprestsdk[compression]:x86-windows                 Gzip compression support in the HTTP client.

B. Technical model

As a conclusion of the above, it is expected that all packages will be buildable with all features disabled (just the core feature) and with all features enabled.

C. Proposed Control File Syntax

OpenCV and CUDA

To add the feature CUDA to OpenCV, we will adopt the following syntax in the CONTROL file:

# opencv/CONTROL
Source: opencv
Version: 3.2.0-1
Build-Depends: zlib, libpng, libjpeg-turbo, tiff
Description: computer vision library

Feature: cuda
Build-Depends: cuda
Description: parallel computing platform

Feature: contrib
Build-Depends: opencv[cuda]
Description: library of OpenCV Extensions


# signalrclient/CONTROL
Source: signalrclient
Version: 1.0.0-beta1
Build-Depends: cpprestsdk[ws-client]
Description: C++ client for SignalR.
# cpprestsdk/CONTROL
Source: cpprestsdk
Version: 2.9.0-2
Description: C++11 JSON, REST, and OAuth library ...
Default-Features: compression, ws-client

Feature: compression
Build-Depends: zlib (windows)
Description: Gzip compression support in the HTTP client.

Feature: ws-client
Build-Depends: boost (windows), openssl (windows), websocketpp (windows)
Description: Websocket client support based on websocketspp

D. Additional Control File Technical Details

Cargo's Features (from Rust):

The proposed feature packages are exceedingly similar to Cargo's Features, with the following changes:

Gentoo's USE flags:

Gentoo's USE flags can be shortly summarized as a global set of keywords that is used to make cross-cutting changes to the entire package graph's build configuration. This system standardizes many common settings such that they can be simultaneously toggled for the entire graph.

The most common example of this would be using KDE vs Gnome. A user who knows that, given the choice, they would prefer the KDE/Qt interface can manage the massive space of package configuration efficiently without learning the particular term that each package has decided to call "build using Qt instead of GTK".

USE flags can be customized hierarchically when needed, including at the per-package level. They can be depended upon by other packages, both positively and negatively. USE flags themselves can be used in any boolean expression to determine the complete set of package dependencies, including removing dependencies when flags are enabled.

Problems with USE flags: