yuzu/vcpkg
1
0
Fork 0
mirror of https://github.com/microsoft/vcpkg synced 2024-11-21 16:09:03 -07:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

[vcpkg] More versioning docs (#15693)

Browse source 
* [vcpkg] Versioning docs: Getting started

* Versioning docs: Getting started [revised]

* Link to example of manifests in msbuild

* [WIP] Reference docs

* [vcpkg] Fix links in Getting Started with Versioning
This commit is contained in:
Victor Romero 2021-02-08 12:03:37 -08:00 committed by GitHub
parent 5eea585548
commit 3187f09a3c
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
4 changed files with 635 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

273
docs/users/versioning.getting-started.md Normal file
View file

@ -0,0 +1,273 @@
# Getting started with versioning
Vcpkg lets you take control of which version of packages to install in your projects using manifests.
## Enabling versions
To start using the versioning feature, first you need to enable the `versions` feature flag in any of the following manners:
* Setting the `VCPKG_FEATURE_FLAGS` environment variable.
```PowerShell
# Example for PowerShell
$env:VCPKG_FEATURE_FLAGS="versions"
./vcpkg install
```
* Passing the feature flags in the vcpkg command line.
```PowerShell
./vcpkg --feature-flags="versions" install
```
## Using versions with manifests
With the `versions` feature flag enabled you can start addding version constraints to your dependencies.
Let's start with creating a simple CMake project that depends on `fmt` and `zlib`.
Create a folder with the following files:
**vcpkg.json**
```
{
"name": "versions-test",
"version": "1.0.0",
"dependencies": [
{
"name": "fmt",
"version>=": "7.1.3"
},
"zlib"
],
"builtin-baseline": "b60f003ccf5fe8613d029f49f835c8929a66eb61"
}
```
**main.cpp**
```c++
#include <fmt/core.h>
int main()
{
fmt::print("fmt version is {}\n"
"zlib version is {}\n",
FMT_VERSION, ZLIB_VERSION);
return 0;
}
```
**CMakeLists.txt**
```CMake
cmake_minimum_required(VERSION 3.18)
project(versions-test CXX)
add_executable(main main.cpp)
find_package(ZLIB REQUIRED)
find_package(fmt CONFIG REQUIRED)
target_link_libraries(main PRIVATE ZLIB::ZLIB fmt::fmt)
```
And now we build and run our project with CMake:
1. Create the build directory for the project.
```
PS D:\versions-test> mkdir build
PS D:\versions-test> cd build
```
2. Configure CMake.
```
PS D:\versions-test\build> cmake -G Ninja -DCMAKE_TOOLCHAIN_FILE=D:/vcpkg/scripts/buildsystems/vcpkg.cmake ..
-- Running vcpkg install
Detecting compiler hash for triplet x86-windows...
The following packages will be built and installed:
fmt[core]:x86-windows -> 7.1.3 -- D:\vcpkg\buildtrees\versioning\versions\fmt\dd8cf5e1a2dce2680189a0744102d4b0f1cfb8b6
zlib[core]:x86-windows -> 1.2.11#9 -- D:\vcpkg\buildtrees\versioning\versions\zlib\827111046e37c98153d9d82bb6fa4183b6d728e4
...
```
3. Build the project.
```
PS D:\versions-test\build> cmake --build .
[2/2] Linking CXX executable main.exe
```
4. Run it!
```
PS D:\versions-test\build> ./main.exe
fmt version is 70103
zlib version is 1.2.11
```
Take a look at the output:
```
fmt[core]:x86-windows -> 7.1.3 -- D:\vcpkg\buildtrees\versioning\versions\fmt\dd8cf5e1a2dce2680189a0744102d4b0f1cfb8b6
zlib[core]:x86-windows -> 1.2.11#9 -- D:\vcpkg\buildtrees\versioning\versions\zlib\827111046e37c98153d9d82bb6fa4183b6d728e4
```
Instead of using the portfiles in `ports/`; vcpkg is checking out the files for each version in `buildtrees/versioning/versions/`. The files in `ports/` are still used when running vcpkg in classic mode or when the `versions` feature flag is disabled.
_NOTE: Output from the vcpkg while configuring CMake is only available when using CMake version `3.18` or newer. If you're using an older CMake you can check the `vcpkg-manifest-install.log` file in your build directory instead._
Read our [manifests announcement blog post](https://devblogs.microsoft.com/cppblog/vcpkg-accelerate-your-team-development-environment-with-binary-caching-and-manifests/#using-manifests-with-msbuild-projects) to learn how to use manifests with MSBuild.
### Manifest changes
If you have used manifests before you will notice that there are some new JSON properties. Let's review these changes:
* **`version`**
```
{
"name": "versions-test",
"version": "1.0.0",
...
}
```
This is your project's version declaration. Previously, you could only declare versions for your projects using the `version-string` property. Now that versioning has come around, vcpkg is aware of some new versioning schemes.
Version scheme | Description
-- | --
`version` | Dot-separated numerics: `1.0.0`.
`version-semver` | Compliant [semantic versions](https://semver.org): `1.2.0` and `1.2.0-rc`.
`version-date` | Dates in `YYYY-MM-DD` format: `2021-01-01`
`version-string` | Arbitrary strings: `vista`, `candy`.
* **`version>=`**
```
"dependencies": [
{
"name": "fmt",
"version>=": "7.1.3"
},
"zlib"
],
```
This property is used to express minimum version constraints, it is allowed only as part of the `"dependencies"` declarations. In our example we set an explicit constraint on version `7.1.3` of `fmt`.
Vcpkg is allowed to upgrade this constraint if a transitive dependency requires a newer version. For example, if `zlib` were to declare a dependency on `fmt` version `7.1.4` then vcpkg would install `7.1.4` instead.
Vcpkg uses a minimum version approach, in our example, even if `fmt` version `8.0.0` were to be released, vcpkg would still install version `7.1.3` as that is the minimum version that satisfies the constraint. The advantages of this approach are that you don't get unexpected dependency upgrades when you update vcpkg and you get reproducible builds (in terms of version used) as long as you use the same manifest.
If you want to upgrade your dependencies, you can bump the minimum version constraint or use a newer baseline.
* **`builtin-baseline`**
```
"builtin-baseline": "b60f003ccf5fe8613d029f49f835c8929a66eb61"
```
This field declares the versioning baseline for all ports. But what is a baseline? What does it do? Why is the value a SHA?
From the [versioning documentation](versioning.md):
> The baseline references a commit within the vcpkg repository that
establishes a minimum version on every dependency in the graph. If
no other constraints are specified (directly or transitively),
then the version from the baseline of the top level manifest will
be used.
In our example, you can notice that we do not declare a version constraint for `zlib`; instead, the version is taken from the baseline. Internally, vcpkg will look in commit `b60f003ccf5fe8613d029f49f835c8929a66eb61` to find out what version of `zlib` was the latest at that point in time (in our case it was `1.2.11#9`).
During version resolution, baseline versions are treated as minimum version constraints. If you declare an explicit constraint that is lower than a baseline version, the explicit constraint will be upgraded to the baseline version.
For example, if we modified our dependencies like this:
```
"dependencies": [
{
"name": "fmt",
"version>=": "7.1.3"
},
{
"name": "zlib",
"version>=": "1.2.11#7"
}
]
```
_NOTE: The value `1.2.11#7` represents version `1.2.11`, port version `7`._
Since the baseline introduces a minimum version constraint for `zlib` at `1.2.11#9` and a higher version does satisfy the minimum version constraint for `1.2.11#7`, vcpkg is allowed to upgrade it.
Baselines are also a convenient mechanism to upgrade multiple versions at a time, for example, if you wanted to depend on multiple `boost` libraries, it is more convenient to set the `baseline` once than declaring a version constraint on each package.
But what if you want to pin a version older than the baseline?
* **`overrides`**
Since baselines establish a version floor for all packages and explicit constraints get upgraded when they are lower than the baseline, we need another mechanism to downgrade versions past the baseline.
The mechanism vcpkg provides for that scenario is `overrides`. When an override is declared on a package, vcpkg will ignore all other version constraints either directly declared in the manifest or from transitive dependencies. In short, `overrides` will force vcpkg to use the exact version declared, period.
Let's modify our example once more, this time to force vcpkg to use version `6.0.0` of `fmt`.
```
{
"name": "versions-test",
"version": "1.0.0",
"dependencies": [
{
"name": "fmt",
"version>=": "7.1.3"
},
{
"name": "zlib",
"version>=": "1.2.11#7"
}
],
"builtin-baseline": "b60f003ccf5fe8613d029f49f835c8929a66eb61",
"overrides": [
{
"name": "fmt",
"version": "6.0.0"
}
]
}
```
Rebuild our project:
```
PS D:\versions-test\build> rm ./CMakeCache.txt
PS D:\versions-test\build> rm -r ./vcpkg_installed
PS D:\versions-test\build> cmake -G Ninja -DCMAKE_TOOLCHAIN_FILE=D:/vcpkg/scripts/buildsystems/vcpkg.cmake ..
-- Running vcpkg install
Detecting compiler hash for triplet x86-windows...
The following packages will be built and installed:
fmt[core]:x86-windows -> 6.0.0 -- D:\vcpkg\buildtrees\versioning\versions\fmt\d99b6a35e1406ba6b6e09d719bebd086f83ed5f3
zlib[core]:x86-windows -> 1.2.11#9 -- D:\vcpkg\buildtrees\versioning\versions\zlib\827111046e37c98153d9d82bb6fa4183b6d728e4
...
PS D:\versions-test\build> cmake --build .
[2/2] Linking CXX executable main.exe
```
And run it!
```
PS D:\versions-test\build> .\main.exe
fmt version is 60000
zlib version is 1.2.11
```
Notice how the `fmt` is now at version `6.0.0` just like we wanted.
## Versions and custom ports
The last thing to discuss is how overlay ports interact with versioning resolution. The answer is: they don't.
Going into more detail, when you provide an overlay for a port, vcpkg will always use the overlay port without caring what version is contained in it. The reasons are two-fold: (1) it is consistent with the existing behavior of overlay ports of completely masking the existing port, and (2) overlay ports do not (and are not expected to) provide enough information to power vcpkg's versioning feature.
If you want to have flexible port customization along with versioning features, you should consider making your own custom registry. See our [registries specification for more details](../specifications/registries.md).
## Further reading
If you're interested in delving deeper into the details of how versioning works we recommended that you read the [original versioning specification](../specifications/versioning.md) and the [implementation details](versioning.implementation-details.md).
See also:
* [Versioning docs](versioning.md)
* [Original specification](../specifications/versioning.md)
* [Versioning reference](versioning.reference.md)
* [Versioning implementation details](versioning.implementation-details.md)

132
docs/users/versioning.implementation-details.md Normal file
View file

@ -0,0 +1,132 @@
# Versioning: Implementation details
## Contents
* [Minimum versioning](#minimum-versioning)
* [Constraint resolution](#constraint-resolution)
* [Acquiring port versions](#acquiring-port-versions)
### Minimum versioning
Vcpkg uses a minimal selection approach to versioning, inspired by the one [used by Go](https://research.swtch.com/vgo-mvs). But modified in some ways:
* Always starts from a fresh install, eliminates the need for upgrade/downgrade operations.
* Allow unconstrained dependencies by introducing baselines.
The minimal selection principle, however, stays the same. Given a set of constraints, vcpkg will use the "oldest" possible versions of packages that can satisfy all the constraints.
Using a minimum version approach has the following advantages:
* Is predictable and easy to understand.
* User controls when upgrades happen, as in, no upgrades are performed automatically when a new version is released.
* Avoids using a SAT solver.
To give an example, consider the following package graph:
```
(A 1.0) -> (B 1.0)
(A 1.1) -> (B 1.0)
-> (C 3.0)
(A 1.2) -> (B 2.0)
-> (C 3.0)
(C 2.0)
```
And the following manifest:
```
{
"name": "example",
"version": "1.0.0",
"dependencies": [
{ "name": "A", "version>=": "1.1" },
{ "name": "C", "version>=": "2.0" }
],
"builtin-baseline": "<some git commit with A's baseline at 1.0>"
}
```
After accounting for transitive dependencies we have the following set of constraints:
* A >= 1.1
* B >= 1.0
* C >= 3.0
* C >= 2.0
Since vcpkg has to satisfy all the constraints, the set of installed packages becomes:
* `A 1.1`, even when `A 1.2` exists, there are no constraints higher than `1.1` so vcpkg selects the minimum version possible.
* `B 1.0`, transitively required by `A 1.1`.
* `C 3.0`, upgraded by the transitive constraint added by `B 1.0` in order to satisfy version constraints.
## Constraint resolution
Given a manifest with a set of versioned dependencies, vcpkg will attempt to calculate a package installation plan that satisfies all the constraints.
Version constraints come in the following flavors:
* **Declared constraints**: Constraints declared explicitly in the top-level manifest using `version>=`.
* **Baseline constraints**: Constraints added implicitly by the `builtin-baseline`.
* **Transitive constraints**: Constraints added indirectly by dependencies of your dependencies.
* **Overriden constraints**: Constraints overriden in the top-level manifest using `overrides` declarations.
To compute an installation plan, vcpkg follows roughly these steps:
* Add all top-level constraints to the plan.
* Recursively add transitive constraints to the plan.
* Each time a new package is added to the plan, also add its baseline constraint to the plan.
* Each time a constraint is added:
* If an override exists for the package
* Select the version in the override.
* Otherwise:
* If there is no previous version selected.
* Select the minimal version that satisfies the constraint.
* If there is a previous version selected:
* If the versioning scheme of the new constraint does not match that of the previously selected version:
* Add a version conflict.
* If the constraint’s version is not comparable to the previously selected version. For example, comparing “version-string: apple” to “version-string: orange”:
* Add a version conflict.
* If the constraints version is higher than the previously selected version:
* Select the highest version.
* Otherwise:
* Keep the previous selection.
* Review the plan:
* If there are no conflicts
* Install the selected packages
* Otherwise:
* Report the conflicts to the user
## Acquiring port versions
Although the concept of package versions has always been present in vcpkg, the concept of version constraints has been not.
With the introduction of versioning constraints, it is now possible that a package depends on a port version that does not match the one available locally. This raises a problem as vcpkg needs to know how to acquire the port files for the requested version.
To solve this problem, a new set of metadata files was introduced. These files are located in the `versions/` directory at the root level of the vcpkg repository.
The `versions/` directory, will contain JSON files for each one of the ports available in the registry. Each file will list all the versions available for a package and contain a Git tree-ish object that vcpkg can check out to obtain that version’s portfiles.
Example: `zlib.json`
```
{
"versions": [
{
"git-tree": "2dfc991c739ab9f2605c2ad91a58a7982eb15687",
"version-string": "1.2.11",
"port-version": 9
},
...
{
"git-tree": "a516e5ee220c8250f21821077d0e3dd517f02631",
"version-string": "1.2.10",
"port-version": 0
},
{
"git-tree": "3309ec82cd96d752ff890c441cb20ef49b52bf94",
"version-string": "1.2.8",
"port-version": 0
}
]
}
```
For each port, its corresponding versions file should be located in `versions/{first letter of port name}-/{port name}.json`. For example, zlib’s version file will be located in `versions/z-/zlib.json`. Aside from port version files, the current baseline file is located in `versions/baseline.json`.

2
docs/users/versioning.md
View file

@ -5,6 +5,8 @@
Versioning allows you to deterministically control the precise revisions of dependencies used by
your project from within your manifest file.
See our guide to [getting started with versioning](versioning.getting-started.md).
## Version schemes
### Schemes

228
docs/users/versioning.reference.md Normal file
View file

@ -0,0 +1,228 @@
# Versioning reference
## Contents
* [Version schemes](#version-schemes)
* [Version files](#version-files)
* [Version constraints](#version-constraints)
## Version schemes
Ports in vcpkg should attempt to follow the versioning conventions used by the package’s authors. For that reason, when declaring a package’s version the appropriate scheme should be used.
Each versioning scheme defines its own rules on what is a valid version string and more importantly the rules for how to sort versions using the same scheme.
The versioning schemes understood by vcpkg are:
Manifest property | Versioning scheme
------------------|------------------------------------
`version` | For dot-separated numeric versions
`version-semver` | For SemVer compliant versions
`version-date` | For dates in the format YYYY-MM-DD
`version-string` | For arbitrary strings
A manifest must contain only one version declaration.
#### `version`
Accepts version strings that follow a relaxed, dot-separated-, semver-like scheme.
The version is logically composed of dot-separated (`.`) numeric sections. Each section must contain an integer positive number with no leading zeroes.
The regex pattern for this versioning scheme is: `(0|[1-9]\d*)(\.(0|[1-9]\d*))*`
_Sorting behavior_: When comparing two versions, each section is compared from left to right by their numeric value, until the first difference is found. A version with the smallest set of sections takes precedence over another with a larger set of sections, given that all their preceding sections compare equally.
Example:
`0` < `0.1` < `0.1.0` < `1` < `1.0.0` < `1.0.1` < `1.1`< `2.0.0`
#### `version-semver`
Accepts version strings that follow semantic versioning conventions as described in the [semantic versioning specification](https://semver.org/#semantic-versioning-specification-semver).
_Sorting behavior_: Strings are sorted following the rules described in the semantic versioning specification.
Example:
`1.0.0-1` < `1.0.0-alpha` < `1.0.0-beta` < `1.0.0` < `1.0.1` < `1.1.0`
#### `version-date`
Accepts version strings that can be parsed to a date following the ISO-8601 format `YYYY-MM-DD`. Disambiguation identifiers are allowed in the form of dot-separated-, positive-, integer-numbers with no leading zeroes.
The regex pattern for this versioning scheme is: `\d{4}-\d{2}-\d{2}(\.(0|[1-9]\d*))*`
_Sorting behavior_: Strings are sorted first by their date part, then by numeric comparison of their disambiguation identifiers. Disambiguation identifiers follow the rules of the relaxed (`version`) scheme.
Examples:
`2021-01-01` < `2021-01-01.1` < `2021-02-01.1.2` < `2021-02-01.1.3` < `2021-02-01`
#### `version-string`
For packages using version strings that do not fit any of the other schemes, it accepts most arbitrary strings. The `#` which is used to denote port versions is disallowed.
_Sorting behavior_: No sorting is attempted on the version string itself. However, if the strings match exactly, their port versions can be compared and sorted.
Examples:
* `apple` <> `orange` <> `orange.2` <> `orange2`
* `watermelon#0`< `watermelon#1`
#### `port-version`
A positive integer value that increases each time a vcpkg-specific change is made to the port.
The rules for port versions are:
* Start at 0 for the original version of the port,
* increase by 1 each time a vcpkg-specific change is made to the port that does not increase the version of the package,
* and reset to 0 each time the version of the package is updated.
_NOTE: Whenever vcpkg output a version it follows the format `<version>#<port version>`. For example `1.2.0#2` means version `1.2.0` port version `2`. When the port version is `0` the `#0` suffix is omitted (`1.2.0` implies version `1.2.0` port version `0`)._
_Sorting behavior_: If two versions compare equally, their port versions are compared by their numeric value, lower port versions take precedence.
Examples:
* `1.2.0` < `1.2.0#1` < `1.2.0#2` < `1.2.0#10`
* `2021-01-01#20` < `2021-01-01.1`
* `windows#7` < `windows#8`
## Version files
Vcpkg uses a set of metadata files to power its versioning feature.
These files are located in the following locations:
* `${VCPKG_ROOT}/versions/baseline.json`, (this file is common to all ports) and
* `${VCPKG_ROOT}/versions/${first-letter-of-portname}-/${portname}.json` (one per port).
For example, for `zlib` the relevant files are:
* `${VCPKG_ROOT}/versions/baseline.json`
* `${VCPKG_ROOT}/versions/z-/zlib.json`
CI checks validate that each time a port is added or updated, its respective version files are also updated.
### Baseline file
The baseline file located in `${VCPKG_ROOT}/versions/baseline.json` is used to declared the current baseline versions of all packages.
A small section of the contents of this file is presented below:
```
{
"default": {
"3fd": { "baseline": "2.6.3", "port-version": 0 },
"7zip": { "baseline": "19.00", "port-version": 2 },
"abseil": { "baseline": "2020-09-23", "port-version": 1 }
...
}
}
```
Provided that there are no local modifications to the ports, the versions of all packages in the baseline file should map to the version of their corresponding portfiles in the `ports/` directory.
### Versions file
Each port in vcpkg has a corresponding versions file, the location of a port's versions file follows the pattern:
```
${VCPKG_ROOT}/versions/${first-letter-of-portname}-/${portname}.json
````
For example, for `zlib` the corresponding versions file is:
```
${VCPKG_ROOT}/versions/z-/zlib.json
```
These files contain an array of all the versions available for a given port.
For example, the contents of `versions/z-/zlib.json` declare the following versions:
```
{
"versions": [
{
"git-tree": "827111046e37c98153d9d82bb6fa4183b6d728e4",
"version-string": "1.2.11",
"port-version": 9
},
{
"git-tree": "068430e3e24fa228c302c808ba99f8a48d126557",
"version-string": "1.2.11",
"port-version": 8
},
...
]
}
```
Each version declared in this file uses the same syntax used in manifest files, but adds an extra `git-tree` property. The value of `git-tree` is the SHA hash, as calculated by Git, of the directory containing the portfiles for the declared version.
### Updating the version files
The recommended method to update these files is to run the `x-add-version` command.
For example, if you have made changes to `zlib`:
```
vcpkg x-add-version zlib
```
If you're updating multiple ports at the same time, instead you can run:
```
vcpkg x-add-version --all
```
To update the files for all modified ports at once.
_NOTE: These commands require you to have committed your changes to the ports before running them. The reason is that the Git SHA of the port directory is required in these version files. But don't worry, the `x-add-version` command will warn you if you have local changes that haven't been committed._
## Version constraints
### `builtin-baseline`
Accepts a Git commit ID. Vcpkg will try to find a baseline file in the given commit ID and use that to set the baseline versions (lower bound versions) of all declared dependencies.
When resolving version constraints for a package, vcpkg will look for a baseline version:
* First by looking at the baseline file in the given commit ID.
* If the given commit ID does not contain a baseline file, vcpkg will fallback to use the local baseline file instead.
* If there’s no local baseline file, vcpkg will use the version currently available in the ports directory.
_NOTE: If a baseline file is found, but it does not contain an entry for the package, the vcpkg invocation will fail._
Example:
```
{
"name": "project",
"version": "1.0.0",
"dependencies": ["zlib", "fmt"],
"builtin-baseline":"9fd3bd594f41afb8747e20f6ac9619f26f333cbe"
}
```
Baselines can be used without any other version constraints to obtain behavior close to using “classic” mode.
### `version>=`
Expresses a minimum version requirement, `version>=` declarations put a lower boundary on the versions that can be used to satisfy a dependency.
Example:
```
{
"name": "project",
"version-semver": "1.0.0",
"dependencies": [
{ "name": "zlib", "version>=": "1.2.11#9" },
{ "name": "fmt", "version>=": "7.1.3" }
],
"builtin-baseline":"9fd3bd594f41afb8747e20f6ac9619f26f333cbe"
}
```
As part of a version constraint declaration, a port version can be specified by adding the suffix `#<port-veresion>`, in the previous example `1.2.11#9` refers to version `1.2.11` port version `9`.
### `overrides`
Declaring an override forces vcpkg to ignore all other version constraints, both top-level and transitive constraints, and use the version specified in the override. This is useful for pinning exact versions and for resolving version conflicts.
Overrides are declared as an array of package version declarations.
For an override to take effect, the overridden package must form part of the dependency graph. That means that a dependency must be declared either by the top-level manifest or be part of a transitive dependency.
```
{
"name": "project",
"version-semver": "1.0.0",
"dependencies": [
{ "name": "zlib", "version>=": "1.2.11#9" },
"fmt"
],
"builtin-baseline":"9fd3bd594f41afb8747e20f6ac9619f26f333cbe",
"overrides": [
{ "name": "fmt", "version": "6.0.0" }
]
}
```