This PEP proposes a new file format for specifying dependencies to enable reproducible installation in a Python environment. The format is designed to be human-readable and machine-generated. Installers consuming the file should be able to calculate what to install without the need for dependency resolution at install-time.
Currently, no standard exists to create an immutable record, such as a lock file, which specifies what direct and indirect dependencies should be installed into a virtual environment.
Considering there are at least five well-known solutions to this problem in the community (PDM, pip freeze, pip-tools, Poetry, and uv), there seems to be an appetite for lock files in general.
Those tools also vary in what locking scenarios they support. For instance, pip freeze and pip-tools only generate single-use lock files for the current environment while PDM, Poetry, and uv can/try to lock for multiple environments and use-cases at once. There’s also concerns around the lack of secure defaults in the face of supply chain attacks by some tools (e.g. including hashes for files).
The lack of a standard also has some drawbacks. For instance, any tooling that wants to work with lock files must choose which format to support, potentially leaving users unsupported (e.g. Dependabot only supporting select tools, same for cloud providers who can do dependency installations on your behalf, etc.). It also impacts portability between tools, which causes vendor lock-in. By not having compatibility and interoperability it fractures tooling around lock files where both users and tools have to choose what lock file format to use upfront, making it costly to use/switch to other formats (e.g. tooling around auditing a lock file). Rallying around a single format removes this cost/barrier.
The closest the community has to a standard are pip’s requirements files which all the aforementioned tools either use directly as their file format or export to (i.e. requirements.txt). Unfortunately, the format is not a standard but is supported by convention. It’s also designed very much for pip’s needs, limiting its flexibility and ease of use (e.g. it’s a bespoke file format). Lastly, it is not secure by default (e.g. file hash support is entirely an opt-in feature, you have to tell pip to not look for other dependencies outside of what’s in the requirements file, etc.).
Note
Much of the motivation from PEP 665 also applies to this PEP.
The file format proposed by this PEP is designed to be human-readable. This is so that the contents of the file can be audited by a human to make sure no undesired dependencies end up being included in the lock file.
The file format is also designed to not require a resolver at install time. This greatly simplifies reasoning about what would be installed when consuming a lock file. It should also lead to faster installs which are much more frequent than creating a lock file.
The data in the file should be consumable by tools not written in Python. This allows for e.g. cloud hosting providers to write their own tool to perform installations in their preferred programming language. This introduces the concept of lockers which write the lock file, and installers which install from a lock file (they can be the same tool).
The file format should promote good security defaults. As the format is not meant to be human-writable, this means having tools provide security-related details is reasonable and not a costly burden.
The contents of a lock file should be able to replace the vast majority of uses of requirements files when used as a lock file (e.g. what pip-tools and pip freeze emit). This means the file format specified by this PEP can, at minimum, act as an export target for tools which have their own internal lock file format.
Lock files can be single-use and multi-use. Single-use lock files are things like requirements.txt files, which serve a single use-case/purpose (hence why it isn’t uncommon for a project to have multiple requirements files, each for a different use-case). Multi-use lock files represent multiple use-cases within a single file, often expressed through extras and Dependency Groups. As such, this PEP supports additions to Environment Markers that allows for specifying extras and dependency groups as appropriate. This allows for a single lock file to support those cases. This not only cuts down on the number of potential lock files, but it also makes it easier when a single package is meant to be consistent across all use-cases (with multiple single-use lock files it requires coordinating across multiple lock files). This support also means that this PEP supports all package installation-related data that can be recorded in a pyproject.toml file. The hope is that this support will allow some tools to drop their internal lock file entirely and solely rely on what this PEP specifies.
A lock file MUST be named pylock.toml or match the regular expression r"^pylock\.([^.]+)\.toml$" if a name for the lock file is desired or if multiple lock files exist. The use of the .toml file extension is to make syntax highlighting in editors easier and to reinforce the fact that the file format is meant to be human-readable. The prefix and suffix of a named file MUST be lowercase when possible, for easy detection and removal, e.g.:
The expectation is that services that automatically install from lock files will search for:
E.g. a cloud host service named “spam” would first look for pylock.spam.toml to install from, and if that file didn’t exist then install from pylock.toml and look for a dependency group named “spam” to use if present.
The lock file(s) SHOULD be located in the directory as appropriate for the scope of the lock file. Locking against a single pyproject.toml, for instance, would place the pylock.toml in the same directory. If the lock file covered multiple projects in a monorepo, then the expectation is the pylock.toml file would be in the directory that held all the projects being locked.
The format of the file is TOML.
Tools SHOULD write their lock files in a consistent way to minimize noise in diff output. Keys in tables – including the top-level table – SHOULD be recorded in a consistent order (if inspiration is desired, this PEP has tried to write down keys in a logical order). As well, tools SHOULD sort arrays in consistent order. Usage of inline tables SHOULD also be kept consistent.
See packages.vcs.subdirectory.
[packages.archive]See packages.vcs.url.
packages.archive.pathSee packages.vcs.path.
packages.archive.sizeSee packages.vcs.subdirectory.
packages.indexSee packages.archive.upload-time.
packages.sdist.urlSee packages.archive.url.
packages.sdist.pathSee packages.archive.path.
packages.sdist.sizeSee packages.archive.size.
packages.sdist.hashesSee packages.archive.hashes.
[[packages.wheels]]See packages.archive.upload-time.
packages.wheels.urlSee packages.archive.url.
packages.wheels.pathSee packages.archive.path.
packages.wheels.sizeSee packages.archive.size.
packages.wheels.hashesSee packages.archive.hashes.
[[packages.attestation-identities]]This PEP proposes adding to the Environment Markers specification such that extras and dependency group relationships for an entry in [[packages]] can be expressed in packages.marker. The additions outlined in this PEP ONLY apply in the context of lock files as defined by this PEP and not in other contexts where marker syntax is used (e.g. METADATA, pyproject.toml).
First, two new markers will be introduced: extras and dependency_groups. They represent the extras and dependency groups that have been requested to be installed, respectively:
This does NOT preclude using the same syntax parser in other contexts, only where the new markers happen to be considered valid based on context.
Second, the marker specification will be changed to allow sets for values (on top of the current support for strings and versions). ONLY the new markers introduced in this PEP will allow for a set for their value (which defaults to an empty set). This specifically does NOT update the spec to allow for set literals.
Third, the marker expression syntax specification will be updated to allow for operations involving sets:
Fourth, use of extras and dependency_groups will be considered an error outside of lock file (much like extra outside of Core metadata specifications).
These changes, along with packages.extras/ packages.dependency-groups and marker expressions’ Boolean logic support, allow for expressing arbitrary, exhaustive requirements for when a package should be installed based on the extras and dependency groups (not) requested. For example, if a lock file has extras = ["extra-1", "extra-2"], you can specify if a package should be installed when:
(This list is not exhaustive of all possible Boolean logic expressions.)
The same flexibility applies to dependency groups.
How users tell a tool what extras and/or dependency groups they want installed is up to the tool. Installers MUST support the marker expression syntax additions as proposed by this PEP. Lockers MAY support writing lock files that utilize the proposed marker expression syntax additions (i.e. lockers can choose to only support writing single-use lock files).
The following outlines the steps to be taken to install from a lock file (while the requirements are prescriptive, the general steps and order are a suggestion):
Ignoring formatting, there are a few differences between lock files as proposed by this PEP and those that are possible via a requirements file.
Some of the differences are in regards to security. Requiring hashes, recording file sizes, and where a file was found – both the index and the location of the file itself – help with auditing and validating the files that were locked against. Compare that with requirements files which can optionally include hashes, but it is an opt-in feature and can be bypassed. The optional inclusion of a file’s upload time and where the files can be found is also different.
Being explicit about the supported Python versions and environments for the file overall is also unique to this PEP. This is to alleviate the issue of not knowing when a requirements file targets a specific platform.
The [tool] tables don’t have a direct correlation in requirements files. They do support comments, but they are not inherently structured like the [tool] table is thanks to being in TOML.
While comments in a requirements file could record details that are helpful for auditing and understanding what the lock file contains, providing the structured support to record such things makes auditing easier. Recording the required Python version for a package upfront helps with this as well as erroring out sooner if an install is going to fail. Recording the wheel file name separate from the URL or path is also to help make reading the list of wheel files easier as it encodes information that can be useful when understanding and auditing a file. Recording the sdist file name is for the same reason.
This PEP supports multi-use lock files while requirements files are single-use.
This PEP does NOT fully replace requirements files because:
Because there is no preexisting lock file format, there are no explicit backwards-compatibility concerns in terms of Python packaging standards.
As for packaging tools themselves, that will be a per-tool decision as to whether they choose to support this PEP and in what way (i.e. as an export target or as the primary way they record their lock file).
The hope is that by standardizing on a lock file format which starts from a security-first posture it will help make overall packaging installation safer. However, this PEP does not solve all potential security concerns.
One potential concern is tampering with a lock file. If a lock file is not kept in source control and properly audited, a bad actor could change the file in nefarious ways (e.g., point to a malware version of a package). Tampering could also occur in transit to e.g. a cloud provider who will perform an installation on the user’s behalf. Both could be mitigated by signing the lock file either within the file in a [tool] entry or via a side channel external to the lock file itself.
This PEP does not do anything to prevent a user from installing incorrect packages. While including many details to help in auditing a package’s inclusion, there isn’t any mechanism to stop e.g. name confusion attacks via typosquatting. Tools may be able to provide some UX to help with this (e.g. by providing download counts for a package).
Users should be informed that when they ask to install some package, the package may have its own dependencies, those dependencies may have dependencies, and so on. Without writing down what gets installed as part of installing the package they requested, things could change from underneath them (e.g. package versions). Changes to the underlying dependencies can lead to accidental breakage of their code. Lock files help deal with that by providing a way to write down what was installed so you can install the exact same thing in the future.
Having what to install written down also helps in collaborating with others. By agreeing to a lock file’s contents, everyone ends up with the same packages installed. This helps make sure no one relies on e.g. an API that’s only available in a certain version that not everyone working on the project has installed.
Lock files also help with security by making sure you always get the same files installed and not a malicious one that someone may have slipped in. It also lets one be more deliberate in upgrading their dependencies and thus making sure the change is on purpose and not one slipped in by a bad actor.
Lock files can support only certain environments. What must be installed for the environment they are installing for may be different compared to another, different environment. Some lock files do try to be universal, though, and work for any possible environment (sdist and source tree installation success permitting).
Lock files can be single-use or multi-use. Single-use lock files are for single use-cases. Multi-use lock files can be used for multiple use-cases based on extras and dependency groups. It is up to the tool(s) you use that decide whether multi-use lock files are possible. All tools dealing with lock files at least support single-use lock files. Neither type of lock file is better or worse than the other, it just changes how much can be written down in a single file.
Lock files that follow this PEP can be installed by any installer that implements the specification. This allows users of a lock file to perform an installation to not be tied to the locker used by the person who produced the lock file. But it is not the case that using a different locker will lead to the same result. This could be for various reasons, including using different algorithms to determine what to lock.
A proof-of-concept implementing single-use lock files can be found at https://github.com/brettcannon/mousebender/tree/pep . Other tools such as PDM and Poetry implement semantically similar approaches of various parts of this PEP.
A previous version of this PEP recorded the dependency graph of packages instead of a set of packages to install. The idea was that by recording the dependency graph you not only got more information, but it provided more flexibility by supporting more features innately (e.g. platform-specific dependencies without explicitly propagating markers).
In the end, though, it was deemed to add complexity that wasn’t worth the cost (e.g. it impacted the ease of auditing for details which were not necessary for this PEP to reach its goals).
At one point, to handle the issue of metadata varying between files and thus require examining every released file for a package and version for accurate locking results, the idea was floated to introduce a new core metadata version which would require all metadata for all wheel files be the same for a single version of a package. Ultimately, though, it was deemed unnecessary as this PEP will put pressure on people to make files consistent for performance reasons or to make indexes provide all the metadata separate from the wheel files themselves. As well, there’s no easy enforcement mechanism, and so community expectation would work as well as a new metadata version.
In order to support a format more akin to how Poetry worked when this PEP was drafted, it was suggested that lockers effectively record the packages and their versions which may be necessary to make an install work in any possible scenario, and then the installer resolves what to install. But that complicates auditing a lock file by requiring much more mental effort to know what packages may be installed in any given scenario. Also, one of the Poetry developers suggested that markers as represented in the package locking approach of this PEP may be sufficient to cover the needs of Poetry. Not having the installer do a resolution also simplifies their implementation, centralizing complexity in lockers.
It was proposed to require a baseline hash algorithm for the files. This was rejected as no other Python packaging specification requires specific hash algorithm support. As well, the minimum hash algorithm suggested may eventually become an outdated/unsafe suggestion, requiring further updates. In order to promote using the best algorithm at all times, no baseline is provided to avoid simply defaulting to the baseline in tools without considering the security ramifications of that hash algorithm.
It was proposed to put the pylock constant part of the file name after the identifier for the purpose of the lock file. It was decided not to do this so that lock files would sort together when looking at directory contents instead of purely based on their purpose which could spread them out in a directory.
Not using .toml as the file extension and instead making it .pylock itself was proposed. This was decided against so that code editors would know how to provide syntax highlighting to a lock file without having special knowledge about the file extension.
Having no requirements or guidance for a lock file’s name was considered, but ultimately rejected. By having a standardized naming convention it makes it easy to identify a lock file for both a human and a code editor. This helps facilitate discovery when e.g. a tool wants to know all of the lock files that are available.
Since having a format that is machine-writable was a goal of this PEP, it was suggested to use JSON. But it was deemed less human-readable than TOML while not improving on the machine-writable aspect enough to warrant the change.
Some argued that YAML met the machine-writable/human-readable requirement in a better way than TOML. But as that’s subjective and pyproject.toml already existed as the human-writable file used by Python packaging standards it was deemed more important to keep using TOML.
Earlier versions of this PEP proposed having a single hash algorithm be specified per file instead of any number of algorithms per file. The thinking was that by specifying a single algorithm it would help with auditing the file when a specific hash algorithm was mandated for use.
In the end there was some objection to this idea. Typically, it centered around the cost of rehashing large wheel files (e.g. PyTorch). There was also concern about making hashing decisions upfront on the installer’s behalf which they may disagree with. In the end it was deemed better to have flexibility and let people audit the lock file as they see fit.
Hashing the contents of the bytes of the file and storing hash value within the file itself was proposed at some point. This was removed to make it easier when merging changes to the lock file as each merge would have to recalculate the hash value to avoid a merge conflict.
Hashing the semantic contents of the file was also proposed, but it would lead to the same merge conflict issue.
Regardless of which contents were hashed, either approach could have the hash value stored outside of the file if such a hash was desired.
To know how potentially stale the lock file was, an earlier proposal suggested recording the creation date of the lock file. But for the same merge conflict reasons as storing the hash of the file contents, this idea was dropped.
Recording what package indexes were used to create the lock file was considered. In the end, though, it was rejected as it was deemed unnecessary bookkeeping.
An earlier version of this PEP tried to lock the build requirements for sdists under a packages.build-requires key. Unfortunately, it confused enough people about how it was expected to operate and there were enough edge case issues to decide it wasn’t worth trying to do in this PEP upfront. Instead, a future PEP could propose a solution.
Earlier versions had a dedicated packages.direct key to flag when something should be considered as originating from a direct URL reference. But there are explicitly only three cases where a direct URL reference can occur (VCS, directory, and archive). Since all three cases are explicit in [[packages]], the setting of the key was technically superfluous.
The single drawback of not having this key is for wheels and sdists that now fall under packages.archive. With the separate key (or having it specified as a part of packages.sdist or packages.wheels), it would allow for identifying in the lock file itself that an archive file is an sdist or wheel. As it currently stands, an installer must infer that detail itself.
The package version is optional since it can only be reliably recorded when an sdist or wheel file is used. And since both sources record the version in file names it is technically redundant.
But in discussions it was decided the version number is useful for auditing enough to still state it separately.
At least one person has commented how their work has unstable URLs for all sdists and wheels. As such, they have to search for all files at install regardless of where the file was found previously. Dropping the requirement to provide the URL or path to a file would have helped solve the issue of recording known-bad information.
The decision to allow tools to look for a file in other ways beyond the URL provided alleviated the need to make the URL optional.
At least one person has said that their work modifies all wheels and sdists with internal files. That means any recorded hashes and file sizes will be wrong. By making the file size and hashes optional – very likely through some opt-out mechanism – then they could continue to produce lock files that meet this PEP’s requirements.
The decision was made that this weakens security too much. It also prevents installing files from alternative locations.
While incompatible with dropping the URL/path requirement, the package version, and hashes, recording the sdist file name is technically not necessary at all (right now recording the file name is optional). The file name only encodes the project name and version, so no new info is conveyed about the file (when the package version is provided). And if the location is recorded then getting the file is handled regardless of the file name.
But recording the file name can be helpful when looking for an appropriate file when the recorded file location is no longer available (while sdist file names are now standardized thanks to PEP 625, that has only been true since 2020 and thus there are many older sdists with names that may not be guessable).
The decision was made to require the sdist file name out of simplicity.
One could see writing out wheel file details as a table keyed on the file name. For example:
In general, though, people did not prefer this over the approach this PEP has taken.
The [tool] table is included as it has been found to be very useful for pyproject.toml files. Providing similar flexibility to this PEP is done in hopes that similar benefits will materialize.
But some people have been concerned that such a table will be too enticing to tools and will lead to files that are tool-specific and unusable by other tools. This could cause issues for tools trying to do installation, auditing, etc. as they would not know what details in the [tool] table are somehow critical.
As a compromise, this PEP specifies that the details recorded in [tool] must be disposable and not affect installation of packages.
Right now the file does not record the requirements that acted as inputs to the file. This is for simplicity reasons and to not explicitly constrain the file in some unforeseen way (e.g., updating the file after initial creation for a new platform that has different requirements, all without having to resolve how to write a comprehensive set of requirements).
But it may help in auditing and any recreation of the file if the original requirements were somehow recorded. This could be a single string or an array of strings if multiple requirements were used with the file.
In the end it was deemed too complicated to try and capture the inputs that a tool used to construct the lock file in a generic fashion.
Recording the dependents of a package is not necessary to install it. As such, it has been left out of the PEP as it can be included via [tool].
But knowing how critical a package is to other packages may be beneficial. This information is included by pip-tools , so there’s prior art in including it. A flexible approach could be used to record the dependents, e.g. as much detail as to differentiate from any other entry for the same package in the file (inspired by uv).
In the end, though, it was decided that recording the dependencies is a better thing to record.
Thanks to everyone who participated in the discussions on discuss.python.org. Also thanks to Randy Döring, Seth Michael Larson, Paul Moore, and Ofek Lev for providing feedback on a draft version of this PEP before going public. Thanks to Randy Döring of Poetry, Charlie Marsh of uv, and Frost Ming of PDM for providing feedback on behalf of their respective projects.
This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.
Source: https://github.com/python/peps/blob/main/peps/pep-0751.rst
Last modified: 2026-03-30 01:01:49 UTC