It's worth noting that uv also supports a workflow that directly replaces pyenv, virtualenv and pip without mandating a change to a lockfile/pyproject.toml approach.
uv python pin <version> will create a .python-version file in the current directory.
uv virtualenv will download the version of Python specified in your .python-version file (like pyenv install) and create a virtualenv in the current directory called .venv using that version of Python (like pyenv exec python -m venv .venv)
uv pip install -r requirements.txt will behave the same as .venv/bin/pip install -r requirements.txt.
uv run <command> will run the command in the virtualenv and will also expose any env vars specified in a .env file (although be careful of precedence issues: https://github.com/astral-sh/uv/issues/9465)
uv and its flexibility is an a absolute marvel. Where pip took 10 minutes, uv can handle it in 20-30s.
It’s an absolute godsend. I thought poetry was a nice improvement but it had its flaws as well (constant merge conflicts in the lock file in particular).
Uv works more or less the same as I’m used to with other tooling in Ruby, JS, Rust, etc.
+1, this is the exact reason I started using uv. Extremely convenient.
For some reason uv pip has been very slow, however. Unsure why, might be my org doing weird network stuff.
Or very difficult package spec
Doesn't it store the python version in the pyproject.toml though, is the python version file needed?
It’s not:
> uv will respect Python requirements defined in requires-python in the pyproject.toml file during project command invocations. The first Python version that is compatible with the requirement will be used, unless a version is otherwise requested, e.g., via a .python-version file or the --python flag.
— https://docs.astral.sh/uv/concepts/python-versions/#project-...
cheers
# Ensure we always have an up to date lock file.
if ! test -f uv.lock || ! uv lock --check 2>/dev/null; then
uv lock
fi
Hi author here.
If you end up with an invalid lock file, it doesn't silently fail and move on with a generated lock file. The `uv lock` file will fail with a helpful message and then errexit from the shell script kicks in.
For example, I made my lock file invalid by manually switching one of the dependencies to a version that doesn't match the expected SHA.
Then I ran the same script you partially quoted and it yields this error which blocks the build and gives a meaningful message that a human can react to:
1.712 Using CPython 3.13.3 interpreter at: /usr/local/bin/python3
1.716 error: Failed to parse `uv.lock`
1.716 Caused by: The entry for package `amqp` v5.3.4 has wheel `amqp-5.3.1-py3-none-any.whl` with inconsistent version: v5.3.1
------
failed to solve: process "/bin/sh -c chmod 0755 bin/* && bin/uv-install" did not complete successfully: exit code: 2
As for a missing lock file, yep it will generate one but we want that. The expectation there is we have nothing to base things off of, so let's generate a fresh one and use it moving forward. The human expectation in a majority of the cases is to generate one in this spot and then you can commit it so moving forward one exists.
This is actually covered by the --locked option that uv sync provides.
If you do `uv sync --locked` it will not succeed if the lock file does not exist or is out of date.
Edit: I slightly misread your comment. I strongly agree that having no lock file or a lockfile that does not match your specified dependencies is a case where a human should intervene. That's why I suggest you should always use the --locked option in your build.
Yes this is a major bug in the process. I came to the comments to say this as well.
They say this but do the exact opposite as you point out:
> The --frozen flag ensures the lock file doesn’t get updated. That’s exactly what we want because we expect the lock file to have a complete list of exact versions we want to use for all dependencies that get installed.
It's not a major bug, check my reply in: https://news.ycombinator.com/edit?id=44370311
In the Python world, I often see lockfiles treated a one "weird step in the installation process", and not committed to version control.
In my experience, this is fundamentally untrue. pip-tools has extensive support for recording the explicit version numbers, package hashes and whatnot directly in the requirements.txt based on requirements.in and constraints files.
There are many projects that use pip-compile to lock things down. You couldn’t use python in a regulated environment if you didn’t. I’ve written many Makefiles that explicitly forbid CI from ever creating or updating the actual requirements.txt. It has to be reviewed by a human, or more.
There are lots of tools that allow you to generate what are essentially lock files. But I think what the previous poster is saying is that most people either don't use these tools or don't use them correctly. That certainly matches my experience, where I've seen some quite complicated projects get put into production without any sort of dependency locking whatsoever - and where I've also seen the consequences of that where random dependencies have upgraded and broken everything and it's been almost impossible to figure out why.
To me, one of the big advantages of UV (and similar tools) is that they make locked dependencies the default, rather than something you need to learn about and opt into. These sorts of better defaults are sorely needed in the Python ecosystem.
They're not saying that's how it's supposed to be used, they're saying that's how it's often used by people who are unfamiliar with lock files
In the almost every world, Ruby and elsewhere too, constraints in library package metadata are supposed to express the full supported possibilities of allowed constraints while lock files represent current specific state. That's why they're not committed in that case to allow greater flexibility/interoperability for downstream users.
For applications, it's recommended (but still optional) to commit lock files so that very specific and consistent dependencies are maintained to prevent arbitrary, unsupervised package upgrades leading to breakage.
I know Cargo recommended your approach for a while, but ended up recommending that all projects always check in a lock file. This is also the norm in most other ecosystems I've used including Javascript and other Python package managers.
When you're developing a library, you still want consistent, reproducible dependency installs. You don't want, for example, a random upgrade to a testing library to break your CI pipelines or cause delays while releasing. So you check in the lock file for the people working on the library.
But when someone installs the library via a package manager, that package manager will ignore the lock file and just use the constraints in the package metadata. This avoids any interoperability issues for downstream users.
I've heard of setups where there are even multiple lock files checked in so different combinations of dependency can be tested in CI, but I've not seen that in practice, and I imagine it's very much dependent on how the ecosystem as a whole operates.
This is kinda how I treat it. I figured that I have already set the requirements in the pyproject.toml file.
Should I be committing the lock file?
If your pyproject.toml does not list all your dependencies (including dependencies of your dependencies) and a fixed version for each, you may get different versions of the dependencies in future installs.
A lock file ensures all installations resolve the same versions, and the environment doesn’t differ simply because installations were made on different dates. Which is usually what you want for an application running in production.
It's what I used to do with package-lock.json when I had little production experience.
What are the possible remediation steps, however? If there is no lock file at all, this is likely the first run, or it will be overwritten from a git upstream later on anyway; if it's broken, chances are high someone messed up a package installation and creating a fresh lock file seems like the only sensible thing to do.
I also feel like this handles rare edge cases, but it seems like a pretty straightforward way to do so.
If there's no lock file at all, you haven't locked your dependencies, and you should just install whatever is current (don't create a lockfile). If it's broken, you have problems, and you need to abort the deploy.
There is never a reason for an automated system to create a lockfile.
The reason is simple: it allows you to do the install using "sync" in all cases, whether the lockfile exists or not.
Where the lockfile doesn't exist, it creates it from whatever current is, and the lockfile then gets thrown away later. So it's equivalent to what you're saying, it just avoids having two completely separate install paths. I think it's the correct approach.
I don't understand, you can already run `uv sync` if the lockfile doesn't exist. It just creates a new one. Why do it explicitly, like here?
IMO, this is the process for building an application image for deployment to production. If the lock file is not present, then the developer has done something wrong and the deployment should fail catastrophically because only manual intervention by the developer can fix it correctly.
If the lock file is missing the only sensible thing to do is require human intervention. Either it’s the unusual case of somebody initialising a project but never syncing it, or something has gone seriously wrong – with potential security implications. The upside to automating this is negligible and the downside is large.
? It has always been the case that if you don't specify a version, the latest is implied.
Whether it’s the latest or not is irrelevant. What’s important is the actual package hash. This is the only way to have fully reproducible builds that are immune to poison-the-well attacks.
That would be true if anyone actually ever reviewed the dependencies. Which is not the case. So the version doesn't matter when any version is as likely to contain malware.
The fix is to generate the lockfile and commit it to the repository. Every build should be based on the untouched lockfile from the repo. It's the entire point of it.
I am totally against Python tooling being written in a language other than Python. I get that C extensions exist and for the most part Python is synonymous with CPython.
I think 2 languages are enough, we don't need a 3rd one that nobody asked for.
I have nothing against Rust. If you want a new tool, go for it. If you want a re-write of an existing tool, go for it. I'm against it creeping into an existing eco-system for no reason.
A popular Python package called Pendulum went over 7 months without support for 3.13. I have to imagine this is because nobody in the Python community knew enough Rust to fix it. Had the native portion of Pendulum been written in C I would have fixed it myself.
https://github.com/python-pendulum/pendulum/issues/844
In my ideal world if someone wanted fast datetimes written in Rust (or any other language other than C) they'd write a proper library suitable for any language to consume over FFI.
So far this Rust stuff has left a bad taste in my mouth and I don't blame the Linux community for being resistant.
I appreciate this perspective, but I think building a tool like uv in Rust is a good idea because it's a tool for managing Python stuff, not a tool to be called from within Python code.
Having your python management tools also be written in python creates a chicken-and-egg situation. Now you have to have a working python install before you can start your python management tool, which you are presumably using because it's superior to managing python stuff any other way. Then you get a bunch of extra complex questions like, what python version and specific executable is this management tool using? Is the actual code you're running using the same or a different one? How about the dependency tree? What's managing the required python packages for the installation that the management tool is running in? How do you know that the code you're running is using its own completely independent package environment? What happens if it isn't, and there's a conflict between a package or version your app needs and what the management tool needs? How do you debug and fix it if any of this stuff isn't actually working quite how you expected?
Having the management tool be a compiled binary you can just download and use, regardless of what language it was written in, blows up all of those tricky questions. Now the tool actually does manage everything about python usage on your system and you don't have to worry about using some separate toolchain to manage the tool itself and whether that tool potentially has any conflicts with the tool you actually wanted to use.
Python is my favorite language, but I have fully embraced uv. It’s so easy, and so fast, that there is nothing else remotely close.
Need modern Python on an ancient server running with EOL’d distro that no one will touch for fear of breaking everything? uv.
Need a dependency or two for a small script, and don’t want to hassle with packaging to share it? uv.
That said, I do somewhat agree with your take on extensions. I have a side project I’ve been working on for some years, which started as pure Python. I used it as a way to teach myself Python’s slow spots, and how to work around them. Then I started writing the more intensive parts in C, and used ctypes to interface. Then I rewrote them using the Python API. I eventually wrote so much of it in C that I asked myself why I didn’t just write all of it in C, to which my answer was “because I’m not good enough at C to trust myself to not blow it up,” so now I’m slowly rewriting it in Rust, mostly to learn Rust. That was a long-winded way to say that I think if your external library functions start eclipsing the core Python code, that’s probably a sign you should write the entire thing in the other language.
> I am totally against Python tooling being written in a language other than Python
I will be out enjoying the sunshine while you are waiting for your Pylint execution to finish
Linting is the new "compiling!"
Linting and type checking are very CPU intensive tasks so I would excuse anyone implementing those types of tools in $LANG where using all CPU juice matters.
I can't help but think uv is fast not because it's written in Rust but because it's a fast reimplementation. Dependency solving in the average Python project is hardly computationally expensive, it's just downloading and unpacking packages with a "global" package cache. I don't see why uv couldn't have been implemented in Python and be 95% as fast.
Edit: Except implementing uv in Python requires shipping a Python interpreter kinda defeating some of it's purpose of being a package manager able to install Python as well.
You also have to factor in startup time and concurrency. Caching an SAT solvers can't get python to 95% of uv.
>I am totally against Python tooling being written in a language other than Python. I get that C extensions exist and for the most part Python is synonymous with CPython.
>I think 2 languages are enough, we don't need a 3rd one that nobody asked for.
Enough for what? The uv users dont have to deal with that. Most ecosystems use a mix of language for tooling. It's not a detail the user of the tool has to worry about.
>I'm against it creeping into an existing eco-system for no reason.
It's much faster. Because its not written in Python.
The tooling is for the user. The language of the tooling is for the developer of the tooling. These dont need to be the same people.
The important thing is if the tool solves a real problem in the ecosystem (it does). Do people like it?
> I think 2 languages are enough, we don't need a 3rd one that nobody asked for.
Look at the number of stars ruff and uv got on github. That's a meteoric rise. So they were validated with ruff, and continued with uv, this we can call "was asked for".
> I'm against it creeping into an existing eco-system for no reason.
It's not no reason. A lot of other things have been tried. It's for big reasons: Good performance, and secondly independence from Python is a feature. When your python managing tool does not depend on Python itself, it simplifies some things.
I, on the other hand, don't care what language the tools are written in.
I do get the sentiment that a user of these tools, being a Python developer could in theory contribute to them.
But, if a tool does its job, I don't care if it's not "in Python". Moreover, I imagine there is a class of problems with the Python environment setup that'd break the tool that could help you fix it if the tool itself is written in Python.
It is well known, and not Python-specific, that using a different language/interpreter for development tools eliminates large classes of bootstrapping complications and conflicts.
If there are two versions of X, it becomes possible to use the wrong one.
If a tool to manage X depends on X, some of the changes that we would like the tool to perform are more difficult, imperfect or practically impossible.
Rust offers a feature-set that neither Python nor C has. If Rust is the right tool for the job, I would rather the code be written in Rust. Support has more to do with incentive structures than implementation language.
In theory, I can get behind what your saying, but in practice I just haven't found any package manager written in Python to be as good as uv, and I'm not even talking about speed. uv as I like it could be written in Python, but it hasn't been
I really dig rye, have you tried that?
rye is also written in Rust and it's being replaced by uv.
From its homepage: https://rye.astral.sh/
> If you're getting started with Rye, consider uv, the successor project from the same maintainers.
> While Rye is actively maintained, uv offers a more stable and feature-complete experience, and is the recommended choice for new projects.
> Having trouble migrating? Let us know what's missing.
It's also Rust.
What, exactly, is your objection to using rust (or any non-python/C language) for python tooling? You didn't actually give any reasons
I believe he alluded to it here...
"I have to imagine this is because nobody in the Python community knew enough Rust to fix it. Had the native portion of Pendulum been written in C I would have fixed it myself."
Correct. There better be a damn good reason to add another language to the ecosystem other than it's that particular developer's new favorite language.
Is there anything being done in uv that couldn't be done in Python?
How many people are digging into and contributing to any python tooling? How is C meaningfully more accessible than rust? Plenty of people (yet also a significant minority overall) write each of them.
> Is there anything being done in uv that couldn't be done in Python?
Speed, at the very least.
You could just ignore uv and use whatever you want...
> How is C meaningfully more accessible than rust
They've been teaching C in universities for like 40 years to every Computer Science and Engineering student. The number of professionally trained developers who know C compared to Rust is not even close. (And a lot of us are writing Python because it's easy and productive, not because we don't know other languages.)
If c + Python is so wonderful and so ubiquitous, why hasn't someone already created uv in C?
Ps the government and others have all recommended moving from C/C++ to Rust... It's irrelevant whether or not that's well-founded - it simply is.
And plenty of other cli tools have been successfully and popularly ported to Rust.
To quote Movie Mark Zuckerberg from The Social Network:
> If Python developers were the inventors of uv - they'd have invented uv
speed
I don't see any meaningful speedup. The 10x claims are not reproducible. He's also comparing it to the much older style of requirements.txt projects and not a poetry project with a lockfile.
I detailed this in another comment but pip (via requirements.txt): 8.1s, poetry: 3.7s, uv: 2.1s.
Not even 10x against pip and certainly not against poetry.
You must be holding it wrong, because everyone else raves about uv
Usually uv pip is only about x2 as fast as regular pip for me. Occasionally I'll have some combination of dependencies that will cause pip to take 2-5 minutes to resolve that uv will handle in 10-20 seconds.
They said "no meaningful speedup". 2x is meaningful
The impact of a 2x speedup is relative. For a quick test on one of my projects it's 10 seconds with pip and 4 seconds with uv. That's roughly in line with my previous testing. It's a nice minor speedup on average. It really shines when pip does some non-optimal resolving in the background that takes a minute or more.
How complex are the requirements for this project?
I see. I encourage you to try it with larger projects and see if it makes a difference.
That said, the speed is only one reason I use it. I find its ergonomics are the best in the Python tools I’ve tried. For example it has better dependency resolution than poetry in my estimation, and you can use the uv run —-with command to try things before adding them to your environment.
> I'm against it creeping into an existing eco-system for no reason.
There is a reason: tools that exist today are awful and unusable if you ever wrote anything other than python.
: I'm saying it because the only way I can see someone not realizing it is that they have never seen anything better.
Okay, maybe C and C++ have even worse tooling in some areas, but python is still the top language of having the worst tooling.
>I am totally against Python tooling being written in a language other than Python.
Cool story bro.
I'm totally against Python tooling being in dismal dissaray for 30 years I've been using the language, and if it takes some Rust projects to improve upon it, I'm all for it.
I also not rather have the chicken-and-egg dependency issue with Python tooling written in Python.
>A popular Python package called Pendulum went over 7 months without support for 3.13. I have to imagine this is because nobody in the Python community knew enough Rust to fix it. Had the native portion of Pendulum been written in C I would have fixed it myself.
Somehow the availability and wide knowledge of C didn't make anyone bother writing a datetime management lib in C and making it as popular. It took those Pendulum Rust coders.
And you could of course use pytz or dateutil or some other, but, no, you wanted to use the Rust-Python lib.
Well, when you start the project yourself, you get to decide what language it would be in.
you say "I'm against it creeping into an existing eco-system for no reason.", while you ignore that there is at least one good reason: A lot better performance.
The 10x performance wasn't mentioned in the article at all except the title.
I watched the video and he does mention it going from 30s to 3s when switching from a requirements.txt approach to a uv based approach. No comparison was done against poetry.
I am unable to reproduce these results.
I just copied his dependencies from the pyproject.toml file into a new poetry project. I ran `poetry install` from within Docker (to avoid using my local cache) `docker run --rm -it -v `pwd`:/work python:3.13 /bin/bash` and it took 3.7s
I did the same with an empty repo and a requirements.txt file and it took 8.1s.
I also did through `uv` and it took 2.1s.
Better performance?, sure. A lot better performence?, I can't say that with the numbers I got. 10x performance?... absolutely not.
Also, this isn't a major part of anybody's workflow. Docker builds happen typically on release. Maybe when running tests during CI/CD after the majority of work has been done locally.
I personally don’t care about the performance:
https://news.ycombinator.com/item?id=44359183
I agree it would be better if it was in Python but pypa did not step up, for decades! On the other hand, it is not powershell or ruby, it is a single deployed executable that works. I find that acceptable if not perfect.
Better performance than C? This is news to me
There are cases where single-threaded Rust and C are faster than each other, though usually only by single-digit percentages. But Rust is so much easier to parallelize than C that it isn't even funny.
According to the very link you provide, the sticking point was a dependency which does not use rust, and the maintainer probably being busy.
I updated a rust-implemented wheel to 3.13 compat myself and literally all that required was bumping pyo3 (which added support back in June) and adding the classifier. Afaik cryptography had no trouble either, iirc what they had to wait on was a 3.13 compatible cffi .
The PR which enabled 3.13 did have changes to Rust code.
Because they did more than just support 3.13:
> I'm sure some of the changes are going too far. We are open to revert them if there's an interest from maintainers to merge this PR :)
Notably they bumped the bindings (“O3”) for better architecture coverage, and that required some renaming as 0.23 completed an API migration.
Upvoting for interesting/important/sympathetic perspective, but am very much in disagreement
Offtopic, but thank you. I really wish this way of treating up/downvotes was more widespread. Down should mean it doesn't contribute to the conversation, not that you disagree with their opinion.
I love rust but I tend to agree, python tooling should be maintainable by the community without learning a new language.
However rust is a thousand times faster than python.
At the end, if you don't like it don't use it.
I had a situation, admittedly niche, where some git based package dependency wasn't being updated properly (tags vs. commit hashes) and thanks to poetry being written in Python I was able to quickly debug and solve the problem. I think it's more a matter of core functionality (that affects everyone) vs. more esoteric or particular use cases (like dataframe libraries) that make sense to FFI.
Did you even read the issue that you pointed to? It's not even the rust part that was the issue.
Or maybe the community will embrace Rust as it is implemented... There's no reason to think because you or the current gen of Python devs are focused on C then the next gen or further will too.
I understand this sentiment. Part of it was people trying to build up their cv for Rust. On the other hand, some tools/libraries in Python were old. Take pandas for example, it was not good for modern use. We desperately needed something like polars and even that is being outpaced by current trends.
Curious what you see as outpacing polars, hybrid analytical/streaming query engines?
PSA careful replacing `pip` with `uv` thinking it's a drop-in replacement.
By default `uv` won't generate `pyc` files which might make your service much slower to start.
See https://docs.astral.sh/uv/reference/settings/#pip_compile-by...
uv's guide for use in containers is a better reference for this: https://docs.astral.sh/uv/guides/integration/docker/#compili...
Agreed if you're going there from the start.
I stumbled on this by porting something using that was previously pip, and that surprisingly different default has been a foot gun.
been using uv on a flask container and honestly the diff in build times is just boringly huge. not even the speed tho, it's how predictable things get. no stupid “why did pip install this version” moments. you write a pyproject.toml, freeze with uv lock, done.
>In docker you can just raw COPY pyproject.toml uv.lock* . then run uv sync --frozen --no-install-project. this skips your own app so your install layer stays cacheable. real ones know how painful it is to rebuild entire layers just cuz one package changed.
>UV_PROJECT_ENVIRONMENT=/home/python/.local bypasses venv. which means base images can be pre-warmed or shared across builds. saves infra cost silently. just flip UV_COMPILE_BYTECODE=1 and get .pyc at build.
> It kills off mutable environments. forces you to respect reproducibility. if your build is broken, it's your lockfile's fault now. accountability becomes visible
UV just works. Easily one of the best things to happen to Python packaging in years.
I have software people use up on pypi and I'd love to switch to uv for personal use to benefit from the improved speed but I'd need some guarantees that things work EXACTLY how they work on pip or that I could run them concurrently. If I put instructions up for users to "just run pip install xxx" I need to know that if they see any errors I can see those too for debugging/troubleshooting.
It does not work EXACTLY how pip works, big differences are covered here: https://docs.astral.sh/uv/pip/compatibility/
Some of these are uv following the standards while pip is still migrating away from legacy behavior, some of these are design choices that uv has made, because the standard is underdefined, it's a tool specific choice, or uv decided not to follow the standards for whatever reason.
2025 and python packaging and dependencies management is still a mess.
It's only a mess because not everyone has adopted uv yet (IMO)
It’s a mess because there’s no first party, properly thought out and working solution.
So every year we get a new “new way” to do it. Like that xkcd… this time this is the standard that will work!
UV looks very promising but I can assure you, if everyone adopted it tomorrow we would see a long tail of scenarios that UV does not work well for.
I am perfectly content for a 90% solution, we should not let the perfect be the enemy of the good.
I would love to see that happen and see how astral responds. I would love to see uv get built into Python 4
Python package management is a classic example of xkcd #927, and the community cannot be blamed for developing a Pavlovian response when it comes to yet-another-package-manager that promises to be the final solution.
Yep. The lesson is to get this right early in your language design. Do not punt until version 2.0. Think twice before putting the package/module/whatever metadata in an executable script. If you do decide to do that, think a third time. It works out better for some languages (like Common Lisp) than others (like Python).
never had a problem with dependencies. how is it a mess? you have requirements.txt and venv per project. doesn't get easier than that
Yes, it's a mess (New: now with Rust!)
What originally convinced me to try uv was the promise of faster container builds, and it certainly delivered on that.
As someone who usually used platform pythons, despite advise against that, uv is now what got me to finally stop doing so.
I'd like to see a security breakdown of uv versus pip versus conda versus whatever fashionable package manager I've not heard of yet.
Speed is okay, but security of a package manager is far more important.
uv is generally more secure than pip. It resolves dependencies without executing arbitrary code, verifies package hashes by default, and avoids common risks like typosquatting and code execution during install. It's also faster and more reproducible.
https://chaitalks.tech/uv-a-modern-python-package-manager-in...
I'd be interested to know under what circumstances pip executes arbitrary code while resolving dependencies ... how does that work ?
And while I'm here ... how does uv go about mitigating typosquatting risks ? I could imagine how it might issue warnings if you perhaps it notices you requesting "dlango", which would work OK for the top 10% but are you suggesting there's some more general solution built into uv ?
I did a quick search but 'typosquatting' is not an easy string to cut through.
To install a package and its dependencies, you need the list of dependencies. This metadata is not always statically available!
Python packages are often just a zip file full of py files, with one of them called 'setup.py'. Running this file installs the package (originally using [distutils](https://docs.python.org/3.9/install/index.html#install-index)). This installation may fail if dependencies are not present, but there’s no method provided for installing those dependencies. You’re supposed to read the error message, go download the source for the missing dependencies, then run their setup.py scripts to install them.
How does uv get around this?
I don’t know; I’ve only just heard of uv. I know about the packaging problem because I’ve had to deal with that nonsense a few times.
Don’t think it does. You see the import errors when you run your code and you add the requirements to your project.
because instead of running setup.py it directly fetches the specified dependencies
a)"Thanks to backwards compatibility, a package offered only as a source distribution and with the legacy setup.py file for configuration and metadata specification will run the code in setup.py as part of the installation." https://blog.phylum.io/python-package-installation-attacks/
b) pip now has an option _not_ to run arbitrary code by disallowing source distributions, by passing --only-binary :all:
"By default, pip does not perform any checks to protect against remote tampering and involves running arbitrary code from distributions. It is, however, possible to use pip in a manner that changes these behaviours, to provide a more secure installation mechanism." https://pip.pypa.io/en/stable/topics/secure-installs/
For a source package based on setup tools, setup.py is executed with a minimal environment and can run arbitrary code.
You can (and should!) tell pip not to do this with '--only-binary=:all:'. Building from source is a lousy default.
Requiring increasingly long arcane incantations in the name of backwards compatibility is a terrible design philosophy and introduces security fatigue. Most users will not use aliases, and it's poor security posture to ask them to.
Given how often the python community already deals with breaking changes, it shouldn't be much different for pip to adopt saner defaults in a new major version.
While I agree, pip has very strong backward compatibility requirements. I'm not sure why, maybe because people tend to upgrade it without considering the consequences.
> security breakdown of uv versus pip versus conda versus whatever fashionable package manager
In the end, every package manager (so far at least) download and runs untrusted (unless you've verified it manually) 3rd party code. Whatever the security difference is between uv and pip implementation-wise is dwarfed compared to if you haven't found a way of handling untrusted 3rd party code yet.
Just generate a requirements.txt with uv, ship that with docker, and then there's no need for all this dance
I got excited about poetry a few years back because it seemed that the community might be finally able to rally around _one_ packaging solution. My favorite thing about Go is that there isn't a shiny new packaging tool every year.. just go.mod. uv definitely looks cool and the performance is very impressive.. but should I switch all my projects over? Are you going to be around in 5 years time?
Just try it. The hype is real. I stopped using pyenv, pyenv-virtualenv, poetry, and native pip because uv is just that sticky.
Just install it and try running something using the —-with flag. That’s where I became intrigued.
It takes 5 minutes to switch most projects. And less to go back to pip. Seems silly to waste a bunch of wall time when using pip when there is a super easy alternative that doesn't have high switching costs.
Ok but that's really not the cost for a company that's been using tool X for years, accumulated some expertise in that, built their own tooling on top of it etc.
I really like uv, easily my favourite Python package manager, but the last time I tried it with Docker and Django it was a nightmare, particularly with environment variables for some reason. Maybe this will help
Some thoughts on the patterns here:
- Removing requirements.txt makes it harder to track the high-level deps your code requires (and their install options/flags). Typically requirements.txt should be the high level requirements, and you should pass them to another process that produces pinned versions. You regenerate the pinned versions/deps from the requirements.txt, so you have a way to reset all dependencies as your core ones gain or lose nested dependencies.
- +COPY --from=ghcr.io/astral-sh/uv:0.7.13 /uv /uvx /usr/local/bin/ seems useful, but the upstream docker tag could be repinned on a different hash, causing conflicts. Use the hash, or use a different way to stage your dependencies and copy them into the file. Whenever possible, confirm your artifacts match known hashes.
- Installing into the container's /home/project/.local may preserve the uv pattern, but it's going to make a container that's harder to debug. Production containers (if not all containers) should install files into normal global paths so that it's easy to find the, reason about them, and use standard tools to troubleshoot. This allows non-uv users to diagnose the application running, and removes extra abstraction layers which create unneeded complexity.
- +RUN chmod 0755 bin/ && bin/uv-install* - using scripts makes things easier to edit, but it makes it harder to understand what's going on in a container, because you have to run around the file tree reading files and building a mental map of execution. Whenever possible, just shove all the commands into RUN lines in the Dockerfile. This allows a user to just view the Dockerfile and know the entire execution without extra effort. It also removes some complexity in terms of checking out files, building Docker context, etc.
- Try to avoid docker compose and other platform-constrained tools for the running of your tests, for the freezing of versions, etc. You SDLC should first be composed of your build tools/steps using just native tools/environments. Then on top of that should go the CI tools. This separation of "dev/test environment" from CI allows you to take your "dev/test environment" and run it on any CI platform - Docker Compose, GitHub Actions, CircleCI, GitLab CI, Jenkins, etc - without modifying the "dev/test environment" tools or workflow. Personally I have a dev.sh that sets up the dev environment, build.sh to run any build steps, test.sh to run all the test stuff, ci.sh to run ci/cd specific stuff (it just calls the CI/CD system's API and waits for status), and release.sh to cut new releases.
I find it's clearer to store all pinned dependencies in requirements.txt using pip-compile (or pip freeze). There's no finagling in trying to determine which file contains the dependency snapshot for installing an application. High level dependencies can be defined in requirements.in or pyproject.toml.
That's what I meant, I just call my requirements.in requirements.txt, and the pinned versions go in requirements.txt.frozen or pyproject.toml. As long as there's one file with high level and one file with pinned
If you write out `requirements.txt` by hand, do you also need to resolve deps and transitive deps by hand?
This is what pushed me to use Poetry.
I write my high level dependencies by hand in a "requirements.in" for applications, and in "pyproject.toml" for libraries.
A simple "requirements.in" I did over this weekend was a single dependency:
miniboss >=0.4, <0.5
pip-compile -o requirements.txt requirements.in
attrs==25.3.0
...13 more dependencies
I've largely settled on this scheme for work and person projects because it's simple (only dev dependency is pip-tools or uv), and it doesn't tie me to a particular Python project management tool (pipenv, pdm, poetry, etc.).
You do not write requirements by hand. You write requirements.in and uv pip-compile to requirements.txt
Does `pip-compile` solve deps and transitive deps?
I thought it only locks down hashes?
> Removing requirements.txt makes it harder to track the high-level deps your code requires
The very first section of the article talks about replacing requirements.txt with pyproject.toml which contains a similar high-level list of deps
Agreed, custom scripts are great for the person who wrote them and/or uses them all the time but I much prefer to add as little veneer over upstream tools as possible lest I get messages like “hey how do I actually restart this process/get these logs/upgrade this package?”
Only thing I’m not sure about: why is having your list of requirements in requirements.txt vs project.toml? Isn’t it just one file vs another?
What is Astral's economic model? This is my primary hesitation to full adoption. Is Astral's future the same story as Anaconda?
Ok, same story as Anaconda but they haven't gotten as far.
Is there no apt repo to install uv? My Docker builds using pip take around 2 minutes, not sure the juice is worth the squeeze upgrading to uv.
Current Dockerfile pip is as simple as:
COPY --chown=python:python requirements.txt .
RUN pip install --no-cache-dir --upgrade pip && \
pip install --no-cache-dir --compile -r requirements.txt
COPY --chown=python:python . .
RUN python -m compileall -f .Installation in Docker just looks like
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
(We'd recommend pinning the version or SHA in production)
With a cache directory that can be 0 minute.
RUN --mount=type=cache,target=/root/.cache/pip pip install ...uv is not in Debian's or Ubuntu's apt repositories to my knowledge. What I do instead because I don't like piping shell scripts from URLs is "pip install --upgrade pip uv", and then run "uv pip ...".