Figuring out library holes in the package installation process

Brett Cannon

2018-04-07 00:35:08 UTC

Based on no one speaking up, I'm assuming I wasn't wrong with my outline.
;) That looks like there may be one PEP to write and a few things to add to
'packaging' (or other projects as appropriate).

Post by Brett Cannon
With the big push we are all working towards standards, I thought it would
be interesting to outline the steps it takes to go from requesting a
package to be installed to it ending up on disk (see my first email on
such an outline
<https://groups.google.com/forum/#!msg/pypa-dev/P_UbPhETvjg/adFwjujjCwAJ;context-place=forum/pypa-dev>).
After getting feedback on that outline, I then decided to try and pair each
complicated step of that outline with the PEP and a library that
essentially implements that step. I'm doing this to see where there might
be a missing PEP and where that are missing library implementations of a
specific PEP.
I've ended up with some holes on both the PEP and library side, so I'm
hoping that either I'm overlooking something or that I've at least
identified a place where some help might be appreciated by people.
1. Specify package requirement (PEP 508
<https://www.python.org/dev/peps/pep-0508/>/packaging
<https://packaging.pypa.io/>)
1. Translate name to PyPI-compatible name
2. Tease out requirement details (e.g. version, markers, etc.)
2. Check if package is already installed (PEP 376
<https://www.python.org/dev/peps/pep-0376/>/XXX)
3. Check wheel cache (XXX/XXX)
4. Check PyPI for package
5. Choose appropriate file
1. Get list of files (PEP 503
<https://www.python.org/dev/peps/pep-0503/>/_)
2. Calculate best-fitting wheel (PEPs 425
<https://www.python.org/dev/peps/pep-0425/> & 427
<https://www.python.org/dev/peps/pep-0427/>/XXX)
3. Fallback to .tar.gz sdist if necessary & available
6. Download file
1. Extract
2. Read pyproject.toml (PEP 518/XXX)
3. Create venv
4. Install build dependencies
1. Static (PEP 518 <https://www.python.org/dev/peps/pep-0518/>/_)
2. Dynamic (PEP 517 <https://www.python.org/dev/peps/pep-0517/>
/XXX)
5. Build wheel (PEP 517 <https://www.python.org/dev/peps/pep-0517/>
/XXX)
8. Cache wheel (XXX/XXX)
9. Get dependency requirements of the wheel (PEP 566
<https://packaging.python.org/specifications/core-metadata/>/XXX)
10. Check if dependencies are already installed (like step 2)
11. Install missing dependencies (like step 12 below)
12. Install wheel (PEP 427 <https://www.python.org/dev/peps/pep-0427/>
/XXX)
1. To appropriate location (e.g. user, venv, etc.)
2. Library, scripts, etc. to appropriate locations
13. Record the installation (PEP 427
<https://www.python.org/dev/peps/pep-0427/>/XXX)