Trying to outline the steps taken to go from "I want this package" to it being installed

[Brett Cannon]

I have a project idea, but before I start it I need to make sure that I
have the high-order steps necessary to go from `pip install pip=9.0.1` to
it actually ending up on disk. Now I'm only considered with
modern/bleeding-edge, spec-based stuff, so PEP 517/518 and no setup.py, etc.

Anyway, if people can point out any steps the below outline is missing I
would appreciate it. Thanks!


1. Specify package requirement
1. Translate name to PyPI-compatible name
2. Tease out requirement details (e.g. version, markers, etc.)
2. Check if package is already installed
3. Check PyPI for package
4. Choose appropriate file
1. Get list of files
2. Calculate best-fitting wheel
3. Fallback to .tar.gz sdist
5. Download file
6. If sdist:
1. Extract
2. Read pyproject.toml
3. Create venv
4. Install build dependencies
5. Build wheel
7. Cache wheel
8. Get dependency requirements
9. Check if dependencies are already installed
10. Install missing dependencies
11. Install wheel
1. To appropriate location (e.g. user, venv, etc.)
2. Library, scripts, etc. to appropriate locations
12. Record the installation

[Nick Coghlan]

Post by Brett Cannon
I have a project idea, but before I start it I need to make sure that I
have the high-order steps necessary to go from `pip install pip=9.0.1` to
it actually ending up on disk. Now I'm only considered with
modern/bleeding-edge, spec-based stuff, so PEP 517/518 and no setup.py, etc.
Anyway, if people can point out any steps the below outline is missing I
would appreciate it. Thanks!
1. Specify package requirement
1. Translate name to PyPI-compatible name
2. Tease out requirement details (e.g. version, markers, etc.)
2. Check if package is already installed

Depending on the installer design, a local download/build cache may be
checked before checking PyPI (and since you include a caching step later,
you'll presumably want to cover the caching step as well).

Post by Brett Cannon
1. Check PyPI for package
2. Choose appropriate file
1. Get list of files
2. Calculate best-fitting wheel
3. Fallback to .tar.gz sdist
3. Download file
1. Extract
2. Read pyproject.toml
3. Create venv
4. Install build dependencies

After installing the static build dependencies, you also need to query for
any dynamic build dependencies and install them if they're requested:
https://www.python.org/dev/peps/pep-0517/#get-requires-for-build-wheel

This build dependency installation step can get arbitrarily complicated if
you allow build dependencies to be installed from source, so the initial
implementation in pip requires that build dependencies already be available
as wheel files (either on the index server or in the local artifact cache).

Cheers,
Nick.

[Brett Cannon]

Thanks for the extra details, Nick! I have some documentation to read on
some projects now that I have a complete list, but once that's done I'll
come back here with my idea. ;)

Post by Nick Coghlan

Post by Brett Cannon
I have a project idea, but before I start it I need to make sure that I
have the high-order steps necessary to go from `pip install pip=9.0.1` to
it actually ending up on disk. Now I'm only considered with
modern/bleeding-edge, spec-based stuff, so PEP 517/518 and no setup.py, etc.
Anyway, if people can point out any steps the below outline is missing I
would appreciate it. Thanks!
1. Specify package requirement
1. Translate name to PyPI-compatible name
2. Tease out requirement details (e.g. version, markers, etc.)
2. Check if package is already installed

Depending on the installer design, a local download/build cache may be
checked before checking PyPI (and since you include a caching step later,
you'll presumably want to cover the caching step as well).

Post by Brett Cannon
1. Check PyPI for package
2. Choose appropriate file
1. Get list of files
2. Calculate best-fitting wheel
3. Fallback to .tar.gz sdist
3. Download file
1. Extract
2. Read pyproject.toml
3. Create venv
4. Install build dependencies

After installing the static build dependencies, you also need to query for
https://www.python.org/dev/peps/pep-0517/#get-requires-for-build-wheel
This build dependency installation step can get arbitrarily complicated if
you allow build dependencies to be installed from source, so the initial
implementation in pip requires that build dependencies already be available
as wheel files (either on the index server or in the local artifact cache).
Cheers,
Nick.
--