Question about python packaging

[蕭毅]

It seems that originally I posted to wrong place.
https://groups.google.com/forum/#!topic/python-virtualenv/iExzUJhC_PY

Hi all:

I have a new Python package https://github.com/NAL-i5K/GFF3toolkit with new
release.

In this new release, I have removed some binaries originally in git repo
and make them been install through internet.
https://github.com/NAL-i5K/GFF3toolkit/blob/6cd86e007713f7ab0fbf07adc2caed88889056a9/setup.py#L22
.

However, when I upload the package to PyPI through

python2 setup.py sdist
twine upload --repository-url https://test.pypi.org/legacy/ dist/*

and install through

pip2 install --index-url https://test.pypi.org/simple/

It will invoke

python setup.py bdist_wheel

rather than sdist.

It looks like the customized install command will not be run. Therefore,
binaries file will not be downloaded and installed into site-packages.
How should I do to make it work ?

Thanks !

[Brett Cannon]

Post by 蕭毅
It seems that originally I posted to wrong place.
https://groups.google.com/forum/#!topic/python-virtualenv/iExzUJhC_PY
I have a new Python package https://github.com/NAL-i5K/GFF3toolkit with
new release.
In this new release, I have removed some binaries originally in git repo
and make them been install through internet.
https://github.com/NAL-i5K/GFF3toolkit/blob/6cd86e007713f7ab0fbf07adc2caed88889056a9/setup.py#L22
.
However, when I upload the package to PyPI through
python2 setup.py sdist
twine upload --repository-url https://test.pypi.org/legacy/ dist/*
and install through
pip2 install --index-url https://test.pypi.org/simple/
It will invoke
python setup.py bdist_wheel
rather than sdist.
It looks like the customized install command will not be run. Therefore,
binaries file will not be downloaded and installed into site-packages.
How should I do to make it work ?

It doesn't run `sdist` because all that does is make a .tar.gz of source
code. the bdist_wheel command is what actually makes the wheel that pip
will install. IOW it goes sdist -> wheel -> install. So if you want code to
run at install it needs to happen at the bdist_wheel stage.

[蕭毅]

Hi:

Thanks answering me, but I am not sure if I understand or not. Do you mean
I need to override different command rather than setuptools.command.install
?

Post by Brett Cannon

Post by 蕭毅
It seems that originally I posted to wrong place.
https://groups.google.com/forum/#!topic/python-virtualenv/iExzUJhC_PY
I have a new Python package https://github.com/NAL-i5K/GFF3toolkit with
new release.
In this new release, I have removed some binaries originally in git repo
and make them been install through internet.
https://github.com/NAL-i5K/GFF3toolkit/blob/6cd86e007713f7ab0fbf07adc2caed88889056a9/setup.py#L22
.
However, when I upload the package to PyPI through
python2 setup.py sdist
twine upload --repository-url https://test.pypi.org/legacy/ dist/*
and install through
pip2 install --index-url https://test.pypi.org/simple/
It will invoke
python setup.py bdist_wheel
rather than sdist.
It looks like the customized install command will not be run. Therefore,
binaries file will not be downloaded and installed into site-packages.
How should I do to make it work ?

It doesn't run `sdist` because all that does is make a .tar.gz of source
code. the bdist_wheel command is what actually makes the wheel that pip
will install. IOW it goes sdist -> wheel -> install. So if you want code to
run at install it needs to happen at the bdist_wheel stage.

[Nathaniel Smith]

There is no general way to run arbitrary code at package install time. You
can run arbitrary code when the setuptools bdist_wheel command is run, and
then put the resulting files into your wheel. Putting the files directly
into site-packages won't work though.

Post by 蕭毅
It seems that originally I posted to wrong place.
https://groups.google.com/forum/#!topic/python-virtualenv/iExzUJhC_PY
I have a new Python package https://github.com/NAL-i5K/GFF3toolkit with
new release.
In this new release, I have removed some binaries originally in git repo
and make them been install through internet.
https://github.com/NAL-i5K/GFF3toolkit/blob/6cd86e007713f7ab0fbf07adc2caed88889056a9/setup.py#L22
.
However, when I upload the package to PyPI through
python2 setup.py sdist
twine upload --repository-url https://test.pypi.org/legacy/ dist/*
and install through
pip2 install --index-url https://test.pypi.org/simple/
It will invoke
python setup.py bdist_wheel
rather than sdist.
It looks like the customized install command will not be run. Therefore,
binaries file will not be downloaded and installed into site-packages.
How should I do to make it work ?
Thanks !

[蕭毅]

hmm... I am still not quite understand. In my case, I have binaries that
can only be run under Windows, Mac, Linux, respectively. They are compiled
executable with the same functionalities and interfaces. Do you mean that I
need to ship all these binaries (three binaries for all these three
platforms) into a bdist ?

Post by Nathaniel Smith
There is no general way to run arbitrary code at package install time. You
can run arbitrary code when the setuptools bdist_wheel command is run, and
then put the resulting files into your wheel. Putting the files directly
into site-packages won't work though.

Post by 蕭毅
It seems that originally I posted to wrong place.
https://groups.google.com/forum/#!topic/python-virtualenv/iExzUJhC_PY
I have a new Python package https://github.com/NAL-i5K/GFF3toolkit with
new release.
In this new release, I have removed some binaries originally in git repo
and make them been install through internet.
https://github.com/NAL-i5K/GFF3toolkit/blob/6cd86e007713f7ab0fbf07adc2caed88889056a9/setup.py#L22
.
However, when I upload the package to PyPI through
python2 setup.py sdist
twine upload --repository-url https://test.pypi.org/legacy/ dist/*
and install through
pip2 install --index-url https://test.pypi.org/simple/
It will invoke
python setup.py bdist_wheel
rather than sdist.
It looks like the customized install command will not be run. Therefore,
binaries file will not be downloaded and installed into site-packages.
How should I do to make it work ?
Thanks !

--

Leo (Yi Hsiao),
Bioinformatics Internship,
i5k workspace, National Agricultural Library,
MD, USA

Personal Website: https://hsiaoyi0504.github.io/
GitHub: https://github.com/hsiaoyi0504/
LinkedIn: https://www.linkedin.com/in/hsiaoyi0504/

[Nathaniel Smith]

No, a Windows wheel only needs to have Windows binaries, a MacOS wheel only
needs MacOS binaries, and so forth.

Post by 蕭毅
hmm... I am still not quite understand. In my case, I have binaries that
can only be run under Windows, Mac, Linux, respectively. They are compiled
executable with the same functionalities and interfaces. Do you mean that I
need to ship all these binaries (three binaries for all these three
platforms) into a bdist ?

Post by Nathaniel Smith
There is no general way to run arbitrary code at package install time.
You can run arbitrary code when the setuptools bdist_wheel command is run,
and then put the resulting files into your wheel. Putting the files
directly into site-packages won't work though.

Post by 蕭毅
It seems that originally I posted to wrong place.
https://groups.google.com/forum/#!topic/python-virtualenv/iExzUJhC_PY
I have a new Python package https://github.com/NAL-i5K/GFF3toolkit with
new release.
In this new release, I have removed some binaries originally in git repo
and make them been install through internet.
https://github.com/NAL-i5K/GFF3toolkit/blob/6cd86e007713f7ab0fbf07adc2caed88889056a9/setup.py#L22
.
However, when I upload the package to PyPI through
python2 setup.py sdist
twine upload --repository-url https://test.pypi.org/legacy/ dist/*
and install through
pip2 install --index-url https://test.pypi.org/simple/
It will invoke
python setup.py bdist_wheel
rather than sdist.
It looks like the customized install command will not be run. Therefore,
binaries file will not be downloaded and installed into site-packages.
How should I do to make it work ?
Thanks !

--

Leo (Yi Hsiao),
Bioinformatics Internship,
i5k workspace, National Agricultural Library,
MD, USA
Personal Website: https://hsiaoyi0504.github.io/
GitHub: https://github.com/hsiaoyi0504/
LinkedIn: https://www.linkedin.com/in/hsiaoyi0504/

[蕭毅]

Ok, I got this point. Can you give me the information about "You can run
arbitrary code when the setuptools bdist_wheel command is run" ? I guess
this should be pretty the same as what I did (customized the installation
command from setuptools).

Post by Nathaniel Smith
No, a Windows wheel only needs to have Windows binaries, a MacOS wheel
only needs MacOS binaries, and so forth.

Post by 蕭毅
hmm... I am still not quite understand. In my case, I have binaries that
can only be run under Windows, Mac, Linux, respectively. They are compiled
executable with the same functionalities and interfaces. Do you mean that I
need to ship all these binaries (three binaries for all these three
platforms) into a bdist ?

Post by Nathaniel Smith
There is no general way to run arbitrary code at package install time.
You can run arbitrary code when the setuptools bdist_wheel command is run,
and then put the resulting files into your wheel. Putting the files
directly into site-packages won't work though.

Post by 蕭毅
It seems that originally I posted to wrong place.
https://groups.google.com/forum/#!topic/python-virtualenv/iExzUJhC_PY
I have a new Python package https://github.com/NAL-i5K/GFF3toolkit with
new release.
In this new release, I have removed some binaries originally in git
repo and make them been install through internet.
https://github.com/NAL-i5K/GFF3toolkit/blob/6cd86e007713f7ab0fbf07adc2caed88889056a9/setup.py#L22
.
However, when I upload the package to PyPI through
python2 setup.py sdist
twine upload --repository-url https://test.pypi.org/legacy/ dist/*
and install through
pip2 install --index-url https://test.pypi.org/simple/
It will invoke
python setup.py bdist_wheel
rather than sdist.
It looks like the customized install command will not be run.
Therefore, binaries file will not be downloaded and installed into
site-packages.
How should I do to make it work ?
Thanks !

--

Leo (Yi Hsiao),
Bioinformatics Internship,
i5k workspace, National Agricultural Library,
MD, USA
Personal Website: https://hsiaoyi0504.github.io/
GitHub: https://github.com/hsiaoyi0504/
LinkedIn: https://www.linkedin.com/in/hsiaoyi0504/
--

Leo (Yi Hsiao),
Bioinformatics Internship,
i5k workspace, National Agricultural Library,
MD, USA

Personal Website: https://hsiaoyi0504.github.io/
GitHub: https://github.com/hsiaoyi0504/
LinkedIn: https://www.linkedin.com/in/hsiaoyi0504/

[蕭毅]

It turns out my final solution is to overwrite the build command from distutils,
rather than install command from setuptools.

https://github.com/NAL-i5K/GFF3toolkit/blob/ed2312072f3843a408d2c86238fb5276fce68d37/setup.py

I am not sure if it's the best solution, though.


蕭毅斌 2018幎7月23日星期䞀 UTC-4䞋午6時35分21秒寫道

Post by 蕭毅
Ok, I got this point. Can you give me the information about "You can run
arbitrary code when the setuptools bdist_wheel command is run" ? I guess
this should be pretty the same as what I did (customized the installation
command from setuptools).

Post by Nathaniel Smith
No, a Windows wheel only needs to have Windows binaries, a MacOS wheel
only needs MacOS binaries, and so forth.

Post by 蕭毅
hmm... I am still not quite understand. In my case, I have binaries that
can only be run under Windows, Mac, Linux, respectively. They are compiled
executable with the same functionalities and interfaces. Do you mean that I
need to ship all these binaries (three binaries for all these three
platforms) into a bdist ?

Post by Nathaniel Smith
There is no general way to run arbitrary code at package install time.
You can run arbitrary code when the setuptools bdist_wheel command is run,
and then put the resulting files into your wheel. Putting the files
directly into site-packages won't work though.

Post by 蕭毅
It seems that originally I posted to wrong place.
https://groups.google.com/forum/#!topic/python-virtualenv/iExzUJhC_PY
I have a new Python package https://github.com/NAL-i5K/GFF3toolkit with
new release.
In this new release, I have removed some binaries originally in git
repo and make them been install through internet.
https://github.com/NAL-i5K/GFF3toolkit/blob/6cd86e007713f7ab0fbf07adc2caed88889056a9/setup.py#L22
.
However, when I upload the package to PyPI through
python2 setup.py sdist
twine upload --repository-url https://test.pypi.org/legacy/ dist/*
and install through
pip2 install --index-url https://test.pypi.org/simple/
It will invoke
python setup.py bdist_wheel
rather than sdist.
It looks like the customized install command will not be run.
Therefore, binaries file will not be downloaded and installed into
site-packages.
How should I do to make it work ?
Thanks !

--

Leo (Yi Hsiao),
Bioinformatics Internship,
i5k workspace, National Agricultural Library,
MD, USA
Personal Website: https://hsiaoyi0504.github.io/
GitHub: https://github.com/hsiaoyi0504/
LinkedIn: https://www.linkedin.com/in/hsiaoyi0504/
--

Leo (Yi Hsiao),
Bioinformatics Internship,
i5k workspace, National Agricultural Library,
MD, USA
Personal Website: https://hsiaoyi0504.github.io/
GitHub: https://github.com/hsiaoyi0504/
LinkedIn: https://www.linkedin.com/in/hsiaoyi0504/