Repository navigation
Closed
Description
The pip install process for slycot fails on most clean linux installs. Here are the results running in a circleci/python Docker container:
$ pip install slycot
Defaulting to user installation because normal site-packages is not writeable
Collecting slycot
Using cached slycot-0.3.5.0.tar.gz (1.5 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Collecting numpy
Using cached numpy-1.18.4-cp38-cp38-manylinux1_x86_64.whl (20.7 MB)
Building wheels for collected packages: slycot
Building wheel for slycot (PEP 517) ... error
ERROR: Command errored out with exit status 1:
command: /usr/local/bin/python /usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /tmp/tmprwu6q9xf
cwd: /tmp/pip-install-djjch8io/slycot
Complete output (84 lines):
Not searching for unused variables given on the command line.
CMake Error: CMake was unable to find a build program corresponding to "Ninja". CMAKE_MAKE_PROGRAM is not set. You probably need to select a different build tool.
-- Configuring incomplete, errors occurred!
See also "/tmp/pip-install-djjch8io/slycot/_cmake_test_compile/build/CMakeFiles/CMakeOutput.log".
Not searching for unused variables given on the command line.
-- The C compiler identification is GNU 8.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc - works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- The Fortran compiler identification is unknown
CMake Error at CMakeLists.txt:4 (ENABLE_LANGUAGE):
No CMAKE_Fortran_COMPILER could be found.
Tell CMake where to find the compiler by setting either the environment
variable "FC" or the CMake cache entry CMAKE_Fortran_COMPILER to the full
path to the compiler, or to the compiler name if it is in the PATH.
-- Configuring incomplete, errors occurred!
See also "/tmp/pip-install-djjch8io/slycot/_cmake_test_compile/build/CMakeFiles/CMakeOutput.log".
See also "/tmp/pip-install-djjch8io/slycot/_cmake_test_compile/build/CMakeFiles/CMakeError.log".
--------------------------------------------------------------------------------
-- Trying "Ninja" generator
--------------------------------
---------------------------
----------------------
-----------------
------------
-------
--
--
-------
------------
-----------------
----------------------
---------------------------
--------------------------------
-- Trying "Ninja" generator - failure
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
-- Trying "Unix Makefiles" generator
--------------------------------
---------------------------
----------------------
-----------------
------------
-------
--
--
-------
------------
-----------------
----------------------
---------------------------
--------------------------------
-- Trying "Unix Makefiles" generator - failure
--------------------------------------------------------------------------------
********************************************************************************
scikit-build could not get a working generator for your system. Aborting build.
Building Linux wheels for Python 3.8 requires a compiler (e.g gcc).
It can be installed using debian package manager:
sudo apt-get install build-essential
To build compliant wheels, consider using the manylinux system described in PEP-513.
Get it with "dockcross/manylinux-x64" docker image:
https://github.com/dockcross/dockcross#readme
For more details, please refer to scikit-build documentation:
http://scikit-build.readthedocs.io/en/latest/generators.html#linux
********************************************************************************
----------------------------------------
ERROR: Failed building wheel for slycot
Failed to build slycot
ERROR: Could not build wheels for slycot which use PEP 517 and cannot be installed directly
But the thing is... running sudo apt-get install build-essential doesn't help at all. Installing build-essential and then re-running results in exactly the same error.
I don't have issues pip installing any other packages in the same container.
Metadata
Metadata
Assignees
Labels
No labels
Activity
samuela commentedon May 12, 2020
It's also worth mentioning that gcc is in fact installed in the container:
bnavigator commentedon May 12, 2020
Hi,
To quote from the version 0.3.5 README:
From your log:
gcc 8.3.0 is found, but gfortran is not installed. The generic scikit-build message does not differentiate between the C and Fortran compiler.
samuela commentedon May 12, 2020
Yeah, as I discovered the magic incantation is
bnavigator commentedon May 12, 2020
... or any other more performant blas/lapack implementation
orzechow commentedon Jul 14, 2021
I'm running into the same problem in a CI build job using Ubuntu 18.04 with ROS in a docker container, which installs slycot automatically via pip (as a dependency of our application code).
As the slycot pip build depends on gfortran, but doesn't provide it as dependency itself, the build fails.
Even, if I manually specify gfortran as additional dependency of my application, the build does not always succeed, because we can't define the order in which the dependencies are installed (rosdep only follows the dependency tree).
I'm not an expert on pip, but would it be possible to either
bnavigator commentedon Jul 14, 2021
Wheels are not distro specific. When we provide wheels, which are linked with the blas/lapack libraries of one Ubuntu LTS, people will try to install them on different Linux distros and have a horrible user experience. The NumPy and SciPy wheels provide their own lapack/libraries and we would have to sync with them in order to avoid trouble.
As far as I am aware, pip only follows the Python package dependencies specified via setuptools. A Fortran compiler is not a Python package. It has to be provided through the operating system before running pip install. For Ubuntu, that is
apt-get install gfortranas mentioned above.I am not familiar with ROS containers. Here are two ideas, which you could try:
apt-get install gfortran ...before thepip install your-applicationpip wheeland inject the file into the docker systemIf you need pre-compiled packages, consider using the conda ecosystem instead of pip. Or use a distribution which provides distro specific packages. I know of Archlinux (AUR) and openSUSE Tumleweed.
bnavigator commentedon Jul 14, 2021
Hmm, that sounds more like you have specified a runtime dependency where a buildtime dependency should have been used. Somehow you must specify to rosdep that gfortran is a build requirement for Slycot.
orzechow commentedon Jul 15, 2021
Actually, I did specify gfortran as a build dependency of my ROS package:
but rosdep is interpreting
buildtool_dependasso it thinks gfortran is needed to build
my_ros_pkg, not to build python-slycot-pip.Which makes sense, because gfortran is a build dependency of slycot – not
my_ros_pkg.That's why I'm asking why pip is not installing gfortran, if it needs it or how to get it to do so?
As a workaround I could try to setup a wrapper package with gfortran as build dependency and then add this wrapper package to
my_ros_pkg.Still, this feels hacky…
bnavigator commentedon Jul 15, 2021
Because pip can only install python packages. It is not meant to install system packages at all. If you run pip as root it gives you a stern warning.
I guess you have to fix the ROS package
python-slycot-pip.orzechow commentedon Jul 15, 2021
You're right, pip wouldn't be able to install system packages anyways (And using pip as root is a bad idea, because it can break the system Python installation).
Unfortunately there is no ROS package
python-slycot-pipI could fix. It is simply a dependency definition telling rosdep (the ROS dependency manager) to resolve it to thepython-slycotpip package – probably assuming that all pip packages will be installed from wheels, if they contain binaries.And it seems that I can't add build dependencies to this rosdep entry…
Btw. changing the OS is not an option, as ROS is (more or less) tightly coupled to Ubuntu.
Neither is conda supported by rosdep 😖
I'll take care to install gfortran in all our docker containers and physical machines then (won't hurt anyways) 💁♂️
Thanks for your explanations and efforts @bnavigator !
bnavigator commentedon Jul 15, 2021
You could create a
python-slycotROS package (or even a Debian/Ubuntu one).Don't forget cmake and the lapack/blas headers. Have a look at our GitHub Actions workflow for hints:
Slycot/.github/workflows/slycot-build-and-test.yml
Lines 88 to 99 in a40505a
Slycot/.github/workflows/slycot-build-and-test.yml
Lines 115 to 122 in a40505a
orzechow commentedon Jul 15, 2021
Thanks for the hint!