Troubleshooting hdbscan Installation Error with Poetry for Phoenix-Arize
Hey all, am currently trying to install phoenix-arize with poetry and running into an error installing hdbscan ==0.8.33 - any ideas?
Hey all, am currently trying to install phoenix-arize with poetry and running into an error installing hdbscan ==0.8.33 - any ideas?
I am ssh into an ec2 instance at the moment, is that what you are referring to?
Yep. Assuming it's running on Linux then?
Can you send the full stacktrace?
yeah one second
ubuntu@ip-172-31-47-34:~/poc$ poetry add arize-phoenix Using version ^3.22.0 for arize-phoenix Updating dependencies Resolving dependencies... (2.9s) Package operations: 2 installs, 1 update, 0 removals - Updating scikit-learn (1.2.2 -> 1.4.2) - Installing hdbscan (0.8.33): Failed ChefBuildError Backend subprocess exited when trying to invoke build_wheel running bdist_wheel running build running build_py creating build creating build/lib.linux-x86_64-cpython-310 creating build/lib.linux-x86_64-cpython-310/hdbscan copying hdbscan/robust_single_linkage_.py -> build/lib.linux-x86_64-cpython-310/hdbscan copying hdbscan/hdbscan_.py -> build/lib.linux-x86_64-cpython-310/hdbscan copying hdbscan/validity.py -> build/lib.linux-x86_64-cpython-310/hdbscan copying hdbscan/flat.py -> build/lib.linux-x86_64-cpython-310/hdbscan copying hdbscan/__init__.py -> build/lib.linux-x86_64-cpython-310/hdbscan copying hdbscan/prediction.py -> build/lib.linux-x86_64-cpython-310/hdbscan copying hdbscan/plots.py -> build/lib.linux-x86_64-cpython-310/hdbscan creating build/lib.linux-x86_64-cpython-310/hdbscan/tests copying hdbscan/tests/test_prediction_utils.py -> build/lib.linux-x86_64-cpython-310/hdbscan/tests copying hdbscan/tests/test_rsl.py -> build/lib.linux-x86_64-cpython-310/hdbscan/tests copying hdbscan/tests/test_flat.py -> build/lib.linux-x86_64-cpython-310/hdbscan/tests copying hdbscan/tests/__init__.py -> build/lib.linux-x86_64-cpython-310/hdbscan/tests copying hdbscan/tests/test_hdbscan.py -> build/lib.linux-x86_64-cpython-310/hdbscan/tests running build_ext cythoning hdbscan/_hdbscan_tree.pyx to hdbscan/_hdbscan_tree.c /tmp/tmpesx3p9u1/.venv/lib/python3.10/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/tmplf3egthk/hdbscan-0.8.33/hdbscan/_hdbscan_tree.pyx tree = Parsing.p_module(s, pxd, full_module_name) cythoning hdbscan/_hdbscan_linkage.pyx to hdbscan/_hdbscan_linkage.c /tmp/tmpesx3p9u1/.venv/lib/python3.10/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/tmplf3egthk/hdbscan-0.8.33/hdbscan/_hdbscan_linkage.pyx tree = Parsing.p_module(s, pxd, full_module_name) cythoning hdbscan/_hdbscan_boruvka.pyx to hdbscan/_hdbscan_boruvka.c /tmp/tmpesx3p9u1/.venv/lib/python3.10/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/tmplf3egthk/hdbscan-0.8.33/hdbscan/_hdbscan_boruvka.pyx tree = Parsing.p_module(s, pxd, full_module_name) cythoning hdbscan/_hdbscan_reachability.pyx to hdbscan/_hdbscan_reachability.c /tmp/tmpesx3p9u1/.venv/lib/python3.10/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/tmplf3egthk/hdbscan-0.8.33/hdbscan/_hdbscan_reachability.pyx tree = Parsing.p_module(s, pxd, full_module_name) cythoning hdbscan/_prediction_utils.pyx to hdbscan/_prediction_utils.c /tmp/tmpesx3p9u1/.venv/lib/python3.10/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/tmplf3egthk/hdbscan-0.8.33/hdbscan/_prediction_utils.pyx tree = Parsing.p_module(s, pxd, full_module_name) cythoning hdbscan/dist_metrics.pyx to hdbscan/dist_metrics.c /tmp/tmpesx3p9u1/.venv/lib/python3.10/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/tmplf3egthk/hdbscan-0.8.33/hdbscan/dist_metrics.pxd tree = Parsing.p_module(s, pxd, full_module_name) building 'hdbscan._hdbscan_tree' extension creating build/temp.linux-x86_64-cpython-310 creating build/temp.linux-x86_64-cpython-310/hdbscan x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/tmp/tmpesx3p9u1/.venv/include -I/usr/include/python3.10 -I/tmp/tmpesx3p9u1/.venv/lib/python3.10/site-packages/numpy/core/include -c hdbscan/_hdbscan_tree.c -o build/temp.linux-x86_64-cpython-310/hdbscan/_hdbscan_tree.o error: command 'x86_64-linux-gnu-gcc' failed: No such file or directory at ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/installation/chef.py:164 in _prepare 160│ 161│ error = ChefBuildError("\n\n".join(message_parts)) 162│ 163│ if error is not None: → 164│ raise error from None 165│ 166│ return path 167│ 168│ def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path: Note: This error originates from the build backend, and is likely not a problem with poetry but with hdbscan (0.8.33) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "hdbscan (==0.8.33)"'.
older version of arize-phoenix was working on another instance running linux although different specs for that machine, tried that older version of arize-phoenix on this one but failing as well
What happens when you run this?
pip wheel --no-cache-dir --use-pep517 "hdbscan (==0.8.33)"i'll try now
There's also this comment here that might help: https://github.com/scikit-learn-contrib/hdbscan/issues/293#issuecomment-941032921
thanks looking into this, that first command seemed to have worked, so I guess I have to not use poetry ?
Take a look at that post Dustin sent. Might just be you need to install gcc.