Skip to content

Instantly share code, notes, and snippets.

@samueljohn

samueljohn/samuels_python3_setup.md

Last active January 3, 2020 19:29
Show Gist options
  • Save samueljohn/14d766e49d1a1a43985c to your computer and use it in GitHub Desktop.
Save samueljohn/14d766e49d1a1a43985c to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. samueljohn revised this gist Oct 6, 2015. 1 changed file with 24 additions and 90 deletions.
    114 changes: 24 additions & 90 deletions samuels_python3_setup.md
    View file
    Original file line number Diff line number Diff line change
    @@ -3,132 +3,65 @@
    This document describes in which way, what packages, and from where are
    installed on my Macs to get a decent Python 3.x, scipy/numpy & Co setup.

    Note, I switched to Python's 3.x `pyvenv` from virtualenv.
    Note, I switched to Python's 3.x `pyvenv` from virtualenv in order to handle my `pip`-installable packages. Also thanks to Anaconda, the whole setup is so simple now that I use this document just to remember which additional packages I like.

    ---------------------------------------------------------

    * Download XCode from the Mac App Store or get the command line tools.
    * Install [Homebrew](https://brew.sh)
    * Install [Homebrew](https://github.com/Homebrew/homebrew)

    * Python3 itself:
    - `brew install python3`
    * Install the [Anaconda Python distribution](https://www.continuum.io/downloads)

    * Optional Pyvenv (virtual env)
    - `pyvenv --copies ~/Py`
    to set up a virtual environment. Using the default symlink mode, has got an issue with brew's Python such that things installed with `pip` insdie of the venv are not found when started.
    - `pyvenv --system-site-packages ~/Py`
    to set up a virtual environment. Using the default symlink mode, has got an issue with brew's Python but I am using the executable from the Anaconda distribution now.
    - Don't forget to `source ~/Py/bin/activate`.
    - To upgrade when homebrew has a newer Python:
    `pyvenv --copies --upgrade ~/Py`

    * To Upgrade the venv to a newer Python version
    - `rm ~/Py/bin/python*` this is kind of a hack
    - `pyvenv --upgrade --copies ~/Py`

    * NumPy and SciPy:
    - `brew tap homebrew/python`
    - `brew install gcc` (is this needed or already an dep?)
    - `brew install numpy --with-openblas --with-python3 --without-python`
    - `brew install scipy --with-openblas --with-python3 --without-python`
    - `brew test numpy --verbose`
    - `brew test scipy --verbose`

    * Pandas (for tabular data and stats)
    - `pip3 install pandas`
    - To upgrade when Anaconda has a newer Python:
    `pyvenv --upgrade ~/Py`

    * Unit testing framework:
    - `pip3 install pytest`
    - `pip3 install nose`
    - `pip3 install coverage`
    - `pip3 install nose-progressive` Nose plug-in with progress bars.
    - `pip install coverage`
    - `pip install nose-progressive` Nose plug-in with progress bars.
    Run with `nosetests --with-progressive`

    * Progess bar
    - `pip3 install pyprind`
    - `pip install pyprind`

    * Debugging
    - `pip3 install debug`
    - `pip3 install pprint`

    * Interactive shell:
    - `brew install zeromq`
    - `pip3 install pyzmq`
    - `pip3 install --pre "ipython[all]"` (the "--pre" is only for the 3.0beta1)

    * Symbolics:
    - `pip3 install sympy`

    * Python Image Library:
    - `pip3 install pillow` to install the PIL

    * Scientific plotting, matplotlib:
    - `pip3 install matplotlib`
    - `pip3 install seaborn`

    * Storing large data files as numpy arrays:
    - `brew install hdf5`
    - `pip3 install h5py`
    - `pip3 install numexpr`
    - `HDF5_DIR=$(brew --prefix)/Cellar/hdf5/<version>/ pip3 install tables`
    - `pip install debug`

    * OpenGL bindings:
    - `pip3 install pyopengl`
    - `pip install pyopengl`

    <!-- * View huge numpy arrays in OpenGL:
    - `pip3 install glumpy`
    - don't need this no longer: use vispy
    -->
    * VisPy: Most promising next gen visualizations
    - `git clone https://github.com/vispy/vispy.git`
    - `cd vispy`
    - `python3 setup.py install `
    - `python setup.py install `

    * GPU computing with numpy arrays:
    - `git clone http://git.tiker.net/trees/pyopencl.git pyopencl`
    - `cd pyopencl`
    - `git submodule init`
    - `git submodule update`
    - `python3 configure.py --cl-enable-gl`
    - `python3 setup.py install`
    - `python3 install mako`
    - `git submodule update --init`
    - `python configure.py --cl-enable-gl`
    - `python setup.py install`

    * Cython for writing C-extensions in Python-like speak
    - `pip3 install cython `
    - `pip install cython `

    * Modular Data Processing Toolkit. (needs numpy):
    - `git clone git://github.com/mdp-toolkit/mdp-toolkit`
    - `cd mdp-toolkit`
    - `python3 setup.py install`
    - `python setup.py install`

    * Memory profiling and shell tools:
    - `pip3 install psutil`
    - `pip3 install memory_profiler`

    * Machine learning
    - `pip3 install scikit-learn`

    * Scikit for image manipulation
    - `pip3 install scikit-image`

    * Fast sparse matrix package
    - `brew install suite-sparse`
    - `pip3 install scikit-umfpack`

    * Statistical models and least squares
    - `pip3 install statsmodels`

    * Manage and track numerical simulations and analyses
    - `pip3 install sumatra`
    - `pip3 install GitPython` for git addon of sumatra

    * Read Excel files
    - `pip3 install xlrd`
    - `pip install psutil`
    - `pip install memory_profiler`

    * Read/write DICOM files (medical software)
    - `pip3 install pydicom`
    - `pip install pydicom`

    (or git clone and install by python3 setup.py install)


    * * * * * * * * * * *

    Todo
    @@ -140,8 +73,9 @@ figure out best way to pass ars to pip so that the includes and libs of the home

    * PyYAML ... can't find libyaml. fuck.

    * CFFI (C calling foregin functions)
    - ???
    * CFFI (C calling foregin functions. Needed for PyOpenCL if built from source)
    - `brew install libffi`
    - `PKG_CONFIG_PATH=$(brew --prefix)/opt/libffi/lib/pkgconfig pip install cffi`

    * GUI toolkits:
    - todo: py3 ready?
  2. samueljohn revised this gist Apr 14, 2015. 1 changed file with 3 additions and 2 deletions.
    5 changes: 3 additions & 2 deletions samuels_python3_setup.md
    View file
    Original file line number Diff line number Diff line change
    @@ -7,8 +7,8 @@ Note, I switched to Python's 3.x `pyvenv` from virtualenv.

    ---------------------------------------------------------

    * Download XCode from the Mac App Store.
    * Install [Homebrew](https://github.com/Homebrew/homebrew)
    * Download XCode from the Mac App Store or get the command line tools.
    * Install [Homebrew](https://brew.sh)

    * Python3 itself:
    - `brew install python3`
    @@ -22,6 +22,7 @@ Note, I switched to Python's 3.x `pyvenv` from virtualenv.
    `pyvenv --copies --upgrade ~/Py`

    * To Upgrade the venv to a newer Python version
    - `rm ~/Py/bin/python*` this is kind of a hack
    - `pyvenv --upgrade --copies ~/Py`

    * NumPy and SciPy:
  3. samueljohn revised this gist Apr 14, 2015. 1 changed file with 3 additions and 0 deletions.
    3 changes: 3 additions & 0 deletions samuels_python3_setup.md
    View file
    Original file line number Diff line number Diff line change
    @@ -21,6 +21,9 @@ Note, I switched to Python's 3.x `pyvenv` from virtualenv.
    - To upgrade when homebrew has a newer Python:
    `pyvenv --copies --upgrade ~/Py`

    * To Upgrade the venv to a newer Python version
    - `pyvenv --upgrade --copies ~/Py`

    * NumPy and SciPy:
    - `brew tap homebrew/python`
    - `brew install gcc` (is this needed or already an dep?)
  4. samueljohn revised this gist Apr 14, 2015. 1 changed file with 2 additions and 1 deletion.
    3 changes: 2 additions & 1 deletion samuels_python3_setup.md
    View file
    Original file line number Diff line number Diff line change
    @@ -14,8 +14,9 @@ Note, I switched to Python's 3.x `pyvenv` from virtualenv.
    - `brew install python3`

    * Optional Pyvenv (virtual env)
    - `pyvenv --copies --system-site-packages ~/Py`
    - `pyvenv --copies ~/Py`
    to set up a virtual environment. Using the default symlink mode, has got an issue with brew's Python such that things installed with `pip` insdie of the venv are not found when started.
    - `pyvenv --system-site-packages ~/Py`
    - Don't forget to `source ~/Py/bin/activate`.
    - To upgrade when homebrew has a newer Python:
    `pyvenv --copies --upgrade ~/Py`
  5. samueljohn revised this gist Apr 8, 2015. 1 changed file with 4 additions and 3 deletions.
    7 changes: 4 additions & 3 deletions samuels_python3_setup.md
    View file
    Original file line number Diff line number Diff line change
    @@ -14,9 +14,8 @@ Note, I switched to Python's 3.x `pyvenv` from virtualenv.
    - `brew install python3`

    * Optional Pyvenv (virtual env)
    - `pyvenv --copies ~/Py`
    - `pyvenc --system-site-packages ~/Py`
    to set up a virtual environment. Using the default symlink mode, has got an issue with brew's Python such that things installed with `pip` insdie of the venv are not found when started. Also these two commands have to be entered one after each other so that `pip` is first created and then later we still can get access to the system site packages.
    - `pyvenv --copies --system-site-packages ~/Py`
    to set up a virtual environment. Using the default symlink mode, has got an issue with brew's Python such that things installed with `pip` insdie of the venv are not found when started.
    - Don't forget to `source ~/Py/bin/activate`.
    - To upgrade when homebrew has a newer Python:
    `pyvenv --copies --upgrade ~/Py`
    @@ -64,6 +63,8 @@ Note, I switched to Python's 3.x `pyvenv` from virtualenv.
    * Storing large data files as numpy arrays:
    - `brew install hdf5`
    - `pip3 install h5py`
    - `pip3 install numexpr`
    - `HDF5_DIR=$(brew --prefix)/Cellar/hdf5/<version>/ pip3 install tables`

    * OpenGL bindings:
    - `pip3 install pyopengl`
  6. samueljohn revised this gist Mar 25, 2015. 1 changed file with 3 additions and 2 deletions.
    5 changes: 3 additions & 2 deletions samuels_python3_setup.md
    View file
    Original file line number Diff line number Diff line change
    @@ -14,8 +14,9 @@ Note, I switched to Python's 3.x `pyvenv` from virtualenv.
    - `brew install python3`

    * Optional Pyvenv (virtual env)
    - `pyvenv --copies --system-site-packages ~/Py`
    to set up a virtual environment. Using the default symlink mode, has got an issue with brew's Python such that things installed with `pip` insdie of the venv are not found when started.
    - `pyvenv --copies ~/Py`
    - `pyvenc --system-site-packages ~/Py`
    to set up a virtual environment. Using the default symlink mode, has got an issue with brew's Python such that things installed with `pip` insdie of the venv are not found when started. Also these two commands have to be entered one after each other so that `pip` is first created and then later we still can get access to the system site packages.
    - Don't forget to `source ~/Py/bin/activate`.
    - To upgrade when homebrew has a newer Python:
    `pyvenv --copies --upgrade ~/Py`
  7. samueljohn created this gist Mar 19, 2015.
    153 changes: 153 additions & 0 deletions samuels_python3_setup.md
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,153 @@
    # Samuel's Python3 setup for Mac:

    This document describes in which way, what packages, and from where are
    installed on my Macs to get a decent Python 3.x, scipy/numpy & Co setup.

    Note, I switched to Python's 3.x `pyvenv` from virtualenv.

    ---------------------------------------------------------

    * Download XCode from the Mac App Store.
    * Install [Homebrew](https://github.com/Homebrew/homebrew)

    * Python3 itself:
    - `brew install python3`

    * Optional Pyvenv (virtual env)
    - `pyvenv --copies --system-site-packages ~/Py`
    to set up a virtual environment. Using the default symlink mode, has got an issue with brew's Python such that things installed with `pip` insdie of the venv are not found when started.
    - Don't forget to `source ~/Py/bin/activate`.
    - To upgrade when homebrew has a newer Python:
    `pyvenv --copies --upgrade ~/Py`

    * NumPy and SciPy:
    - `brew tap homebrew/python`
    - `brew install gcc` (is this needed or already an dep?)
    - `brew install numpy --with-openblas --with-python3 --without-python`
    - `brew install scipy --with-openblas --with-python3 --without-python`
    - `brew test numpy --verbose`
    - `brew test scipy --verbose`

    * Pandas (for tabular data and stats)
    - `pip3 install pandas`

    * Unit testing framework:
    - `pip3 install pytest`
    - `pip3 install nose`
    - `pip3 install coverage`
    - `pip3 install nose-progressive` Nose plug-in with progress bars.
    Run with `nosetests --with-progressive`

    * Progess bar
    - `pip3 install pyprind`

    * Debugging
    - `pip3 install debug`
    - `pip3 install pprint`

    * Interactive shell:
    - `brew install zeromq`
    - `pip3 install pyzmq`
    - `pip3 install --pre "ipython[all]"` (the "--pre" is only for the 3.0beta1)

    * Symbolics:
    - `pip3 install sympy`

    * Python Image Library:
    - `pip3 install pillow` to install the PIL

    * Scientific plotting, matplotlib:
    - `pip3 install matplotlib`
    - `pip3 install seaborn`

    * Storing large data files as numpy arrays:
    - `brew install hdf5`
    - `pip3 install h5py`

    * OpenGL bindings:
    - `pip3 install pyopengl`

    <!-- * View huge numpy arrays in OpenGL:
    - `pip3 install glumpy`
    - don't need this no longer: use vispy
    -->
    * VisPy: Most promising next gen visualizations
    - `git clone https://github.com/vispy/vispy.git`
    - `cd vispy`
    - `python3 setup.py install `

    * GPU computing with numpy arrays:
    - `git clone http://git.tiker.net/trees/pyopencl.git pyopencl`
    - `cd pyopencl`
    - `git submodule init`
    - `git submodule update`
    - `python3 configure.py --cl-enable-gl`
    - `python3 setup.py install`
    - `python3 install mako`

    * Cython for writing C-extensions in Python-like speak
    - `pip3 install cython `

    * Modular Data Processing Toolkit. (needs numpy):
    - `git clone git://github.com/mdp-toolkit/mdp-toolkit`
    - `cd mdp-toolkit`
    - `python3 setup.py install`

    * Memory profiling and shell tools:
    - `pip3 install psutil`
    - `pip3 install memory_profiler`

    * Machine learning
    - `pip3 install scikit-learn`

    * Scikit for image manipulation
    - `pip3 install scikit-image`

    * Fast sparse matrix package
    - `brew install suite-sparse`
    - `pip3 install scikit-umfpack`

    * Statistical models and least squares
    - `pip3 install statsmodels`

    * Manage and track numerical simulations and analyses
    - `pip3 install sumatra`
    - `pip3 install GitPython` for git addon of sumatra

    * Read Excel files
    - `pip3 install xlrd`

    * Read/write DICOM files (medical software)
    - `pip3 install pydicom`

    (or git clone and install by python3 setup.py install)

    * * * * * * * * * * *

    Todo

    Install kivy


    figure out best way to pass ars to pip so that the includes and libs of the homebrew stuff is found.

    * PyYAML ... can't find libyaml. fuck.

    * CFFI (C calling foregin functions)
    - ???

    * GUI toolkits:
    - todo: py3 ready?
    - `brew install qt`
    - `brew install pyside` or pyqt?
    - `brew install vtk --python`
    (the `--qt` option is currently not working on Xcode-only Macs)


    * Audio and sound generation:
    - `brew install liblo libsndfile portaudio portmidi`
    - `cd tmp`
    - `svn checkout http://pyo.googlecode.com/svn/trunk/ pyo`
    - `cd pyo`
    - `sudo python setup.py install --use-coreaudio --use-double`
    - use scikit-audiolab instead?