Skip to content

Instantly share code, notes, and snippets.

@winnerineast

winnerineast/tensorflow_macos_sse41_sse42_avx_avx2_fma_cuda.md

Forked from philster/tensorflow_macos_sse41_sse42_avx_avx2_fma_cuda.md
Last active August 1, 2020 05:56
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save winnerineast/05f63146e4b1e81ae08d14da2b38b11f to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save winnerineast/05f63146e4b1e81ae08d14da2b38b11f to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. winnerineast revised this gist Jul 22, 2018. 1 changed file with 20 additions and 33 deletions.
    53 changes: 20 additions & 33 deletions tensorflow_macos_sse41_sse42_avx_avx2_fma_cuda.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,6 +1,6 @@
    # Build TensorFlow 1.9 with AVX2/FMA on macOS High Sierra 10.13 (updated July 22, 2018)

    These instructions were inspired by [Mistobaan](https://github.com/mistobaan)'s [gist](https://gist.github.com/Mistobaan/dd32287eeb6859c6668d#file-tensorflow_cuda_osx-md), [ageitgey](https://github.com/ageitgey)'s [gist](https://gist.github.com/ageitgey/819a51afa4613649bd18#file-build-tensorflow-on-osx-with-nvidia-cuda-support-md), and [mattiasarro](https://github.com/mattiasarro)'s [tutorial](https://medium.com/@mattias.arro/installing-tensorflow-1-2-from-sources-with-gpu-support-on-macos-4f2c5cab8186), and []()
    These instructions were inspired by [Mistobaan](https://github.com/mistobaan)'s [gist](https://gist.github.com/Mistobaan/dd32287eeb6859c6668d#file-tensorflow_cuda_osx-md), [ageitgey](https://github.com/ageitgey)'s [gist](https://gist.github.com/ageitgey/819a51afa4613649bd18#file-build-tensorflow-on-osx-with-nvidia-cuda-support-md), and [mattiasarro](https://github.com/mattiasarro)'s [tutorial](https://medium.com/@mattias.arro/installing-tensorflow-1-2-from-sources-with-gpu-support-on-macos-4f2c5cab8186), and [Philster](https://gist.github.com/philster)'s [gist](https://gist.github.com/philster/042fabcf73f2269e6b1e6d28fbeeb7e3)

    ## Background
    I always encountered the following warnings when running my scripts using the precompiled TensorFlow Python package:
    @@ -16,29 +16,28 @@ I realized I can make these warnings go away by [compiling from source](https://
    ## Machine setup

    ### Hardware
    - Model: MacBook Pro (Retina, 15-inch, Mid 2014)
    - Model: MacBook Pro (Retina, 15-inch, Mid 2015)
    - Processor: 2.5 GHz Intel Core i7
    - Memory: 16 GB 1600 MHz DDR3
    - Graphics: Intel Iris Pro 1536 MB RAM + NVIDIA GeForce GT 750M 2048 MB RAM
    - Graphics: Intel Iris Pro 1536 MB RAM

    ### Software
    - OS: macOS Sierra 10.12.6
    - TensorFlow version: 1.3.1
    - OS: macOS Sierra 10.13.6
    - TensorFlow version: 1.9.0
    - Python version: 3.6.2 (conda)
    - Bazel version: 0.6.0-homebrew
    - CUDA/cuDNN version: 8.0/6.0
    - Bazel version: 0.15.2-homebrew

    ## Prerequisites

    ### macOS Sierra (10.12)
    ### macOS Sierra (10.13)

    I tested on macOS Sierra 10.12. It may also work on Yosemite (10.10) and El Capitan (10.11), but I have not verified.
    I tested on macOS High Sierra 10.13. It may also work on Yosemite (10.10) and El Capitan (10.11), but I have not verified.

    ### Xcode Command-Line Tools

    I successfully compiled using Xcode 8.2.1 (Refer to http://docs.nvidia.com/cuda/cuda-installation-guide-mac-os-x/index.html#system-requirements).
    I successfully compiled using Xcode 9.4.1 (Refer to http://docs.nvidia.com/cuda/cuda-installation-guide-mac-os-x/index.html#system-requirements).

    ### Disable SIP (System Integrity Protection) on Mac
    ### Disable SIP (System Integrity Protection) on Mac (I didn't encounter such error but I leave it be in case you have similar issue.)

    For some reason I had to [disable SIP](https://www.howtogeek.com/230424/how-to-disable-system-integrity-protection-on-a-mac-and-why-you-shouldnt/) in order for `bazel build` to build the TensorFlow pip package successfully. For security reasons, remember to re-enable SIP after your build.

    @@ -51,42 +50,30 @@ Note: Many steps were based on https://www.tensorflow.org/install/install_source
    - Install [conda](https://conda.io/miniconda.html) (I wanted a Python environment that will not mess with system Python. I downloaded Miniconda for Python 2.7 and intended to create a Python 3.6 environment)
    - Create and activate Python 3.6 environment
    ```bash
    conda create --name compiletf python=3
    conda create --name tf3.6 python=3
    # wheel 0.29.0 will already be installed
    source activate compiletf
    source activate tf3.6
    conda install numpy six
    # numpy 1.13.1 and six 1.10.0 will have been installed
    # numpy 1.14.5 and six 1.11.0 will have been installed
    ```
    Alternatively, you can do:
    ```bash
    conda create --name compiletf python=3 anaconda
    # numpy 1.12.1, six 1.10.0, and wheel 0.29.0 will already be installed
    source activate compiletf
    conda create --name tf3.6 python=3 anaconda
    # numpy 1.14.5, six 1.11.0, and wheel 0.39.0 will already be installed
    source activate tf3.6
    conda update numpy
    # numpy 1.13.1 will have been installed
    # numpy 1.14.5 will have been installed
    ```
    - Verify that the following packages are installed:
    - `six`
    - `numpy`
    - has to be at least `1.13` so you don't get a `ModuleNotFoundError: No module named 'numpy.lib.mixins'` error later on during `bazel build`
    - `wheel`
    - Install CUDA support prerequisites
    - Install GNU coreutils and swig
    ```bash
    brew install coreutils swig
    ```
    - Refer to [this](http://docs.nvidia.com/cuda/cuda-installation-guide-mac-os-x/index.html) for more detailed CUDA installation instructions.
    - Install [CUDA Toolkit 8.0](https://developer.nvidia.com/cuda-toolkit)
    - Install [cudNN 6.0](https://developer.nvidia.com/cudnn)
    - Set environment variable `DYLD_LIBRARY_PATH`
    ```bash
    export DYLD_LIBRARY_PATH=/usr/local/cuda/lib:$DYLD_LIBRARY_PATH
    ```
    - Clone the TensorFlow repository ([instructions](https://www.tensorflow.org/install/install_sources#clone_the_tensorflow_repository)): be sure to checkout the `r1.3` release
    - Clone the TensorFlow repository ([instructions](https://www.tensorflow.org/install/install_sources#clone_the_tensorflow_repository)): be sure to checkout the `r1.9.0` release
    ```bash
    git clone https://github.com/tensorflow/tensorflow
    cd tensorflow
    git checkout r1.3
    git checkout r1.9.0
    ```
    - Configure the installation
    ```bash
    @@ -138,7 +125,7 @@ Note: Many steps were based on https://www.tensorflow.org/install/install_source
    export CUDA_HOME=/usr/local/cuda
    export DYLD_LIBRARY_PATH=$CUDA_HOME/lib:$CUDA_HOME/extras/CUPTI/lib
    export LD_LIBRARY_PATH=$DYLD_LIBRARY_PATH
    bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-msse4.1 --copt=-msse4.2 --config=cuda --action_env LD_LIBRARY_PATH --action_env DYLD_LIBRARY_PATH --verbose_failures -k //tensorflow/tools/pip_package:build_pip_package
    bazel build -c opt --copt=-mavx2 --copt=-mfma --verbose_failures -k //tensorflow/tools/pip_package:build_pip_package
    ```
    - Refer to https://github.com/tensorflow/tensorflow/issues/6729 if you run into any other problems
    - Build the wheel (.whl) file
  2. winnerineast revised this gist Jul 22, 2018. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion tensorflow_macos_sse41_sse42_avx_avx2_fma_cuda.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,6 +1,6 @@
    # Build TensorFlow 1.9 with AVX2/FMA on macOS High Sierra 10.13 (updated July 22, 2018)

    These instructions were inspired by [Mistobaan](https://github.com/mistobaan)'s [gist](https://gist.github.com/Mistobaan/dd32287eeb6859c6668d#file-tensorflow_cuda_osx-md), [ageitgey](https://github.com/ageitgey)'s [gist](https://gist.github.com/ageitgey/819a51afa4613649bd18#file-build-tensorflow-on-osx-with-nvidia-cuda-support-md), and [mattiasarro](https://github.com/mattiasarro)'s [tutorial](https://medium.com/@mattias.arro/installing-tensorflow-1-2-from-sources-with-gpu-support-on-macos-4f2c5cab8186).
    These instructions were inspired by [Mistobaan](https://github.com/mistobaan)'s [gist](https://gist.github.com/Mistobaan/dd32287eeb6859c6668d#file-tensorflow_cuda_osx-md), [ageitgey](https://github.com/ageitgey)'s [gist](https://gist.github.com/ageitgey/819a51afa4613649bd18#file-build-tensorflow-on-osx-with-nvidia-cuda-support-md), and [mattiasarro](https://github.com/mattiasarro)'s [tutorial](https://medium.com/@mattias.arro/installing-tensorflow-1-2-from-sources-with-gpu-support-on-macos-4f2c5cab8186), and []()

    ## Background
    I always encountered the following warnings when running my scripts using the precompiled TensorFlow Python package:
  3. winnerineast revised this gist Jul 22, 2018. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion tensorflow_macos_sse41_sse42_avx_avx2_fma_cuda.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,4 @@
    # Build TensorFlow 1.3 with SSE4.1/SSE4.2/AVX/AVX2/FMA and NVIDIA CUDA support on macOS Sierra 10.12 (updated October 5, 2017)
    # Build TensorFlow 1.9 with AVX2/FMA on macOS High Sierra 10.13 (updated July 22, 2018)

    These instructions were inspired by [Mistobaan](https://github.com/mistobaan)'s [gist](https://gist.github.com/Mistobaan/dd32287eeb6859c6668d#file-tensorflow_cuda_osx-md), [ageitgey](https://github.com/ageitgey)'s [gist](https://gist.github.com/ageitgey/819a51afa4613649bd18#file-build-tensorflow-on-osx-with-nvidia-cuda-support-md), and [mattiasarro](https://github.com/mattiasarro)'s [tutorial](https://medium.com/@mattias.arro/installing-tensorflow-1-2-from-sources-with-gpu-support-on-macos-4f2c5cab8186).

  4. @philster philster revised this gist Oct 6, 2017. 1 changed file with 5 additions and 1 deletion.
    6 changes: 5 additions & 1 deletion tensorflow_macos_sse41_sse42_avx_avx2_fma_cuda.md
    View file
    Original file line number Diff line number Diff line change
    @@ -154,7 +154,11 @@ Note: Many steps were based on https://www.tensorflow.org/install/install_source
    ```bash
    cd ~
    ```
    - Invoke python interactive shell, type in the following script
    - Invoke python interactive shell
    ```bash
    python
    ```
    - Type in the following script
    ```python
    import tensorflow as tf
    with tf.device('/gpu:0'):
  5. @philster philster revised this gist Oct 6, 2017. 1 changed file with 35 additions and 4 deletions.
    39 changes: 35 additions & 4 deletions tensorflow_macos_sse41_sse42_avx_avx2_fma_cuda.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,4 @@
    # Build TensorFlow 1.3 with SSE4.1/SSE4.2/AVX/AVX2/FMA and NVIDIA CUDA support on macOS Sierra 10.12 (updated September 29, 2017)
    # Build TensorFlow 1.3 with SSE4.1/SSE4.2/AVX/AVX2/FMA and NVIDIA CUDA support on macOS Sierra 10.12 (updated October 5, 2017)

    These instructions were inspired by [Mistobaan](https://github.com/mistobaan)'s [gist](https://gist.github.com/Mistobaan/dd32287eeb6859c6668d#file-tensorflow_cuda_osx-md), [ageitgey](https://github.com/ageitgey)'s [gist](https://gist.github.com/ageitgey/819a51afa4613649bd18#file-build-tensorflow-on-osx-with-nvidia-cuda-support-md), and [mattiasarro](https://github.com/mattiasarro)'s [tutorial](https://medium.com/@mattias.arro/installing-tensorflow-1-2-from-sources-with-gpu-support-on-macos-4f2c5cab8186).

    @@ -23,9 +23,9 @@ I realized I can make these warnings go away by [compiling from source](https://

    ### Software
    - OS: macOS Sierra 10.12.6
    - TensorFlow version: 1.3.0
    - TensorFlow version: 1.3.1
    - Python version: 3.6.2 (conda)
    - Bazel version: 0.5.4-homebrew (0.6.0 currently breaking, see https://github.com/tensorflow/tensorflow/issues/13377)
    - Bazel version: 0.6.0-homebrew
    - CUDA/cuDNN version: 8.0/6.0

    ## Prerequisites
    @@ -147,8 +147,39 @@ Note: Many steps were based on https://www.tensorflow.org/install/install_source
    ```
    - Install the pip package
    ```bash
    pip install --upgrade --ignore-installed /tmp/tensorflow_pkg/tensorflow-1.3.0-cp36-cp36m-macosx_10_7_x86_64.whl
    pip install --upgrade --ignore-installed /tmp/tensorflow_pkg/tensorflow-1.3.1-cp36-cp36m-macosx_10_7_x86_64.whl
    ```
    - Validate your installation ([instructions](https://www.tensorflow.org/install/install_sources#validate_your_installation))
    - Change directory to any directory on your system other than the `tensorflow` subdirectory from which you ran `./configure`
    ```bash
    cd ~
    ```
    - Invoke python interactive shell, type in the following script
    ```python
    import tensorflow as tf
    with tf.device('/gpu:0'):
    a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
    b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
    c = tf.matmul(a, b)
    with tf.Session() as sess:
    print (sess.run(c))
    ```
    If you have a supported NVIDIA CUDA GPU, the script should run without a problem and display something similar to this:
    ```
    2017-10-05 22:22:27.025606: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:857] OS X does not support NUMA - returning NUMA node zero
    2017-10-05 22:22:27.025798: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 0 with properties:
    name: GeForce GT 750M
    major: 3 minor: 0 memoryClockRate (GHz) 0.9255
    pciBusID 0000:01:00.0
    Total memory: 2.00GiB
    Free memory: 873.57MiB
    2017-10-05 22:22:27.025819: I tensorflow/core/common_runtime/gpu/gpu_device.cc:976] DMA: 0
    2017-10-05 22:22:27.025826: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 0: Y
    2017-10-05 22:22:27.025842: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GT 750M, pci bus id: 0000:01:00.0)
    [[ 22. 28.]
    [ 49. 64.]]
    ```

    Have fun training your models!
  6. @philster philster revised this gist Sep 29, 2017. 1 changed file with 3 additions and 3 deletions.
    6 changes: 3 additions & 3 deletions tensorflow_macos_sse41_sse42_avx_avx2_fma_cuda.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,4 @@
    # Build TensorFlow 1.3 with SSE4.1/SSE4.2/AVX/AVX2/FMA and NVIDIA CUDA support on macOS Sierra 10.12 (updated September 18, 2017)
    # Build TensorFlow 1.3 with SSE4.1/SSE4.2/AVX/AVX2/FMA and NVIDIA CUDA support on macOS Sierra 10.12 (updated September 29, 2017)

    These instructions were inspired by [Mistobaan](https://github.com/mistobaan)'s [gist](https://gist.github.com/Mistobaan/dd32287eeb6859c6668d#file-tensorflow_cuda_osx-md), [ageitgey](https://github.com/ageitgey)'s [gist](https://gist.github.com/ageitgey/819a51afa4613649bd18#file-build-tensorflow-on-osx-with-nvidia-cuda-support-md), and [mattiasarro](https://github.com/mattiasarro)'s [tutorial](https://medium.com/@mattias.arro/installing-tensorflow-1-2-from-sources-with-gpu-support-on-macos-4f2c5cab8186).

    @@ -25,7 +25,7 @@ I realized I can make these warnings go away by [compiling from source](https://
    - OS: macOS Sierra 10.12.6
    - TensorFlow version: 1.3.0
    - Python version: 3.6.2 (conda)
    - Bazel version: 0.5.4-homebrew
    - Bazel version: 0.5.4-homebrew (0.6.0 currently breaking, see https://github.com/tensorflow/tensorflow/issues/13377)
    - CUDA/cuDNN version: 8.0/6.0

    ## Prerequisites
    @@ -138,7 +138,7 @@ Note: Many steps were based on https://www.tensorflow.org/install/install_source
    export CUDA_HOME=/usr/local/cuda
    export DYLD_LIBRARY_PATH=$CUDA_HOME/lib:$CUDA_HOME/extras/CUPTI/lib
    export LD_LIBRARY_PATH=$DYLD_LIBRARY_PATH
    bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-msse4.1 --copt=-msse4.2 --config=cuda --action_env LD_LIBRARY_PATH --action_env DYLD_LIBRARY_PATH -k //tensorflow/tools/pip_package:build_pip_package
    bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-msse4.1 --copt=-msse4.2 --config=cuda --action_env LD_LIBRARY_PATH --action_env DYLD_LIBRARY_PATH --verbose_failures -k //tensorflow/tools/pip_package:build_pip_package
    ```
    - Refer to https://github.com/tensorflow/tensorflow/issues/6729 if you run into any other problems
    - Build the wheel (.whl) file
  7. @philster philster revised this gist Sep 19, 2017. 1 changed file with 7 additions and 7 deletions.
    14 changes: 7 additions & 7 deletions tensorflow_macos_sse41_sse42_avx_avx2_fma_cuda.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,4 @@
    # Build TensorFlow 1.3 with SSE4.1/SSE4.2/AVX/AVX2/FMA and NVIDIA CUDA support on macOS Sierra 10.12
    # Build TensorFlow 1.3 with SSE4.1/SSE4.2/AVX/AVX2/FMA and NVIDIA CUDA support on macOS Sierra 10.12 (updated September 18, 2017)

    These instructions were inspired by [Mistobaan](https://github.com/mistobaan)'s [gist](https://gist.github.com/Mistobaan/dd32287eeb6859c6668d#file-tensorflow_cuda_osx-md), [ageitgey](https://github.com/ageitgey)'s [gist](https://gist.github.com/ageitgey/819a51afa4613649bd18#file-build-tensorflow-on-osx-with-nvidia-cuda-support-md), and [mattiasarro](https://github.com/mattiasarro)'s [tutorial](https://medium.com/@mattias.arro/installing-tensorflow-1-2-from-sources-with-gpu-support-on-macos-4f2c5cab8186).

    @@ -23,10 +23,10 @@ I realized I can make these warnings go away by [compiling from source](https://

    ### Software
    - OS: macOS Sierra 10.12.6
    - TensorFlow version: 1.3
    - TensorFlow version: 1.3.0
    - Python version: 3.6.2 (conda)
    - Bazel version: 0.5.3-homebrew
    - CUDA/cuDNN version: 8.0/5.1
    - Bazel version: 0.5.4-homebrew
    - CUDA/cuDNN version: 8.0/6.0

    ## Prerequisites

    @@ -77,7 +77,7 @@ Note: Many steps were based on https://www.tensorflow.org/install/install_source
    ```
    - Refer to [this](http://docs.nvidia.com/cuda/cuda-installation-guide-mac-os-x/index.html) for more detailed CUDA installation instructions.
    - Install [CUDA Toolkit 8.0](https://developer.nvidia.com/cuda-toolkit)
    - Install [cudNN 5.1](https://developer.nvidia.com/cudnn)
    - Install [cudNN 6.0](https://developer.nvidia.com/cudnn)
    - Set environment variable `DYLD_LIBRARY_PATH`
    ```bash
    export DYLD_LIBRARY_PATH=/usr/local/cuda/lib:$DYLD_LIBRARY_PATH
    @@ -121,8 +121,8 @@ Note: Many steps were based on https://www.tensorflow.org/install/install_source
    Please specify the CUDA SDK version you want to use, e.g. 7.0. [Leave empty to default to CUDA 8.0]:
    Please specify the location where CUDA 8.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
    Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]:
    Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 6.0]: 5
    Please specify the location where cuDNN 5 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
    Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 6.0]:
    Please specify the location where cuDNN 6 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
    ./configure: line 669: /usr/local/cuda/extras/demo_suite/deviceQuery: No such file or directory
    Please specify a list of comma-separated Cuda compute capabilities you want to build with.
    You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
  8. @philster philster revised this gist Sep 19, 2017. 1 changed file with 2 additions and 2 deletions.
    4 changes: 2 additions & 2 deletions tensorflow_macos_sse41_sse42_avx_avx2_fma_cuda.md
    View file
    Original file line number Diff line number Diff line change
    @@ -133,12 +133,12 @@ Note: Many steps were based on https://www.tensorflow.org/install/install_source
    Configuration finished
    ```
    - Comment out `linkopts = ["-lgomp"],` (line 112) in `tensorflow/third_party/gpus/cuda/BUILD.tpl` (Refer to https://medium.com/@mattias.arro/installing-tensorflow-1-2-from-sources-with-gpu-support-on-macos-4f2c5cab8186)
    - Build the pip package (reference: https://stackoverflow.com/questions/41293077/how-to-compile-tensorflow-with-sse4-2-and-avx-instructions)
    - Build the pip package (reference: https://stackoverflow.com/questions/41293077/how-to-compile-tensorflow-with-sse4-2-and-avx-instructions). It took around 35 minutes on my MacBook Pro.
    ```bash
    export CUDA_HOME=/usr/local/cuda
    export DYLD_LIBRARY_PATH=$CUDA_HOME/lib:$CUDA_HOME/extras/CUPTI/lib
    export LD_LIBRARY_PATH=$DYLD_LIBRARY_PATH
    bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-msse4.1 --copt=-msse4.2 --config=cuda --action_env PATH --action_env LD_LIBRARY_PATH --action_env DYLD_LIBRARY_PATH -k //tensorflow/tools/pip_package:build_pip_package
    bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-msse4.1 --copt=-msse4.2 --config=cuda --action_env LD_LIBRARY_PATH --action_env DYLD_LIBRARY_PATH -k //tensorflow/tools/pip_package:build_pip_package
    ```
    - Refer to https://github.com/tensorflow/tensorflow/issues/6729 if you run into any other problems
    - Build the wheel (.whl) file
  9. @philster philster revised this gist Aug 29, 2017. 1 changed file with 8 additions and 10 deletions.
    18 changes: 8 additions & 10 deletions tensorflow_macos_sse41_sse42_avx_avx2_fma_cuda.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,4 @@
    # Build TensorFlow 1.3 with SSE4.1/SSE4.2/AVX/AVX2/FMA (and optionally, NVIDIA CUDA support) on macOS Sierra 10.12
    # Build TensorFlow 1.3 with SSE4.1/SSE4.2/AVX/AVX2/FMA and NVIDIA CUDA support on macOS Sierra 10.12

    These instructions were inspired by [Mistobaan](https://github.com/mistobaan)'s [gist](https://gist.github.com/Mistobaan/dd32287eeb6859c6668d#file-tensorflow_cuda_osx-md), [ageitgey](https://github.com/ageitgey)'s [gist](https://gist.github.com/ageitgey/819a51afa4613649bd18#file-build-tensorflow-on-osx-with-nvidia-cuda-support-md), and [mattiasarro](https://github.com/mattiasarro)'s [tutorial](https://medium.com/@mattias.arro/installing-tensorflow-1-2-from-sources-with-gpu-support-on-macos-4f2c5cab8186).

    @@ -11,7 +11,7 @@ W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn'
    W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
    W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
    ```
    I realized I can make these warnings go away by [compiling from source](https://www.tensorflow.org/install/install_sources), in addition to improve training speed. It was not as easy and straightforward as I thought, but I finally succeeded in creating a working build. As a further step, I also created a build with [CUDA support](https://developer.nvidia.com/cuda-toolkit). Here I outline the steps I took, in the hopes it may benefit those who have encountered similar challenges.
    I realized I can make these warnings go away by [compiling from source](https://www.tensorflow.org/install/install_sources), in addition to improve training speed. It was not as easy and straightforward as I thought, but I finally succeeded in creating a working build. Here I outline the steps I took, in the hopes it may benefit those who have encountered similar challenges.

    ## Machine setup

    @@ -24,7 +24,7 @@ I realized I can make these warnings go away by [compiling from source](https://
    ### Software
    - OS: macOS Sierra 10.12.6
    - TensorFlow version: 1.3
    - Python version: 3.6.1 (Miniconda)
    - Python version: 3.6.2 (conda)
    - Bazel version: 0.5.3-homebrew
    - CUDA/cuDNN version: 8.0/5.1

    @@ -36,7 +36,7 @@ I tested on macOS Sierra 10.12. It may also work on Yosemite (10.10) and El Capi

    ### Xcode Command-Line Tools

    I successfully compiled with SSE4.2/AVX/FMA instructions using Xcode 8.3.3, but in order to compile for CUDA support I used Xcode 8.2.1 (Refer to http://docs.nvidia.com/cuda/cuda-installation-guide-mac-os-x/index.html#system-requirements).
    I successfully compiled using Xcode 8.2.1 (Refer to http://docs.nvidia.com/cuda/cuda-installation-guide-mac-os-x/index.html#system-requirements).

    ### Disable SIP (System Integrity Protection) on Mac

    @@ -68,9 +68,9 @@ Note: Many steps were based on https://www.tensorflow.org/install/install_source
    - Verify that the following packages are installed:
    - `six`
    - `numpy`
    - `numpy` has to be at least `1.13` so you don't get a `ModuleNotFoundError: No module named 'numpy.lib.mixins'` error later on during `bazel build`
    - has to be at least `1.13` so you don't get a `ModuleNotFoundError: No module named 'numpy.lib.mixins'` error later on during `bazel build`
    - `wheel`
    - (optional) Install Tensorflow for GPU (CUDA) prerequisites
    - Install CUDA support prerequisites
    - Install GNU coreutils and swig
    ```bash
    brew install coreutils swig
    @@ -80,7 +80,7 @@ Note: Many steps were based on https://www.tensorflow.org/install/install_source
    - Install [cudNN 5.1](https://developer.nvidia.com/cudnn)
    - Set environment variable `DYLD_LIBRARY_PATH`
    ```bash
    export DYLD_LIBRARY_PATH=/usr/local/cuda/lib:/Developer/NVIDIA/CUDA-8.0/lib:$DYLD_LIBRARY_PATH
    export DYLD_LIBRARY_PATH=/usr/local/cuda/lib:$DYLD_LIBRARY_PATH
    ```
    - Clone the TensorFlow repository ([instructions](https://www.tensorflow.org/install/install_sources#clone_the_tensorflow_repository)): be sure to checkout the `r1.3` release
    ```bash
    @@ -134,12 +134,10 @@ Note: Many steps were based on https://www.tensorflow.org/install/install_source
    ```
    - Comment out `linkopts = ["-lgomp"],` (line 112) in `tensorflow/third_party/gpus/cuda/BUILD.tpl` (Refer to https://medium.com/@mattias.arro/installing-tensorflow-1-2-from-sources-with-gpu-support-on-macos-4f2c5cab8186)
    - Build the pip package (reference: https://stackoverflow.com/questions/41293077/how-to-compile-tensorflow-with-sse4-2-and-avx-instructions)
    - Omit `--config=cuda`, the environment variable definitions, and `--action_env` options if you do not wish to build for CUDA support.
    ```bash
    export CUDA_HOME=/usr/local/cuda
    export DYLD_LIBRARY_PATH=$CUDA_HOME/lib:$CUDA_HOME/extras/CUPTI/lib:/Developer/NVIDIA/CUDA-8.0/lib
    export DYLD_LIBRARY_PATH=$CUDA_HOME/lib:$CUDA_HOME/extras/CUPTI/lib
    export LD_LIBRARY_PATH=$DYLD_LIBRARY_PATH
    export PATH=$DYLD_LIBRARY_PATH:$PATH
    bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-msse4.1 --copt=-msse4.2 --config=cuda --action_env PATH --action_env LD_LIBRARY_PATH --action_env DYLD_LIBRARY_PATH -k //tensorflow/tools/pip_package:build_pip_package
    ```
    - Refer to https://github.com/tensorflow/tensorflow/issues/6729 if you run into any other problems
  10. @philster philster revised this gist Aug 29, 2017. 1 changed file with 2 additions and 2 deletions.
    4 changes: 2 additions & 2 deletions tensorflow_macos_sse41_sse42_avx_avx2_fma_cuda.md
    View file
    Original file line number Diff line number Diff line change
    @@ -38,9 +38,9 @@ I tested on macOS Sierra 10.12. It may also work on Yosemite (10.10) and El Capi

    I successfully compiled with SSE4.2/AVX/FMA instructions using Xcode 8.3.3, but in order to compile for CUDA support I used Xcode 8.2.1 (Refer to http://docs.nvidia.com/cuda/cuda-installation-guide-mac-os-x/index.html#system-requirements).

    ### Disable SIP () on Mac
    ### Disable SIP (System Integrity Protection) on Mac

    For some reason I had to do this for `bazel build` to build the TensorFlow pip package successfully. See
    For some reason I had to [disable SIP](https://www.howtogeek.com/230424/how-to-disable-system-integrity-protection-on-a-mac-and-why-you-shouldnt/) in order for `bazel build` to build the TensorFlow pip package successfully. For security reasons, remember to re-enable SIP after your build.

    ## Steps

  11. @philster philster revised this gist Aug 29, 2017. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion tensorflow_macos_sse41_sse42_avx_avx2_fma_cuda.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,6 +1,6 @@
    # Build TensorFlow 1.3 with SSE4.1/SSE4.2/AVX/AVX2/FMA (and optionally, NVIDIA CUDA support) on macOS Sierra 10.12

    These instructions were inspired by [Mistobaan](https://github.com/mistobaan)'s [gist](https://gist.github.com/Mistobaan/dd32287eeb6859c6668d#file-tensorflow_cuda_osx-md) and [ageitgey](https://github.com/ageitgey)'s [gist](https://gist.github.com/ageitgey/819a51afa4613649bd18#file-build-tensorflow-on-osx-with-nvidia-cuda-support-md).
    These instructions were inspired by [Mistobaan](https://github.com/mistobaan)'s [gist](https://gist.github.com/Mistobaan/dd32287eeb6859c6668d#file-tensorflow_cuda_osx-md), [ageitgey](https://github.com/ageitgey)'s [gist](https://gist.github.com/ageitgey/819a51afa4613649bd18#file-build-tensorflow-on-osx-with-nvidia-cuda-support-md), and [mattiasarro](https://github.com/mattiasarro)'s [tutorial](https://medium.com/@mattias.arro/installing-tensorflow-1-2-from-sources-with-gpu-support-on-macos-4f2c5cab8186).

    ## Background
    I always encountered the following warnings when running my scripts using the precompiled TensorFlow Python package:
  12. @philster philster revised this gist Aug 29, 2017. 1 changed file with 79 additions and 80 deletions.
    159 changes: 79 additions & 80 deletions tensorflow_macos_sse41_sse42_avx_avx2_fma_cuda.md
    View file
    Original file line number Diff line number Diff line change
    @@ -50,27 +50,26 @@ Note: Many steps were based on https://www.tensorflow.org/install/install_source
    - Install [bazel](https://bazel.build/versions/master/docs/install.html#mac-os-x)
    - Install [conda](https://conda.io/miniconda.html) (I wanted a Python environment that will not mess with system Python. I downloaded Miniconda for Python 2.7 and intended to create a Python 3.6 environment)
    - Create and activate Python 3.6 environment
    ```bash
    conda create --name compiletf python=3
    source activate compiletf
    # wheel 0.29.0 will already be installed
    conda install numpy six
    # numpy 1.13.1 and six 1.10.0 will have been installed
    ```
    Alternatively, you can do:
    ```bash
    conda create --name compiletf python=3 anaconda
    # numpy 1.12.1, six 1.10.0, and wheel 0.29.0 will already be installed
    source activate compiletf
    ```
    ```bash
    conda create --name compiletf python=3
    # wheel 0.29.0 will already be installed
    source activate compiletf
    conda install numpy six
    # numpy 1.13.1 and six 1.10.0 will have been installed
    ```
    Alternatively, you can do:
    ```bash
    conda create --name compiletf python=3 anaconda
    # numpy 1.12.1, six 1.10.0, and wheel 0.29.0 will already be installed
    source activate compiletf
    conda update numpy
    # numpy 1.13.1 will have been installed
    ```
    - Verify that the following packages are installed:
    - `six`
    - `numpy`
    - `numpy` has to be at least `1.13` so you don't get a `ModuleNotFoundError: No module named 'numpy.lib.mixins'` error later on during `bazel build`
    - `wheel`
    - Update numpy to at least `1.13` (so that you don't get a `ModuleNotFoundError: No module named 'numpy.lib.mixins'` error later on during `bazel build`)
    ```bash
    conda update numpy
    ```
    - (optional) Install Tensorflow for GPU (CUDA) prerequisites
    - Install GNU coreutils and swig
    ```bash
    @@ -80,78 +79,78 @@ conda update numpy
    - Install [CUDA Toolkit 8.0](https://developer.nvidia.com/cuda-toolkit)
    - Install [cudNN 5.1](https://developer.nvidia.com/cudnn)
    - Set environment variable `DYLD_LIBRARY_PATH`
    ```bash
    export DYLD_LIBRARY_PATH=/usr/local/cuda/lib:/Developer/NVIDIA/CUDA-8.0/lib:$DYLD_LIBRARY_PATH
    ```
    ```bash
    export DYLD_LIBRARY_PATH=/usr/local/cuda/lib:/Developer/NVIDIA/CUDA-8.0/lib:$DYLD_LIBRARY_PATH
    ```
    - Clone the TensorFlow repository ([instructions](https://www.tensorflow.org/install/install_sources#clone_the_tensorflow_repository)): be sure to checkout the `r1.3` release
    ```bash
    git clone https://github.com/tensorflow/tensorflow
    cd tensorflow
    git checkout r1.3
    ```
    ```bash
    git clone https://github.com/tensorflow/tensorflow
    cd tensorflow
    git checkout r1.3
    ```
    - Configure the installation
    ```bash
    bazel clean
    ./configure
    ```
    ```bash
    bazel clean
    ./configure
    ```
    My `configure` settings (Enter `N` for CUDA support if you do not want CUDA support or do not have a NVIDIA GPU):
    ```
    Please specify the location of python. [Default is /Users/phil.wee/miniconda2/envs/compiletf/bin/python]:
    Found possible Python library paths:
    /Users/phil.wee/miniconda2/envs/compiletf/lib/python3.6/site-packages
    Please input the desired Python library path to use. Default is [/Users/phil.wee/miniconda2/envs/compiletf/lib/python3.6/site-packages]
    Using python library path: /Users/phil.wee/miniconda2/envs/compiletf/lib/python3.6/site-packages
    Do you wish to build TensorFlow with MKL support? [y/N]
    No MKL support will be enabled for TensorFlow
    Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]:
    Do you wish to build TensorFlow with Google Cloud Platform support? [y/N]
    No Google Cloud Platform support will be enabled for TensorFlow
    Do you wish to build TensorFlow with Hadoop File System support? [y/N]
    No Hadoop File System support will be enabled for TensorFlow
    Do you wish to build TensorFlow with the XLA just-in-time compiler (experimental)? [y/N]
    No XLA support will be enabled for TensorFlow
    Do you wish to build TensorFlow with VERBS support? [y/N]
    No VERBS support will be enabled for TensorFlow
    Do you wish to build TensorFlow with OpenCL support? [y/N]
    No OpenCL support will be enabled for TensorFlow
    Do you wish to build TensorFlow with CUDA support? [y/N] Y
    CUDA support will be enabled for TensorFlow
    Do you want to use clang as CUDA compiler? [y/N]
    nvcc will be used as CUDA compiler
    Please specify the CUDA SDK version you want to use, e.g. 7.0. [Leave empty to default to CUDA 8.0]:
    Please specify the location where CUDA 8.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
    Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]:
    Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 6.0]: 5
    Please specify the location where cuDNN 5 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
    ./configure: line 669: /usr/local/cuda/extras/demo_suite/deviceQuery: No such file or directory
    Please specify a list of comma-separated Cuda compute capabilities you want to build with.
    You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
    Please note that each additional compute capability significantly increases your build time and binary size.
    [Default is: "3.5,5.2"]: 3.0
    Do you wish to build TensorFlow with MPI support? [y/N]
    MPI support will not be enabled for TensorFlow
    Configuration finished
    ```
    ```
    Please specify the location of python. [Default is /Users/phil.wee/miniconda2/envs/compiletf/bin/python]:
    Found possible Python library paths:
    /Users/phil.wee/miniconda2/envs/compiletf/lib/python3.6/site-packages
    Please input the desired Python library path to use. Default is [/Users/phil.wee/miniconda2/envs/compiletf/lib/python3.6/site-packages]
    Using python library path: /Users/phil.wee/miniconda2/envs/compiletf/lib/python3.6/site-packages
    Do you wish to build TensorFlow with MKL support? [y/N]
    No MKL support will be enabled for TensorFlow
    Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]:
    Do you wish to build TensorFlow with Google Cloud Platform support? [y/N]
    No Google Cloud Platform support will be enabled for TensorFlow
    Do you wish to build TensorFlow with Hadoop File System support? [y/N]
    No Hadoop File System support will be enabled for TensorFlow
    Do you wish to build TensorFlow with the XLA just-in-time compiler (experimental)? [y/N]
    No XLA support will be enabled for TensorFlow
    Do you wish to build TensorFlow with VERBS support? [y/N]
    No VERBS support will be enabled for TensorFlow
    Do you wish to build TensorFlow with OpenCL support? [y/N]
    No OpenCL support will be enabled for TensorFlow
    Do you wish to build TensorFlow with CUDA support? [y/N] Y
    CUDA support will be enabled for TensorFlow
    Do you want to use clang as CUDA compiler? [y/N]
    nvcc will be used as CUDA compiler
    Please specify the CUDA SDK version you want to use, e.g. 7.0. [Leave empty to default to CUDA 8.0]:
    Please specify the location where CUDA 8.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
    Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]:
    Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 6.0]: 5
    Please specify the location where cuDNN 5 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
    ./configure: line 669: /usr/local/cuda/extras/demo_suite/deviceQuery: No such file or directory
    Please specify a list of comma-separated Cuda compute capabilities you want to build with.
    You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
    Please note that each additional compute capability significantly increases your build time and binary size.
    [Default is: "3.5,5.2"]: 3.0
    Do you wish to build TensorFlow with MPI support? [y/N]
    MPI support will not be enabled for TensorFlow
    Configuration finished
    ```
    - Comment out `linkopts = ["-lgomp"],` (line 112) in `tensorflow/third_party/gpus/cuda/BUILD.tpl` (Refer to https://medium.com/@mattias.arro/installing-tensorflow-1-2-from-sources-with-gpu-support-on-macos-4f2c5cab8186)
    - Build the pip package (reference: https://stackoverflow.com/questions/41293077/how-to-compile-tensorflow-with-sse4-2-and-avx-instructions)
    - Omit `--config=cuda`, the environment variable definitions, and `--action_env` options if you do not wish to build for CUDA support.
    ```bash
    export CUDA_HOME=/usr/local/cuda
    export DYLD_LIBRARY_PATH=$CUDA_HOME/lib:$CUDA_HOME/extras/CUPTI/lib:/Developer/NVIDIA/CUDA-8.0/lib
    export LD_LIBRARY_PATH=$DYLD_LIBRARY_PATH
    export PATH=$DYLD_LIBRARY_PATH:$PATH
    bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-msse4.1 --copt=-msse4.2 --config=cuda --action_env PATH --action_env LD_LIBRARY_PATH --action_env DYLD_LIBRARY_PATH -k //tensorflow/tools/pip_package:build_pip_package
    ```
    ```bash
    export CUDA_HOME=/usr/local/cuda
    export DYLD_LIBRARY_PATH=$CUDA_HOME/lib:$CUDA_HOME/extras/CUPTI/lib:/Developer/NVIDIA/CUDA-8.0/lib
    export LD_LIBRARY_PATH=$DYLD_LIBRARY_PATH
    export PATH=$DYLD_LIBRARY_PATH:$PATH
    bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-msse4.1 --copt=-msse4.2 --config=cuda --action_env PATH --action_env LD_LIBRARY_PATH --action_env DYLD_LIBRARY_PATH -k //tensorflow/tools/pip_package:build_pip_package
    ```
    - Refer to https://github.com/tensorflow/tensorflow/issues/6729 if you run into any other problems
    - Build the wheel (.whl) file
    ```bash
    bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
    ```
    ```bash
    bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
    ```
    - Install the pip package
    ```bash
    pip install --upgrade --ignore-installed /tmp/tensorflow_pkg/tensorflow-1.3.0-cp36-cp36m-macosx_10_7_x86_64.whl
    ```
    ```bash
    pip install --upgrade --ignore-installed /tmp/tensorflow_pkg/tensorflow-1.3.0-cp36-cp36m-macosx_10_7_x86_64.whl
    ```
    - Validate your installation ([instructions](https://www.tensorflow.org/install/install_sources#validate_your_installation))

    Have fun training your models!
  13. @philster philster revised this gist Aug 29, 2017. 1 changed file with 4 additions and 4 deletions.
    8 changes: 4 additions & 4 deletions tensorflow_macos_sse41_sse42_avx_avx2_fma_cuda.md
    View file
    Original file line number Diff line number Diff line change
    @@ -72,10 +72,10 @@ source activate compiletf
    conda update numpy
    ```
    - (optional) Install Tensorflow for GPU (CUDA) prerequisites
    - Install GNU coreutils
    ```bash
    brew install coreutils
    ```
    - Install GNU coreutils and swig
    ```bash
    brew install coreutils swig
    ```
    - Refer to [this](http://docs.nvidia.com/cuda/cuda-installation-guide-mac-os-x/index.html) for more detailed CUDA installation instructions.
    - Install [CUDA Toolkit 8.0](https://developer.nvidia.com/cuda-toolkit)
    - Install [cudNN 5.1](https://developer.nvidia.com/cudnn)
  14. @philster philster revised this gist Aug 29, 2017. 1 changed file with 24 additions and 14 deletions.
    38 changes: 24 additions & 14 deletions tensorflow_macos_sse41_sse42_avx_avx2_fma_cuda.md
    View file
    Original file line number Diff line number Diff line change
    @@ -38,49 +38,59 @@ I tested on macOS Sierra 10.12. It may also work on Yosemite (10.10) and El Capi

    I successfully compiled with SSE4.2/AVX/FMA instructions using Xcode 8.3.3, but in order to compile for CUDA support I used Xcode 8.2.1 (Refer to http://docs.nvidia.com/cuda/cuda-installation-guide-mac-os-x/index.html#system-requirements).

    ### Disable SIP () on Mac

    For some reason I had to do this for `bazel build` to build the TensorFlow pip package successfully. See

    ## Steps

    Note: Many steps were based on https://www.tensorflow.org/install/install_sources ; I just happened to have a slightly different order that worked out for me.

    - Install [homebrew](https://brew.sh/)
    - Install [bazel](https://bazel.build/versions/master/docs/install.html#mac-os-x)
    - Install [conda](https://conda.io/miniconda.html) (I wanted a Python environment that will not mess with system Python. I downloaded Miniconda for Python 2.7 and intended to create a Python 3.6 environment)
    - Create Python 3.6 environment
    - Create and activate Python 3.6 environment
    ```bash
    conda create --name compiletf python=3
    source activate compiletf
    # wheel 0.29.0 will already be installed
    conda install numpy six
    # numpy 1.13.1 and six 1.10.0 will have been installed
    ```
    Alternatively, you can do:
    ```bash
    conda create --name compiletf python=3 anaconda
    ```
    - Activate the Python environment
    ```
    # numpy 1.12.1, six 1.10.0, and wheel 0.29.0 will already be installed
    source activate compiletf
    ```
    - Verify that the following packages are installed:
    - `six`
    - `numpy`
    - `wheel`
    - Update numpy to at least `1.13` (so that you don't get a `ModuleNotFoundError: No module named 'numpy.lib.mixins'` error later on during `bazel build`)
    ```
    ```bash
    conda update numpy
    ```
    - (optional) Install Tensorflow for GPU (CUDA) prerequisites
    - Install GNU coreutils
    ```
    ```bash
    brew install coreutils
    ```
    - Refer to [this](http://docs.nvidia.com/cuda/cuda-installation-guide-mac-os-x/index.html) for more detailed CUDA installation instructions.
    - Install [CUDA Toolkit 8.0](https://developer.nvidia.com/cuda-toolkit)
    - Install [cudNN 5.1](https://developer.nvidia.com/cudnn)
    - Set environment variable `DYLD_LIBRARY_PATH`
    ```
    export DYLD_LIBRARY_PATH=/usr/local/cuda/lib:$DYLD_LIBRARY_PATH
    ```bash
    export DYLD_LIBRARY_PATH=/usr/local/cuda/lib:/Developer/NVIDIA/CUDA-8.0/lib:$DYLD_LIBRARY_PATH
    ```
    - Clone the TensorFlow repository ([instructions](https://www.tensorflow.org/install/install_sources#clone_the_tensorflow_repository)): be sure to checkout the `r1.3` release
    ```
    ```bash
    git clone https://github.com/tensorflow/tensorflow
    cd tensorflow
    git checkout r1.3
    ```
    - Configure the installation
    ```
    ```bash
    bazel clean
    ./configure
    ```
    @@ -126,20 +136,20 @@ Configuration finished
    - Comment out `linkopts = ["-lgomp"],` (line 112) in `tensorflow/third_party/gpus/cuda/BUILD.tpl` (Refer to https://medium.com/@mattias.arro/installing-tensorflow-1-2-from-sources-with-gpu-support-on-macos-4f2c5cab8186)
    - Build the pip package (reference: https://stackoverflow.com/questions/41293077/how-to-compile-tensorflow-with-sse4-2-and-avx-instructions)
    - Omit `--config=cuda`, the environment variable definitions, and `--action_env` options if you do not wish to build for CUDA support.
    ```
    ```bash
    export CUDA_HOME=/usr/local/cuda
    export DYLD_LIBRARY_PATH=$CUDA_HOME/lib:$CUDA_HOME:$CUDA_HOME/extras/CUPTI/lib
    export DYLD_LIBRARY_PATH=$CUDA_HOME/lib:$CUDA_HOME/extras/CUPTI/lib:/Developer/NVIDIA/CUDA-8.0/lib
    export LD_LIBRARY_PATH=$DYLD_LIBRARY_PATH
    export PATH=$DYLD_LIBRARY_PATH:$PATH
    bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-msse4.1 --copt=-msse4.2 --config=cuda --action_env PATH --action_env LD_LIBRARY_PATH --action_env DYLD_LIBRARY_PATH -k //tensorflow/tools/pip_package:build_pip_package
    ```
    - Refer to https://github.com/tensorflow/tensorflow/issues/6729 if you run into any other problems
    - Build the wheel (.whl) file
    ```
    ```bash
    bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
    ```
    - Install the pip package
    ```
    ```bash
    pip install --upgrade --ignore-installed /tmp/tensorflow_pkg/tensorflow-1.3.0-cp36-cp36m-macosx_10_7_x86_64.whl
    ```
    - Validate your installation ([instructions](https://www.tensorflow.org/install/install_sources#validate_your_installation))
  15. @philster philster renamed this gist Aug 29, 2017. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion tensorflow_macos_sse42_avx_fma_cuda.md → ...ow_macos_sse41_sse42_avx_avx2_fma_cuda.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,4 @@
    # Build TensorFlow 1.3 with SSE4.2/AVX/FMA (and optionally, NVIDIA CUDA support) on macOS Sierra 10.12
    # Build TensorFlow 1.3 with SSE4.1/SSE4.2/AVX/AVX2/FMA (and optionally, NVIDIA CUDA support) on macOS Sierra 10.12

    These instructions were inspired by [Mistobaan](https://github.com/mistobaan)'s [gist](https://gist.github.com/Mistobaan/dd32287eeb6859c6668d#file-tensorflow_cuda_osx-md) and [ageitgey](https://github.com/ageitgey)'s [gist](https://gist.github.com/ageitgey/819a51afa4613649bd18#file-build-tensorflow-on-osx-with-nvidia-cuda-support-md).

  16. @philster philster revised this gist Aug 28, 2017. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion tensorflow_macos_sse42_avx_fma_cuda.md
    View file
    Original file line number Diff line number Diff line change
    @@ -70,7 +70,7 @@ brew install coreutils
    - Install [CUDA Toolkit 8.0](https://developer.nvidia.com/cuda-toolkit)
    - Install [cudNN 5.1](https://developer.nvidia.com/cudnn)
    - Set environment variable `DYLD_LIBRARY_PATH`
    ```
    ```
    export DYLD_LIBRARY_PATH=/usr/local/cuda/lib:$DYLD_LIBRARY_PATH
    ```
    - Clone the TensorFlow repository ([instructions](https://www.tensorflow.org/install/install_sources#clone_the_tensorflow_repository)): be sure to checkout the `r1.3` release
  17. @philster philster revised this gist Aug 28, 2017. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion tensorflow_macos_sse42_avx_fma_cuda.md
    View file
    Original file line number Diff line number Diff line change
    @@ -70,7 +70,7 @@ brew install coreutils
    - Install [CUDA Toolkit 8.0](https://developer.nvidia.com/cuda-toolkit)
    - Install [cudNN 5.1](https://developer.nvidia.com/cudnn)
    - Set environment variable `DYLD_LIBRARY_PATH`
    ```
    ```
    export DYLD_LIBRARY_PATH=/usr/local/cuda/lib:$DYLD_LIBRARY_PATH
    ```
    - Clone the TensorFlow repository ([instructions](https://www.tensorflow.org/install/install_sources#clone_the_tensorflow_repository)): be sure to checkout the `r1.3` release
  18. @philster philster created this gist Aug 28, 2017.
    147 changes: 147 additions & 0 deletions tensorflow_macos_sse42_avx_fma_cuda.md
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,147 @@
    # Build TensorFlow 1.3 with SSE4.2/AVX/FMA (and optionally, NVIDIA CUDA support) on macOS Sierra 10.12

    These instructions were inspired by [Mistobaan](https://github.com/mistobaan)'s [gist](https://gist.github.com/Mistobaan/dd32287eeb6859c6668d#file-tensorflow_cuda_osx-md) and [ageitgey](https://github.com/ageitgey)'s [gist](https://gist.github.com/ageitgey/819a51afa4613649bd18#file-build-tensorflow-on-osx-with-nvidia-cuda-support-md).

    ## Background
    I always encountered the following warnings when running my scripts using the precompiled TensorFlow Python package:
    ```
    W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
    W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
    W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
    W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
    W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
    ```
    I realized I can make these warnings go away by [compiling from source](https://www.tensorflow.org/install/install_sources), in addition to improve training speed. It was not as easy and straightforward as I thought, but I finally succeeded in creating a working build. As a further step, I also created a build with [CUDA support](https://developer.nvidia.com/cuda-toolkit). Here I outline the steps I took, in the hopes it may benefit those who have encountered similar challenges.

    ## Machine setup

    ### Hardware
    - Model: MacBook Pro (Retina, 15-inch, Mid 2014)
    - Processor: 2.5 GHz Intel Core i7
    - Memory: 16 GB 1600 MHz DDR3
    - Graphics: Intel Iris Pro 1536 MB RAM + NVIDIA GeForce GT 750M 2048 MB RAM

    ### Software
    - OS: macOS Sierra 10.12.6
    - TensorFlow version: 1.3
    - Python version: 3.6.1 (Miniconda)
    - Bazel version: 0.5.3-homebrew
    - CUDA/cuDNN version: 8.0/5.1

    ## Prerequisites

    ### macOS Sierra (10.12)

    I tested on macOS Sierra 10.12. It may also work on Yosemite (10.10) and El Capitan (10.11), but I have not verified.

    ### Xcode Command-Line Tools

    I successfully compiled with SSE4.2/AVX/FMA instructions using Xcode 8.3.3, but in order to compile for CUDA support I used Xcode 8.2.1 (Refer to http://docs.nvidia.com/cuda/cuda-installation-guide-mac-os-x/index.html#system-requirements).

    ## Steps

    Note: Many steps were based on https://www.tensorflow.org/install/install_sources ; I just happened to have a slightly different order that worked out for me.

    - Install [homebrew](https://brew.sh/)
    - Install [bazel](https://bazel.build/versions/master/docs/install.html#mac-os-x)
    - Install [conda](https://conda.io/miniconda.html) (I wanted a Python environment that will not mess with system Python. I downloaded Miniconda for Python 2.7 and intended to create a Python 3.6 environment)
    - Create Python 3.6 environment
    ```
    conda create --name compiletf python=3 anaconda
    ```
    - Activate the Python environment
    ```
    source activate compiletf
    ```
    - Verify that the following packages are installed:
    - `six`
    - `numpy`
    - `wheel`
    - Update numpy to at least `1.13` (so that you don't get a `ModuleNotFoundError: No module named 'numpy.lib.mixins'` error later on during `bazel build`)
    ```
    conda update numpy
    ```
    - (optional) Install Tensorflow for GPU (CUDA) prerequisites
    - Install GNU coreutils
    ```
    brew install coreutils
    ```
    - Refer to [this](http://docs.nvidia.com/cuda/cuda-installation-guide-mac-os-x/index.html) for more detailed CUDA installation instructions.
    - Install [CUDA Toolkit 8.0](https://developer.nvidia.com/cuda-toolkit)
    - Install [cudNN 5.1](https://developer.nvidia.com/cudnn)
    - Set environment variable `DYLD_LIBRARY_PATH`
    ```
    export DYLD_LIBRARY_PATH=/usr/local/cuda/lib:$DYLD_LIBRARY_PATH
    ```
    - Clone the TensorFlow repository ([instructions](https://www.tensorflow.org/install/install_sources#clone_the_tensorflow_repository)): be sure to checkout the `r1.3` release
    ```
    git clone https://github.com/tensorflow/tensorflow
    cd tensorflow
    git checkout r1.3
    ```
    - Configure the installation
    ```
    bazel clean
    ./configure
    ```
    My `configure` settings (Enter `N` for CUDA support if you do not want CUDA support or do not have a NVIDIA GPU):
    ```
    Please specify the location of python. [Default is /Users/phil.wee/miniconda2/envs/compiletf/bin/python]:
    Found possible Python library paths:
    /Users/phil.wee/miniconda2/envs/compiletf/lib/python3.6/site-packages
    Please input the desired Python library path to use. Default is [/Users/phil.wee/miniconda2/envs/compiletf/lib/python3.6/site-packages]
    Using python library path: /Users/phil.wee/miniconda2/envs/compiletf/lib/python3.6/site-packages
    Do you wish to build TensorFlow with MKL support? [y/N]
    No MKL support will be enabled for TensorFlow
    Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]:
    Do you wish to build TensorFlow with Google Cloud Platform support? [y/N]
    No Google Cloud Platform support will be enabled for TensorFlow
    Do you wish to build TensorFlow with Hadoop File System support? [y/N]
    No Hadoop File System support will be enabled for TensorFlow
    Do you wish to build TensorFlow with the XLA just-in-time compiler (experimental)? [y/N]
    No XLA support will be enabled for TensorFlow
    Do you wish to build TensorFlow with VERBS support? [y/N]
    No VERBS support will be enabled for TensorFlow
    Do you wish to build TensorFlow with OpenCL support? [y/N]
    No OpenCL support will be enabled for TensorFlow
    Do you wish to build TensorFlow with CUDA support? [y/N] Y
    CUDA support will be enabled for TensorFlow
    Do you want to use clang as CUDA compiler? [y/N]
    nvcc will be used as CUDA compiler
    Please specify the CUDA SDK version you want to use, e.g. 7.0. [Leave empty to default to CUDA 8.0]:
    Please specify the location where CUDA 8.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
    Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]:
    Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 6.0]: 5
    Please specify the location where cuDNN 5 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
    ./configure: line 669: /usr/local/cuda/extras/demo_suite/deviceQuery: No such file or directory
    Please specify a list of comma-separated Cuda compute capabilities you want to build with.
    You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
    Please note that each additional compute capability significantly increases your build time and binary size.
    [Default is: "3.5,5.2"]: 3.0
    Do you wish to build TensorFlow with MPI support? [y/N]
    MPI support will not be enabled for TensorFlow
    Configuration finished
    ```
    - Comment out `linkopts = ["-lgomp"],` (line 112) in `tensorflow/third_party/gpus/cuda/BUILD.tpl` (Refer to https://medium.com/@mattias.arro/installing-tensorflow-1-2-from-sources-with-gpu-support-on-macos-4f2c5cab8186)
    - Build the pip package (reference: https://stackoverflow.com/questions/41293077/how-to-compile-tensorflow-with-sse4-2-and-avx-instructions)
    - Omit `--config=cuda`, the environment variable definitions, and `--action_env` options if you do not wish to build for CUDA support.
    ```
    export CUDA_HOME=/usr/local/cuda
    export DYLD_LIBRARY_PATH=$CUDA_HOME/lib:$CUDA_HOME:$CUDA_HOME/extras/CUPTI/lib
    export LD_LIBRARY_PATH=$DYLD_LIBRARY_PATH
    export PATH=$DYLD_LIBRARY_PATH:$PATH
    bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-msse4.1 --copt=-msse4.2 --config=cuda --action_env PATH --action_env LD_LIBRARY_PATH --action_env DYLD_LIBRARY_PATH -k //tensorflow/tools/pip_package:build_pip_package
    ```
    - Refer to https://github.com/tensorflow/tensorflow/issues/6729 if you run into any other problems
    - Build the wheel (.whl) file
    ```
    bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
    ```
    - Install the pip package
    ```
    pip install --upgrade --ignore-installed /tmp/tensorflow_pkg/tensorflow-1.3.0-cp36-cp36m-macosx_10_7_x86_64.whl
    ```
    - Validate your installation ([instructions](https://www.tensorflow.org/install/install_sources#validate_your_installation))

    Have fun training your models!