Created
August 28, 2019 17:55
-
-
Save scalixte-mdsol/754ad97551d21aaa00db3bcf2bc7b75a to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Install Hadoop with Native Libraries on Max OS X | |
| ## Pre-requisites | |
| * maven | |
| * gcc | |
| * GNU Autools Chain: autoconf, automake, libtool | |
| * cmake | |
| * snappy | |
| * gzip | |
| * bzip2 | |
| * protobuf (use this: homebrew/versions/protobuf250) | |
| - homebrew does not include this version `2.5.0`. I'm working on a tap | |
| - I install via make | |
| * zlib | |
| * openssl | |
| ## Install the pre-requisites | |
| > I did it in multiple parts | |
| ### Install core pre-requisites | |
| You need them anyway, and there may be already installed. So verify if you're not sure. | |
| `brew install maven gcc autoconf automake libtool cmake snappy gzip bzip2 zlib openssl` | |
| Once confirmed that `openssl` is present with `which openssl` | |
| - Go to $HOME/.bash_profile | |
| - Insert these two lines: | |
| ``` | |
| export OPENSSL_ROOT_DIR=/usr/local/Cellar/openssl/1.0.2s | |
| export OPENSSL_INCLUDE_DIR=$OPENSSL_ROOT_DIR/include | |
| ``` | |
| > Note your OpenSSL path may be different. | |
| ### Install protobuf | |
| `brew install homebrew/versions/protobuf250` | |
| > Note if this does not work for you, then you need to follow the steps below | |
| - Clone the repo | |
| * `git clone [email protected]:protocolbuffers/protobuf.git` | |
| - Go to folder: `cd protobuf/` | |
| - checkout the tag: `git checkout v2.5.0` | |
| - Create configure file | |
| * Execute `autoconf` | |
| * If it fails, and only then execute: `autoreconf -fvi` | |
| - Execute configure profile: `configure --prefix=/usr/local/` | |
| - Install the library | |
| * make | |
| * make install | |
| - Verify installation | |
| * which protoc | |
| ## Install Hadoop | |
| ### Getting source files | |
| #### From github | |
| * From a location of choice | |
| * Clone the project: `git clone [email protected]:apache/hadoop.git` | |
| * Go to hadoop: `cd hadoop` | |
| * Go to branch: `git checkout branch-2.8.5` | |
| ### From apache site | |
| * Go to `https://hadoop.apache.org/releases.html` | |
| * Follow instructions | |
| * Extract archive to folder | |
| * Go to folder | |
| ### Installing hadoop | |
| ``` | |
| mvn package -Pdist,native -DskipTests -Dtar | |
| ``` | |
| > You may want to extract more information from the install process including errors, so use: | |
| ``` | |
| mvn package -Pdist,native -DskipTests -Dtar -e | |
| ``` |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment