# ⚙️ **LOCAL**

Here you will find the instructions on how to setup ``BLIP`` **LOCALLY**. If you are using the Wilson Cluster, please see the [Wilson Cluster Setup](0.2.WCSetup.md) page.

## **Table of Contents**
1. [Docker Installation [RECOMENDED]](#-docker-container)
2. [Anaconda Installation](#anaconda-installation)
3. [Run BLIP](#blip)
 
## ⭐ **Docker Container**

The easiest way to run Blip is to grab the [docker container](https://hub.docker.com/repository/docker/infophysics/blip/general). First, you must install docker and start it up using the commands,
```bash
sudo apt-get update
sudo apt-get install docker.io
sudo systemctl start docker
sudo systemctl enable docker
```

Now we want to install the nvidia-container-toolkit, which can be done with the following (instructions are [here](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)).  First, set your distribution using the following command:
```bash
distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
      && curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
      && curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
            sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
            sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
```
If you are not using a LTS version of ubuntu, then you can do the following instead:
```bash
distribution=ubuntu18.04 \
      && curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
      && curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
            sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
            sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
```
You should then be able to download the container toolkit,
```bash
sudo apt-get update
sudo apt-get install nvidia-container-toolkit
```
Restart the docker service, and then you should be ready to grab a container,
```bash
sudo systemctl restart docker
```
Now grab the official pytorch container from docker by issuing the command:
```bash
docker pull pytorch/pytorch:2.0.1-cuda11.7-cudnn8-devel
```
The image can then be loaded by issuing
```bash
docker run -it --gpus all $(pwd):/workspace/ pytorch/pytorch:2.0.1-cuda11.7-cudnn8-devel bash
```
From inside the container, we can issue the following commands:
```bash
?
```

## **Anaconda Installation**

* **Environment YAML**

The easiet way to install is to create a conda environment dedicated to the API using the packages defined in ``environment_blip.yml``:
```bash
conda env create -f environment_blip.yml
conda activate blip
```
You can optionally add the flag ``-n <name>`` to specify a name for the environment.

* **MinkowskiEngine**

Due to the nature of the large datasets generated from LArTPC data, parts of Blip make use of SparseTensors in order to be more memory efficient, and to speed up overall performance.  SpraseTensors are handled through the *MinkowskiEngine* package, which interfaces with pytorch.
With the libopenblas dependency, we can install MinkowskiEngine via the following
```bash
sudo apt-get install libopenblas-dev
pip install -U git+https://github.com/NVIDIA/MinkowskiEngine -v --no-deps --install-option="--blas_include_dirs=${CONDA_PREFIX}/include" --install-option="--blas=openblas"
```
You may need to switch to a different version of GCC in order to install CUDA.  To do this, switch to the older version with:
```bash
sudo apt -y install gcc-11 g++-11
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-11 11
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-11 11
```
You'll then need to select the alternative version
```bash
$ sudo update-alternatives --config gcc

There are 2 choices for the alternative gcc (providing /usr/bin/gcc).

Selection    Path             Priority   Status
------------------------------------------------------------
* 0            /usr/bin/gcc-12   12        auto mode
1            /usr/bin/gcc-11   11        manual mode
2            /usr/bin/gcc-12   12        manual mode

Press <enter> to keep the current choice[*], or type selection number: 1
```

## BLIP
From the main folder of Blip you can run:
```bash
pip install .
```
which should install the API for you.


<!-- 
1. [Local Setup](#local-setup)
2. [Wilson Cluster Setup](#wilson-cluster-setup)

## **Local Setup**
First, you must install docker and start it up using the commands,
```bash
sudo apt-get update
sudo apt-get install docker.io
sudo systemctl start docker
sudo systemctl enable docker
```
Then, we can grab the blip container with the following:
```bash
docker pull infophysics/blip:latest
```

To run the image in interactive mode, issue the command
```bash
docker run blip
```

To run the image using the blip_display and gpus, there are various command line parameters that must be set,
```bash
docker run --it --gpus all -p 5006:5006 blip
```
where the *--gpus all* command tells docker to forward GPU access and *-p 5006:5006* port forwards the local 5006 port in the container to the local host 5006 port.  

To access the container with ssh support from the local host, do the following:
```bash
docker run -it --rm -e "USER_ID=$(id -u)" -e GROUP_ID="$(id -g)" \
      -v "$HOME/.ssh:/home/builder/.ssh:rw" \
      -v "$SSH_AUTH_SOCK:/ssh.socket" -e "SSH_AUTH_SOCK=/ssh.socket" \
      --gpus all -p 5006:5006 blip
```

## **Wilson Cluster Setup**
On the Wilson cluster, there exists some preinstalled software for [cloning and running containers](https://computing.fnal.gov/wilsoncluster/containers/).  To activate this software, run
```bash
module load apptainer
```
You'll then want to make sure that you point the image cache to a directory other than nashome, which has limited space.  This can be done by the following
```bash
export APPTAINER_CACHEDIR=/wclustre/my_project_dir/apptainer/.apptainer/cache
```
where *my_project_dir* is the desired cache location.  Once this is done, make a folder to store the blip image
```bash
cd /clustre/my_project_dir/
mkdir images
cd images
```
Then, clone the docker image
```bash
apptainer build blip.sif docker://infophysics/blip:latest
```

To run the image, simply do
```bash
apptainer run blip.sif
``` -->

```{note}
If you are planning to work on the Wilson Cluster check [Wilson Cluster Setup](0.2.WCSetup.md)