. /home/SPACK2019/share/spack/setup-env.sh
spack list  # lists many modules
spack info python  # lists available python versions
spack load python  # defaults to 3.7
python -m venv python37_venv
source python37_venv/bin/activate
module purge # sergei says 'or things get weird' ??still needed??
which pip   # sanity check - this should refer to the 'pip' form the virtual env
pip install --upgrade pip
pip install jupyterlab scipy h5py seaborn astropy pandas 
pip install lalsuite # if desired

If you're using SpEC, you might also want to get diagnose-spec installed:

cd ~
git clone git@github.com:nilsleiffischer/diagnose-spec.git
which pip   # sanity check - this MUST refer to the 'pip' form the virtual env
pip install -e ~/diagnose-spec  

Pick a port number 'xxxx' somewhere above 8800. Use the same number throughout. Now edit your local .ssh/config file to contain:

# your_laptop/.ssh/config
#  ... 
# AEI
host minerva01
hostname      minerva01.aei.mpg.de
ForwardAgent  yes
user          your_minerva_username
#
# ssh-tunnel to access jupyter on minerva:
host minerva01_jupyter
hostname      minerva01.aei.mpg.de
user          your_minerva_username
LocalForward  xxxx localhost:xxxx

First, inside a minerva terminal, launch the jupyter server, and keep it running:

 ssh minerva01
 source python37_venv/bin/activate
 jupyter lab --no-browser --port xxxx   # <-- prints connection info

Ideally, you launch `jupyter` inside a `screen` session, which you keep running indefinitely, independent of disconnections. Starting the jupyter server will print some text with info how to connect. You will need this text soon.

Second, in activate the ssh-tunnel with port-forwarding by ssh-ing into the 'tunneled' connection:

ssh minerva01_jupyter  

Leave this connection open, while you're working in Jupyter,

Finally, in your local web-browser on your laptop, go to the web-page mentioned in the first bullet, i.e. something like `http://localhost:xxxx/lab?token=279f72788d79160801ab9cd2d9485e3d073bc05a78db9e37`

where the 'xxxx' is the port-number you have chosen.

Install a python virtual environment (once)

. /home/SPACK/share/spack/setup-env.sh
module avail # shows now a lot of spack modules
spack load python@3.6
python3 -m venv python3_interactive
source python3_venv/bin/activate
module purge # sergei says 'or things get weird'
which pip   # gibt das pip aus der virtual env
pip install --upgrade pip
pip install jupyterlab   # installs a lot
pip install scipy  # also installs numpy
pip install h5py
pip install seaborn  # also installs matplotlib
pip install astropy
pip install pandas
pip install lalsuite # Sergei: "this should just be working" (installs last lalsuite-release)

2. When logging in, launch environment and start jupyter server

module purge; . /home/SPACK/share/spack/setup-env.sh; spack load libpng
source python3_interactive/bin/activate
jupyter lab --no-browser --port xxxx

where 'xxxx' is a four-digit number around 8800. (pick your own one)

3. From your laptop/desktop use a ssh-tunnel to forward the port xxxx

# your_laptop/.ssh/config

#  ... 

# AEI
host minerva01
hostname      minerva01.aei.mpg.de
ForwardAgent  yes
user          your_minerva_username
# tunnel to allow access to my Jupyter server running on Minerva
# start with 'ssh -N minerva_jupyter' on the machine on which
# you run safari
host minerva01_jupyter
hostname      minerva01.aei.mpg.de
user          your_minerva_username
LocalForward  xxxx localhost:xxxx

4. on your laptop, run a web-browser, and connect to the port-forwarded jupyter server. This is a command like the following, which was output when you started 'jupyter lab' on minerva01:

 http://localhost:8877/?token=46fcf8476e170a16e3c6d632d7c75d04e7115022bc3e81c4

5. If desired: one can run the 'jupyer lab' inside a 'screen' terminal to make it persistent. Google 'screen unix' or wait for instructions which may follow here at some other day.

It is possible to run a Jupyter server on Minerva, and to display the notebooks locally on your laptop/desktop.

First, make sure you have access to jupyter notebook on Minerva. Harald followed Installing LALSuite on Minerva (the SPACK way) until he had a (personal) environment inside which he installed (his personal copy of) jupyter notebook via pip install. Alternatively, Steffen says that spack also has a jupyter package, which may avoid the need to setup one's own environment and pip install's.

Second, log into minerva, and launch a jupyter server:

iMac-Home ~ $ ssh minerva01     # assuming 'minerva01' configured in ~/.ssh/config; otherwise use minerva01.aei.mpg.de

... minerva login message here ...

login01 ~ $ module purge                             # not sure this is needed
login01 ~ $ . /home/SPACK/share/spack/setup-env.sh    
login01 ~ $ spack load libpng                        # otherwise matplotlib inside jupyter will fail
login01 ~ $ source python_interactive/bin/activate   # Harald's personal environment.  Use your own or do spack all the way
(python_interactive) login01 ~ $ jupyter notebook --no-browser
[I 09:29:37.797 NotebookApp] The port 8888 is already in use, trying another port.
[I 09:29:37.820 NotebookApp] Serving notebooks from local directory: /home/hpfeiffer
[I 09:29:37.821 NotebookApp] The Jupyter Notebook is running at:
[I 09:29:37.821 NotebookApp] http://localhost:8889/?token=46fcf8476e170a16e3c6d632d7c75d04e7115022bc3e81c4
[I 09:29:37.821 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 09:29:37.821 NotebookApp] 
    
    Copy/paste this URL into your browser when you connect for the first time,
    to login with a token:
        http://localhost:8889/?token=46fcf8476e170a16e3c6d632d7c75d04e7115022bc3e81c4

Note the port-number and the token printed by jupyter notebook. The jupyter notebook command does not exit, and listens on the printed port on minerva for web-browsers asking to connect. We cannot directly connect to this port from outside, though, because firewalls prevent such connections. The solution is to use ssh to “tunnel” the connection:

Third, in a separate window on your laptop/desktop, setup a ssh-tunnel

iMac-Home ~ $ ssh minerva01 -L 8888:localhost:8889 -N

This connects port '8888' on your laptop/desktop to port 8889 on minerva. The first number (8888) is essentially arbitrary, we've just used the standard number jupyter tries first. The second number (8889) must match the output of the jupyter notebook. This ssh command will not terminate. As long as it is active, it maintains the tunnel. Note that if you happen to have a jupyter notebook server running on your machine, then probably that will have taken over port 8888, so you will need to use a different port number instead of 8888 (e.g. 8889:localhost:8889).

Fourth, in a web-browser on your local machine, connect to

localhost:8888   # or whatever is the first number you specified in 'ssh -L'

This will now connect to the jupyter notebook server on minerva, and will first ask for the 'token', which you can cut-and-paste from the output of jupyter notebook above. Now you should be ready to jupyter away at your heart's content.

1. On minerva, run jupyter notebook inside screen, to make it persistent
2. use jupyter notebook –port nnnn to pick your own non-changing port number
3. on your machine, code the tunnel into ~/.ssh/config with hardcoded local port number
4. use autossh to make the ssh-tunnel more failure resistant
5. make a web-browser bookmark with your hardcoded port-number and the token of the (persistent) jupyter server inside screen
6. … and of course, do not share your token with the world (the one above isn't valid anymore)