.. _landing_zone_singularity: Quickstart for Galileo Landing Zones and Singularity ==================================================== The following is a user guide for deploying a Galileo Landing Zone (LZ) using our official Singularity image. .. contents:: :local: :depth: 2 Prerequisites ------------- Singularity (version >= 3.3.0) must be installed on the machine you wish to use as a Landing Zone. You can find installation instructions `here `_. Singularity's `fakeroot `_ feature is necessary to run jobs without root privileges. That feature requires Linux kernel >= 3.8, but version >= 3.18 is recommended. How It Works ------------ In a nutshell, a Landing Zone receives jobs from users, builds containers for those jobs, and then manages the execution of those containers until the job is complete. So it follows that if the LZ runs inside a container, it must be capable of making containers inside that container! And that's precisely how it works; the Landing Zone image for Singularity has a copy of Singularity installed *inside* it! Job containers are run with the ``--containall``, ``--writable``, and ``--no-home`` flags to maximise isolation and retain new and modified files. Using our official Singularity image to run a Landing Zone in a container is a fantastic way to get started because it's easy to install, update, manage, and clean up after. However there are a couple tricky aspects that the following subsections will show you how to work around. You will find these work arounds referenced throughout the rest of the documentation. Building Images and Root Privileges ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The immediate problem is that Singularity requires root privileges to build a container, and in many cases you might want run an LZ in an environment where you do not have root privileges. Thankfully Singularity added support for a potential workaround in the `fakeroot `_ feature beginning in version 3.3.0. The ``--fakeroot`` flag allows us to mimic having root privileges, so adding this flag to your Singularity invocation of the LZ allows the LZ to inherit this characteristic when building containers for the jobs sent to it, e.g. .. code-block:: bash singularity --fakeroot ... library://hypernetlabs/default/landing-zone-daemon Persisting and Accessing Application Files ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Depending on how you invoke Singularity to run the LZ, this may or may not be a problem for you. By default, Singularity bind mounts ``/home/$USER``, ``/tmp``, and ``$PWD`` into containers at runtime. If you allow for these default mounts, then the only difficulties you may experience will stem from sharing files between host and container that aren't in those mounted directories (like the global config at ``/etc/galileod.conf``). If you choose to disable those default bind mounts, then you need to be aware of the application files you'd like to access while the container is running, or persist after the container has exited. For example, if you choose to keep a copy of the authentication token to use across LZ sessions, then create a specific directory to hold it (e.g. ``$ mkdir $HOME/lz_token``), mount the directory, and use the ``--token`` flag to tell the LZ to look for it at that mount point: .. code-block:: bash singularity ... --bind $HOME/lz_token:/lz_token ... library://hypernetlabs/default/landing-zone-daemon --token /lz_token/auth.token ... A convenient way to mount all non-temporary application files would be to mount a configuration directory and use the ``--config-dir`` flag to tell the LZ where it is mounted. So if you have a directory at ``$HOME/.galileod`` you might use a command like this: .. code-block:: bash singularity ... --bind $HOME/.galileod:/config_dir ... library://hypernetlabs/default/landing-zone-daemon --config-dir /config_dir How to Run the Landing Zone Daemon ---------------------------------- Choose the name you’d like this LZ to have when viewed in the Galileo web interface. We will refer to the LZ name as :code:`$LZ_NAME`. When you see :code:`$LZ_NAME` referenced, simply substitute your chosen name in its place. * The recommended way to invoke the Landing Zone is as a Singularity `service `_ so that it can run in the background. .. code-block:: bash $ singularity instance start --fakeroot library://hypernetlabs/default/landing-zone-daemon landing-zone-daemon --name "$LZ_NAME" That command will create an instance named ``landing-zone-daemon``; you can prove this by running ``singularity instance list``. * Now that the LZ is running, we must authenticate it against your account. Run this command in your terminal .. code-block:: bash $ tail ~/.singularity/instances/logs/"$HOSTNAME"/"$USER"/landing-zone-daemon.out * This should give you some output that looks like this .. code-block:: bash Pulling updates… Already up to date! Please visit the following url in your browser and enter the code below URL: https://galileoapp.auth0.com/activate Code: XXXX-XXXX * Follow those instructions. Once you have confirmed your code at the provided URL you should see your new Landing Zone appear in the Galileo Landing Zones tab! In order to submit jobs to your new LZ, you must add it to a Station. You can create a new Station by going to the Stations tab and clicking the Create Stations button. * You can stop the LZ at any time with .. code-block:: bash $ singularity instance stop landing-zone-daemon Singularity will automatically clean up the container used to run the instance. GPUs ---- Unfortunately GPU access is not supported by Galileo for Singularity at this time. This is because Singularity cannot give containers *exclusive* access to specific GPUs making that resource impossible to allocate fairly to Galileo jobs. We are experimenting with ways to work around this limitation, so if it directly impacts you please let us know! Galileo uses Singularity containers to run jobs on Slurm clusters, but Slurm has its own GPU constraint mechanisms that allow Galileo to support GPUs on those clusters. You can read more :ref:`here`.