Virtual Machines and Linux

Your computer is one of your most important educational companions. So you want to protect the health of your computer and avoid problems caused by redundant or even malicious software. For this reason, it is recommended to install all software used on this website on a so-called Virtual Machine (VM). This page explains what VMs are, what advantages they have exactly, and how you can install Debian Linux, one of the most stable and secure operating systems. It does not matter if you use Apple’s mac OS or Microsoft’s Windows, or if you already use Linux: The VM will look the same at the end.

About¶

What is a Virtual Machine (VM)?¶

A Virtual Machine (VM) represents a virtual operating system (OS) running on a host system that directly runs on a physical computer. The physical computer (hardware) resources of the host system are allocated via so-called hypervisors. For this reason, the virtual machine is referred to as a guest machine and the hypervisor as a host machine. Thus, a guest machine is an isolated, virtualized environment that runs independently on the host operating system.

Why use a VM?¶

Free resources on a computer represent, from an economic point of view, dead storage space that causes costs without generating income. This is why system administrators often create multiple virtual servers to better allocate physical resources and save energy. A VM is also useful for smart end-users such as researchers and engineers. Here are some advantages of a VM:

• Applications and services of multiple VMs do not interfere with each other.

• Independence of the guest system from the operating system of the host system and the physical hardware.

• VMs can be moved or cloned by simply copying them to other systems.

• Hardware resources can be dynamically allocated via the host hypervisor.

• Better and more efficient use of existing hardware resources.

• Short deployment times for systems and applications running on a VM.

• VMs provide high availability and flexibility because of their independence of physical resources.

In the context of hydro-informatics for water resources management, a VM can serve to execute various Python scripts with different dependencies or, most importantly, to set up a clean and efficient environment for the execution of numerical models such as open TELEMAC-MASCARET.

Contents and Debian Linux¶

This page guides through the installation of a Debian Linux virtual machine. The host hypervisor is assumed to be Oracle’s VirtualBox on Windows. If you are not using Windows, just download the VirtualBox installer that suits your system. While VirtualBox works x-platform, there are other, more powerful platform specific utilities for creating VMs:

sudo apt install gnome-boxes
sudo apt install virt-manager qemu-kvm libvirt-daemon-system

The guest machine will run Debian Linux, which is one of the most stable Linux distributions, and it is freely available. Because of its stability, Debian is an ideal baseline for running numerical simulations that may last for days or even weeks. Of course, there are other options, and Debian is rather one of the best options than the best option.

There are a couple of Debian Linux spin-offs, such as Ubuntu with some other derivatives that are more lightweight (and faster). New Linux users will have an easy start with the Ubuntu variants Linux Mint (Xfce) or Lubuntu. Most versions Ubuntu can be used instead of the Debian Linux presented here to complete the tutorials on this website.

Get prerequisites (required software)¶

Estimated duration: 5-30 minutes (depends on connection speed).

• Download and install the latest version of VirtualBox.

  • Oracles’s VirtualBox is a free and open-source hosted hypervisor software.

  • The installation of VirtualBox requires administrator rights on the host machine. So if you are working in a professional environment, talk to your IT administrator.

  • VirtualBox installers are available for Windows, mac OS, and Solaris.

• Download the latest Debian Linux (or one of its spin-offs - see next bullet point) net installer (this is the recommended solution for working with TELEMAC):

  • Find the CD-section and click on the amd64 version

  • Scroll to the bottom of the page and download the latest net installer (debian-xx.x.x-amd64-netinst.iso)

  • If the above link is not working, visit debian.org, find the Getting Debian section (do not click on the one in the top menu), click on CD/USB ISO images and click on Download CD/DVD images using HTTP.

• ISO images for Debian spin-offs can also be used to complete the tutorials on this website (all-round systems):

  • Lubuntu is a fast and light-weight derivative of Ubuntu

  • Linux Mint (Xfce) is another efficient Ubuntu spin-off that is always based on the penultimate Ubuntu version (e.g., if the current Ubuntu version is 20.0, then the current Mint version is 19.0 and builds on Ubuntu 19.0)

Remember where the downloaded Linux .ISO file is stored.

Special feature: The Mint Hyfo VM¶

The Linux Mint Hydro-informatics (Hyfo) VM facilitates working with the contents of this eBook and it is available upon email request. Read more in the next paragraphs OR create your own VM and jump over this section.

About the Mint Hyfo VM¶

To shorten the long path of installing many hydroinformatics applications, many tutorials in this eBook can be leveraged with a tailored virtual Linux Mint 20.3 (last updated on February, 2022) machine that is available upon email request. The Linux Mint VM comes with the following pre-installed software:

• JupyterLab (start in Terminal with jupyter-lab)

• QGIS

• TELEMAC v8p3r0 (stand-alone installation with parallel libraries enabled)

Note that Python, LibreOffice, and Firefox are pre-installed in Linux Mint. To get the download link for the Linux Mint VM appliance (MintHyfo.ova), email Sebastian. Note that the hydro-informatics.com License and Disclaimer apply in addition and on top of any software that comes with the virtual disk image.

Get Started with the Mint Hyfo VM¶

All you need is an installation of VirtualBox (see above) and the downloaded Linux Mint VM appliance (MintHyfo.ova). After downloading the appliance, open VirtualBox and:

• Go to File > Import Appliance...

• Navigate to the directory where you downloaded MintHyfo.ova

• Click Next.

• Check the Appliance settings and click Import.

Now, the virtual machine called Mint Hyfo should be visible in the list of virtual machines in VirtualBox.

Start and Manage the Mint Hyfo VM¶

Once imported, start the Mint Hyfo VM and familiarize yourself with the Mint Desktop. There are a few things you should know about the Mint Hyfo VM:

• The keyboard layout is English (US), which makes, for example, that when you press the Z key on a German keyboard, the VM understands Y. There are a couple more important differences and the VM provides an overview of the English (US) keyboard layout with the English-US-Keyboard.png picture on the Desktop.

• To install or remove software either use:

  • Start > System > Software Manager, or

  • the Terminal and sudo apt install SOFTWARE-NAME

  • your user name on the VM is: hymo

  • your user password is: hxroot

• The first step after the first start of the Mint Hyfo VM is to run an update (should be regularly repreated...):

  • Open Terminal

  • Type sudo apt update (recall the sudo password is hxroot)

  • Type sudo apt full-upgrade

  • Type sudo apt autoremove (removes outdated packages)

  • Alternatively, check if the shield-icon on the bottom right is marked with an orange dot. If this is the case, double-click on the shields icon and click on Install Updates (recall the password is hxroot).

• To transfer data from the host system (e.g., your Windows) to the Mint Hyfo VM, enable folder sharing as described in the below section on folder sharing. Note that you will not need to add your user name to the VM’s vboxsf group (i.e., sudo usermod -aG vboxsf hymo).

• The TELEMAC installation lives in the home directory (cd ~/telemac/v8p3); it is dynamically compiled with parallelization enabled through openmpi, but without the AED2 and MED file libraries. To run TELEMAC use:

cd ~/telemac/v8p3/configs
source pysource.hyfo.sh
config.py

Capacities of the Mint Hyfo VM¶

The VM provides essentials for most tutorials in this eBook:

• the Python tutorials (Python basics and Geospatial Python tutorials) where the additional installation of flusstools is recommended in a virtual environment (read the installation instructions),

• the QGIS Tutorial, and

• the TELEMAC tutorials.

Also BASEMENT can be installed for completing the BASEMENT tutorial.

Create a VM with VirtualBox¶

Estimated duration: 5-8 minutes.

In your host system (e.g., Windows 10), click on Start, type Oracle VM VirtualBox, and hit enter. In the opened VirtualBox manager window:

• Click on the blue New button to open the VM creation wizard and enter:

  • Name: Debian Linux
    Note: The wizard should automatically recognize the Type andVersion fields.

  • Machine Folder: C:\Users\USERNAME\VirtualBox VMs

  • Type: Linux

  • Version: Debian (64-bit)
    > Click on the Next button

  • Allocate memory size: the more memory is allocated to the VM, the faster will be the VM (and for example TELEMAC-MASCARET), but the slower will be the host system (Windows 10). Rule of thumb: stay in the green range of the bar (e.g., allocate 8192 MB)
    > Click on the Next button

  • Select Create a virtual hard disk now and click on the Create button.

  • Select VDI (native to VirtualBox) and click on Next.

  • Preferably choose Dynamically allocated to start with a small virtual disk size, which can take a maximum size to be defined in the next step. Click on the Next button.

  • Leave the default disk name as is and allocate a maximum size for the virtual disk (recommended: min. 32 GB). Click on the Create button.

• Great - the basics are all set now and we are back in the VirtualBox main window, where a Debian Linux VM should be visible now on the left side of the window.

• With the Debian Linux VM highlighted (i.e., just click on it), click on the yellow Settings wheel button, which opens the Settings window:

  • In the System/Motherboard tab, verify the memory allocation and check the Enable EFI (special OSes only) box (enable).

  • In the System/Processor tab, select the number of processors that the VM uses. To not slowing down the host system (Windows 10), stay in the green range of the CPU bar. For parallel processing with TELEMAC-MASCARET, allocate at least 4 CPUs.

  • In the Display tab, check the Enable 3D Acceleration box.

  • In the Storage tab, find the Controller: IDE, where an Empty disk symbol should be located below.

    • Click on the Empty disk symbol and find the Attributes frame on the right side of the window, where a small blue disk symbol should be visible.

    • Click on the small blue disk symbol to Choose a disk file ... > select the Debian Linux net installer (debian-xx.x.x-amd64-netinst.iso) that you downloaded before.

  • Click OK.

Install Debian Linux¶

Estimated duration: 30 minutes.

To install Debian Linux to the VM, start the before-created Debian Linux VM in the VirtualBox manager window (click on the Debian Linux VM and then on the green Start arrow). The VirtualBox VM window will ask for the .iso file to use (confirm the selected one), and start navigating through the installation:

• Inside the VirtualBox VM window, select the Graphical install option.

• Navigate through the language options (recommended: English - English (United States)).

• Optionally define a hostname (e.g., debian-vm) and a domain name (e.g., debian-net).

• Create a root user name and password (write down the credentials somewhere) as well as a user name (no root rights) and password.

• Set up the clock.

• Disk partitioning: Choose the Guided - use entire disk option. Click Continue (2 times).

• Select the All files in one partition (recommended for new users) option. Click Continue.

• Make sure that Finish partitioning and write changes to disk is selected and click Continue.

• Select Yes in the next step (Write the changes to disks?).
  ... grab your favorite beverage and wait while the installation progresses ...

• Select No to answer the question Scan another CD or DVD? and click Continue.

• Select the geographically closest mirror to access Debian archives (software repositories and updates) and click Continue (2 times).

• Skip the proxy information question (just click Continue).

• Optionally, select No to answer the question Participate in the package usage survey? and click Continue.

• Software to install: Select GNOME and keep the other defaults (Debian desktop, print server, and standard system utilities).
  ... continue enjoying your favorite beverage and wait while the installation progresses ...

• Click Continue to finalize the installation and reboot (or shutdown) the VM.

Once the VM is shut down, re-open the VM Settings (from VirtualBox Manager window) and go to the Storage tab. Verify that there is again an Empty disk symbol in the Controller: IDE field.

Re-use (Transfer or Copy) a VM on another Host¶

Once you have created a VM on a virtual hard disk (the *.vdi file), you can always transfer it to another host system with its ecosystem. To copy (or transport) the VM:

• On the original host:

  • Shut down the VM you want to reuse.

  • In VirtualBox go to File > Export Appliance....

  • Follow the instructions to export the VM to a *.ova file (the export process may take 10-15 minutes).

• On the new host:

  • Make sure that VirtualBox is installed.

  • Copy the exported *.ova file to the new host system.

  • Open VirtualBox and go to File > Import Appliance...

  • Navigate to the directory where you copied the *.ova file.

  • Click Next.

  • Check the Appliance settings and click Import.

Now, the virtual machine should be visible in the list of virtual machines in VirtualBox.

Debian Linux¶

Setup Linux¶

Estimated duration: 15 minutes.

Start the Debian Linux VM from the VirtualBox manager window. Once Debian Linux has started, log on with your user credentials.

To enable the full functionality of the system, open the Linux Terminal (CTRL + Alt + T or go to Activities > Files (filing cabinet symbol), right-click in any folder and select Open in Terminal). In Terminal type:

su

Enter the above-created password for the root user name (see installation section).

Install all packages required for building kernel modules:

apt update
apt install build-essential dkms linux-headers-$(uname -r)

Find the Devices drop-down menu of the VirtualBox VM window (not in Debian Linux itself) and select Insert Guest Additions CD image... (depending on the version of VirtualBox, this menu can be on the top or the bottom of the window).

Back in the Debian Linux Terminal, mount the Guest Additions iso file by typing in Terminal:

sudo mkdir -p /mnt/cdrom
sudo mount /dev/cdrom /mnt/cdrom

Navigate to the mounted directory and execute the VBoxLinuxAdditions.run file with the --nox11 flag to avoid spawning an xterm window.

cd /mnt/cdrom
sudo sh ./VBoxLinuxAdditions.run --nox11

The kernel modules will be installed now and Terminal should prompt a message that invites you to reboot the system. Do so by typing:

shutdown -r now

After rebooting, make sure that the installation was successful. In Terminal type:

lsmod | grep vboxguest

The installation was successful if the Terminal’s answer is something like vboxguest 358395 2 vboxsf. Read more about Guest Additions on the VirtualBox developer’s website.

To improve the visual experience do the following:

• In the top-left corner of the Debian Linux Desktop, click on Activities and type displays in the search box.

• Open the Displays settings to select a convenient display resolution.

  • If you choose a too high resolution, the VirtualBox VM window will turn black and jump back to the original resolution after 15-30 seconds.

  • Consider also turning on Night Light to preserve your eye vision.

• Apply the changes and close the Displays settings.

Linux Terminal¶

Estimated duration: 60 minutes.

To get familiar with Debian Linux, go to the Activities menu and find the applications LibreOffice-Writer, Firefox, the Software application (shopping bag symbol), and the File manager (filing container symbol). Find more applications by clicking on the four dots on the left of the menu bar - can you find the Text Editor? To shutdown Debian Linux (i.e., the VM), click on the top-right corner arrow and press the Power symbol.

The GNOME Terminal is one of the most important features, even though it optically shows only an empty window with a blinking cursor at the beginning. There are many ways to open Terminal and here are two options:

1. Go to Applications and type Terminal in the search box, or

2. Open the File browser (Applications > Files - the filing container symbol), navigate to the folder where you want to run Terminal, right-click in the free space, and left-click on Open in Terminal.

Terminal runs many powerful native Linux (UNIX) commands, which is the most robust way to install and execute features. There are a couple of tutorials for learning to use Terminal and one of the most comprehensive is provided on the Linux Ubuntu website (Ubuntu is based on Debian Linux). It is highly recommended to go through the tutorial provided by the Ubuntu community (estimated duration: 51 minutes), for better understanding some contents of this eBook. In particular, memorize the commands cd (change directory), su/sudo (superuser), ls (listen) and mkdir (make directory).

User Rights¶

When installing software later, it is good practice to install it for your user account and not for root. Such system-relevant actions require superuser (su) rights. However, your default user name is not on the so-called sudoers list, which is essentially a file where all user accounts are listed that are authorized to use sudo in front of any command. So add your user account to the su-doers list by opening Terminal and typing:

su
    password: ...
sudo usermod -aG sudo YOUR-USER-NAME

Open a new Terminal tab (Shift + CTRL + T), which should open up in your default user space with default user rights. Check if your account is on the sudoers list by typing:

sudo -v

If visually nothing happens, you are good to go. Otherwise, if you get a message like Sorry, user [username] may not run sudo on [hostname]., verify that you correctly typed the above command and YOUR-USER-NAME (with correct cases).

VM Settings¶

Enable Folder Sharing between Host and Guest¶

Estimated duration: 5-10 minutes.

Sharing data between the host system (e.g., Windows 10) and the guest system (Debian Linux VM) enables the transfer of files to and from the VM to the host system.

• At a place of your convenience, create a new folder on the host system (e.g., Windows 10) and call it shared (e.g., C:\Users\USER\documents\shared\).

• Start VirtualBox and the Debian Linux VM.
  Make sure that the scaled view mode is off (toggle view modes with RIGHT CTRL (Host) and C on the keyboard).

• Go to the VM VirtualBox window’s Devices menu, click on Shared Folders > Shared Folders Settings... and click on the little blue Add new shared folder symbol on the right side of the window (see figure below). Make the following settings in the pop-up window:

  • Folder Path: Select the just created ...\shared folder

  • Check the Enable Auto-mount box

  • Check the Make Permanent box

• Click OK on both pop-up windows.

The shared folder will then be visible in the Files (Activities > Filing cabinet symbol) on the left (e.g., as sf_shared).

A Permission denied message is likely to appear when you click on sf_shared. The message may appear because your user name is not assigned to the vboxsf group. The vboxsf is the one, which is automatically assigned for accessing the shared folder. To verify the group name, go to the shared folder, right-click in the free space, and select Permissions. A window with group names that have access to the shared folder opens. To add your username type (in Terminal):

sudo usermod -aG vboxsf YOUR-USER-NAME

Afterward, reboot the Debian Linux VM and test if you can access the folder, and create and modify files. More information on Permission denied errors is provided in the troubleshooting section on Permission Denied Messages in Linux.

Enable OpenGL¶

VirtualBox experimentally enables OpenGL, which is used by many graphical user interfaces. To make OpenGL work on a virtual machine, the install X.Org X Window System (xserver):

sudo apt install xorg

Run Xorg as a normal user with:

startx

Or run Xorg as root (superuser) with:

sudo service gdm start

To edit the configuration of Xorg run:

sudo editor /etc/X11/xorg.conf

Add Nvidia repositories and drivers (maybe not necessary on newer versions of Debian:

sudo apt install software properties-common
sudo add-apt-repository contrib
sudo add-apt-repository non-free
sudo apt update

Then install OpenGL with:

sudo apt install libopengl0-glvnd-nvidia libglx0-glvnd-nvidia

Software Management¶

Estimated duration: Variable.

Install Software¶

To install other software, preferably use the built-in software manager (Activities > Shopping bag symbol). The Software manager uses official releases in the stable Debian repository (read more about lists of sources).

The last command removes files that are not needed anymore and reduces system garbage.

Instructions for installing particular and Debian-compatible software (e.g., QGIS) can be found directly on the website of software developers. For example, to install Anaconda Python visit docs.anaconda.com and follow the installation instructions for Debian Linux.

The most popular software manager of Debian-based Linux distributions is called aptitude, which is the thing that you invoke when typing [sudo] apt [something]. There are more standard package managers, such as dpkg or synaptic (see the full list in the Debian package management faqs). To install a wide range of proprietary software (e.g., video conferencing tools), package management tools like snapd or flatpak can be useful. However, such third-party package managers are debated, and a more robust approach is to use apt whenever possible, or, if the software is not available through apt, look for AppImages.

Useful Software¶

System Monitoring¶

There are multiple options to monitor system resource of a Debian distro, and this section presents two of them, namely, htop and stacer. htop is a minimalistic tool that runs in Terminal. stacer is a graphical user interface providing more options, including optimization of battery life or system performance.

To install htop for visualizing CPU usage, type:

sudo apt update
sudo apt install htop

Start htop’s CPU monitor with:

htop

To install the more general stacer tool, type:

sudo apt update
sudo apt install stacer

After the installation, stacer can be opened from the system menu (search for stacer).

Add Package Repositories¶

For adding (trusted) software (package) repositories use software-properties-common, which provides the add-apt-repository command:

sudo apt install software-properties-common
sudo add-apt-repository contrib
sudo add-apt-repository non-free
sudo apt update

Find Packages¶

Some software will run into errors because of missing library files (e.g., libGLX.so.0: No such file or directory). To find out what package needs to be installed for getting the missing library file, install apt-file

sudo apt install apt-file
sudo apt-file update

To find out the package name of a missing library file (e.g., libGLX.so.0), tap:

apt-file find libGLX.so.0

After a couple of seconds of searching, apt-file will prompt something like:

libglx0-glvnd-nvidia: /usr/lib/x86_64-linux-gnu/libGLX.so.0

That means, to get the library file libGLX.so.0, the package libglx0-glvnd-nvidia must be installed; for instance:

sudo apt install libglx0-glvnd-nvidia

Update Software¶

To update repositories and upgrade installed packages through apt, that is the standard Debian package manager aptitude, open Terminal and type:

sudo apt update
sudo apt full-upgrade
sudo apt autoremove
sudo apt autoclean

This will not update manually installed software, and packages installed through snapd and flatpak.

Windows Apps on Linux¶

Estimated duration: 10-15 minutes.

The Wine application provides a Windows-like environment on Linux systems, which enables installing and running Windows applications. Wine can be either used through the convenient sandboxes like Bottles or directly installed on your Linux system.

flatpak install flathub com.usebottles.bottles
flatpak install flathub com.obsproject.Studio.Plugin.OBSVkCapture
flatpak install flathub org.freedesktop.Platform.VulkanLayer.MangoHud
flatpak install flathub org.freedesktop.Platform.VulkanLayer.vkBasalt
flatpak install flathub org.freedesktop.Platform.VulkanLayer.gamescope

flatpak run com.usebottles.bottles

sudo apt update
sudo apt install lutris

flatpak install flathub net.lutris.Lutris
# run it
flatpak run net.lutris.Lutris

# add Lutris repository
echo "deb [signed-by=/usr/share/keyrings/lutris.gpg] https://download.opensuse.org/repositories/home:/strycore/Debian_12/ ./" | sudo tee /etc/apt/sources.list.d/lutris.list
# add GPG key
wget -qO- https://download.opensuse.org/repositories/home:/strycore/Debian_12/Release.key | gpg --dearmor | sudo tee /usr/share/keyrings/lutris.gpg > /dev/null
# install it
sudo apt update
sudo apt install lutris

sudo dpkg --add-architecture i386
sudo apt update
sudo apt install steam-installer vulkan-tools \
  mesa-vulkan-drivers mesa-vulkan-drivers:i386 \
  libgl1-mesa-dri libgl1-mesa-dri:i386 gamemode mangohud

sudo dpkg --add-architecture i386
sudo apt update
sudo apt install steam-installer vulkan-tools libvulkan1 libvulkan1:i386 \
  libnvidia-gl-5xx libnvidia-gl-5xx:i386 gamemode mangohud

flatpak install flathub com.valvesoftware.Steam
flatpak install flathub net.davidotek.pupgui2
flatpak run net.davidotek.pupgui2

mangohud gamemoderun %command%
PROTON_LOG=1 %command%
PROTON_NO_ESYNC=1 PROTON_NO_FSYNC=1 %command%
PROTON_USE_WINED3D=1 %command%

sudo apt install winbind playonlinux winetricks -y 

sudo dpkg --add-architecture i386 && sudo apt update

sudo apt install wine wine32 wine64 libwine libwine:i386 fonts-wine