.. _full_installation:

Full Installation
=================

.. _Debian Ubuntu Mint:
.. _RHEL CentOS Fedora:
.. _Arch Manjaro:
.. _OpenSUSE:
.. _Linux Installation:
.. _FreeBSD Installation:
.. _macOS Installation:
.. _Windows Installation:


.. _virtual_environment:

Virtual environment
-------------------

It is preferable to install *mendevi* in a virtual environment. Please refer to the `pyenv main page <https://github.com/pyenv/pyenv>`_. It is possible to use ``python3-venv`` or ``conda`` as well.

.. _install_pyenv:

Install pyenv
^^^^^^^^^^^^^

First install the `python dependencies <https://github.com/pyenv/pyenv/wiki#suggested-build-environment>`_ then install pyenv.

.. tab:: Linux

    .. tab:: Debian Ubuntu Mint

        .. code:: shell

            sudo apt install libedit-dev libncurses5-dev build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev curl git libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev llvm-bolt libbolt-dev

    .. tab:: RHEL CentOS Fedora

        .. code:: shell

            sudo dnf install openssl11-devel --allowerasing
            sudo dnf install gcc make patch zlib-devel bzip2 bzip2-devel readline-devel sqlite sqlite-devel openssl-devel tk-devel libffi-devel xz-devel llvm-bolt

    .. tab:: Arch Manjaro

        .. code:: shell

            yay -S ncurses5-compat-libs

    .. code:: shell

        curl https://pyenv.run | bash
        CONTENT=$'export PYENV_ROOT="$HOME/.pyenv"\ncommand -v pyenv > /dev/null || export PATH="$PYENV_ROOT/bin:$PATH"\neval "$(pyenv init -)"\neval "$(pyenv virtualenv-init -)"'
        grep -q "$CONTENT" ~/.bashrc || echo "$CONTENT" >> ~/.bashrc
        source ~/.bashrc

.. tab:: macOS

    You could install dependencies by using `Homebrew <https://brew.sh>`_.

    .. code:: shell

        brew install ncurses
        brew install openssl readline sqlite3 xz zlib tcl-tk llvm
        brew link llvm --force
        brew install pyenv

.. tab:: Windows

    .. warning:: Windows is crap, so be prepared for a tedious and buggy installation! You should forget Microchiotte-Windaube and go straight to Linux before you pull out all your hair!

    Good luck with the `official guide <https://github.com/pyenv-win/pyenv-win>`_.

If it fails, `this debug link <https://github.com/pyenv/pyenv/wiki/Common-build-problems>`_ may help you.

Configure pyenv
^^^^^^^^^^^^^^^

Create the virtual environment.

.. code:: shell

    pyenv update
    # --enable-experimental-jit=yes
    # --enable-bolt
    export PYTHON_CONFIGURE_OPTS='--enable-shared --enable-optimizations --with-lto=yes'
    export PYTHON_CFLAGS='-march=native -mtune=native'
    pyenv install -v 3.14
    pyenv virtualenv 3.14 mendevi
    pyenv activate mendevi
    pip install uv

.. _dependencies:

Dependencies
------------

.. _install_ffmpeg:

ffmpeg
^^^^^^

.. warning:: If you want the latest version, please build it with: :ref:`build_ffmpeg`. And **DO NOT INSTALL** the precompiled ffmpeg package.

*mendevi* has hard dependency on the FFmpeg package. You should install it first, please refer to the `FFmpeg download page <https://ffmpeg.org/download.html>`_.


.. tab:: Linux

    .. tab:: Debian Ubuntu Mint

        .. code:: shell

            sudo apt install ffmpeg

    .. tab:: RHEL CentOS Fedora

        .. code:: shell

            sudo dnf install ffmpeg

    .. tab:: Arch Manjaro

        .. code:: shell

            sudo pacman -S ffmpeg

    .. tab:: OpenSUSE

        .. code:: shell

            sudo zypper install ffmpeg

.. tab:: FreeBSD

    .. code:: shell

        sudo pkg install ffmpeg

.. tab:: macOS

    You could install the FFmpeg by using `Homebrew <https://brew.sh>`_.

    .. code:: shell

        brew install ffmpeg pkg-config

.. tab:: Windows

    .. warning:: I refuse to believe that you have gone through the previous steps. As you are stubborn and stupid, I must remind you that the game is not yet won! It is still not too late to listen to the voice of wisdom, which is telling you to get rid of this massive piece of Windows shit for ever.

    It is important to configure your environment variable to hook ffmpeg to *mendevi*. You can try to follow `this guide <https://www.geeksforgeeks.org/how-to-install-ffmpeg-on-windows>`_ for example.

.. _install_lshw:

lshw
^^^^

To get the full environment information, *mendevi* use tha system package ``lshw``.

.. tab:: Linux

    .. tab:: Debian Ubuntu Mint

        .. code:: shell

            sudo apt install lshw

    .. tab:: RHEL CentOS Fedora

        .. code:: shell

            sudo dnf install lshw

    .. tab:: Arch Manjaro

        .. code:: shell

            sudo pacman -S lshw

.. _install_perf:

perf
^^^^

To access the RAPL sensors.

.. tab:: Linux

    .. tab:: Debian Ubuntu Mint

        .. code:: shell

            if ! sudo apt install -y linux-perf; then
                sudo apt install -y linux-tools-common linux-tools-generic linux-tools-$(uname -r)
            fi

    .. tab:: RHEL CentOS Fedora

        .. code:: shell

            sudo dnf install perf

    .. tab:: Arch Manjaro

        .. code:: shell

            sudo pacman -S perf

.. _install_transmission:

transmission (optional)
^^^^^^^^^^^^^^^^^^^^^^^

To be able to automatically retrieve and share datasets and videos. For more informations, please have a look on the `transmission webpage <https://transmissionbt.com/>`_.

.. tab:: Linux

    .. tab:: Debian Ubuntu Mint

        .. code:: shell

            sudo apt install --reinstall transmission-daemon


    .. tab:: RHEL CentOS Fedora

        .. code:: shell

            sudo dnf install transmission-daemon

.. tab:: macOS

    You could install transmission by using `Homebrew <https://brew.sh>`_.

    .. code:: shell

        brew install transmission transmission-cli

Ensure that ``transmission-daemon`` is installed correctly with the good permissions:

.. code:: shell

    echo '======== CHECK AND FIX "transmission-daemon" INSTALLATION ========'

    # find transmission-daemon user
    trans_user=$(systemctl show -p User transmission-daemon.service | cut -d= -f2)

    # create a common group with $USER and $trans_user
    if getent group transshare > /dev/null; then
        echo 'ok - the group transshare already exists'
    else
        echo "create a transshare group with $USER and $trans_user..."
        sudo groupadd transshare
        sudo usermod -aG transshare $USER
        sudo usermod -aG transshare $trans_user
        sudo systemctl restart transmission-daemon  # apply the effect
        echo '    done'
    fi

    # set the transshare group permissioin to ~/.cache/mendevi
    mkdir -p ~/.cache/mendevi
    if ! sudo -u $trans_user ls ~/.cache/mendevi/ &>/dev/null; then
        echo 'give the permission in ~/.cache/mendevi for transshare...'
        sudo chgrp -R transshare ~/.cache/mendevi/
        sudo chmod -R g+w ~/.cache/mendevi/
        sudo find ~/.cache/mendevi/ -type d -exec chmod 2775 {} \;
        sudo find ~/.cache/mendevi/ -type f -exec chmod ug+rw {} \;
        sudo chmod o+x ~/
        sudo chmod o+x ~/.cache/
        echo '    done (perhaps reboot required)'
    else
        echo "ok - $trans_user can access to ~/.cache/mendevi"
    fi

    # make transmission-daemon working
    if [ -n "$(transmission-remote -l 2>/dev/null)" ]; then
        echo 'ok - transmission-daemon seems to be reachable'
    else
        echo 'attempted transmission repair...'
        sudo systemctl stop transmission-daemon
        sudo sed -i 's|Type=notify|Type=simple|g' /lib/systemd/system/transmission-daemon.service
        sudo sed -i 's|--log-level=error|--log-level=info|g' /lib/systemd/system/transmission-daemon.service
        sudo sed -i 's|"rpc-authentication-required": true|"rpc-authentication-required": false|g' /etc/transmission-daemon/settings.json
        # sudo sed -i "s|User=$trans_user|User=$USER|g" /lib/systemd/system/transmission-daemon.service
        sudo systemctl daemon-reload
        sudo systemctl start transmission-daemon
        echo "type 'systemctl status transmission-daemon' to check the status"
    fi

    # enable transmission-daemon
    if [ "$(systemctl is-enabled transmission-daemon 2>/dev/null)" = "disabled" ]; then
        echo 'enable transmission-daemon service...'
        sudo rm -f /etc/systemd/system/multi-user.target.wants/transmission-daemon.service
        sudo systemctl enable transmission-daemon
        echo '    done'
    else
        echo 'ok - transmission-daemon is enabled'
    fi

    # open graphical interface in a web browser
    xdg-open http://localhost:9091


.. _install_pyav:

pyav (optional)
^^^^^^^^^^^^^^^

Although it is installed automatically, it is better to install **av** manually to avoid redundancy with ffmpeg. Please refer to the `PyAv installation guide <https://pyav.org/docs/develop/overview/installation.html>`_.

.. tab:: Linux

    .. code:: shell

        uv pip -v install av --no-binary av

.. tab:: FreeBSD

    .. code:: shell

        uv pip -v install av --no-binary av

.. tab:: macOS

    On **MacOS** you may have issues with regards to Python expecting gcc but finding clang. Try to export the following before installation:

    .. code:: shell

        export ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future
        uv pip -v install av --no-binary av

.. tab:: Windows

    .. warning:: I see that you insist on using Windows, this step is the most critical, good luck!

    On **Windows** you must build from source in order to indicate the location of ffmpeg, pleas follow the `PyAv windows installation guide <https://pyav.org/docs/develop/overview/installation.html#windows>`_ and good luck!

.. _install_cuda:

cuda (optional)
^^^^^^^^^^^^^^^

If you have a NVIDIA GPU, please install CUDA then follow the `PyTorch installation guide <https://pytorch.org>`_. Without CUDA, the software is not able to use the GPU (CPU only).

.. tab:: Linux

    Follow the official `NVIDIA CUDA Installation Guide for Linux <https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html>`_.
    Or the `non official guide <https://linuxcapable.com/how-to-install-cuda-on-debian-linux/>`_.

    .. tab:: Debian Ubuntu Mint

        Install nvidia driver.

        .. warning::

            **Do not install "cuda-toolkit"**, witch is conflicual with the official drivers.

        Prerequisite: This block can be executed multiple times

        .. code:: shell

            sudo apt install linux-headers-$(uname -r)
            sudo dkms autoinstall
            sudo modprobe nvidia

        .. code:: shell

            if dpkg -l | grep -q nvidia-driver; then
                INSTALLED_NVIDIA_DRIVER_VERSION=$(dpkg-query -W -f='${Version}' nvidia-driver)
                echo "the nvidia driver $INSTALLED_NVIDIA_DRIVER_VERSION is installed and working"
            else
                echo "no nvidia drivers are installed"

                # add nvidia driver depos
                if ! apt-cache show cuda-drivers >/dev/null 2>&1; then
                    echo "try to add the official nvidia depos"
                    wget https://developer.download.nvidia.com/compute/cuda/repos/debian13/x86_64/cuda-keyring_1.1-1_all.deb
                    sudo dpkg -i cuda-keyring_1.1-1_all.deb
                    sudo rm cuda-keyring_1.1-1_all.deb
                    sudo apt update
                else
                    echo "the official nvidia depos is available"
                fi

                # detect available versions
                AVAILABLE_NVIDIA_DRIVER_VERSION=$(sudo apt show cuda-drivers 2>/dev/null | awk '/Version:/ {print $2}' | head -n1 | cut -d'-' -f1)
                echo "NVIDIA detected dirver version: $AVAILABLE_NVIDIA_DRIVER_VERSION"
                DRIVER_BRANCH=${AVAILABLE_NVIDIA_DRIVER_VERSION%%.*}

                # install cuda
                # sudo apt remove --purge 'nvidia-*' cuda* && sudo apt update && sudo apt autoremove
                sudo apt install linux-headers-$(uname -r)  # or sudo apt install linux-headers-amd64
                sudo apt install cuda-drivers-${DRIVER_BRANCH}
                sudo dkms build -m nvidia -v ${AVAILABLE_NVIDIA_DRIVER_VERSION}
                sudo dkms install -m nvidia -v ${AVAILABLE_NVIDIA_DRIVER_VERSION}

                echo "to verify the installation, re-execute the code block"
            fi

            # wget https://us.download.nvidia.com/tesla/580.126.20/NVIDIA-Linux-x86_64-580.126.20.run
            # chmod +x NVIDIA-Linux-x86_64-580.126.20.run
            # ./NVIDIA-Linux-x86_64-580.126.20.run

        Install cuda context (without the driver)

        .. code:: shell

            sudo apt install nvtop

            # install nvcc
            if command -v nvidia-smi &> /dev/null; then
                CUDA_VERSION=$(nvidia-smi --version | grep -oP 'CUDA Version\s*:\s*\K[0-9\.]+')
                echo "detected cuda version: $CUDA_VERSION"
                if ! command -v nvcc &> /dev/null; then
                    CUDA_MAJOR=${CUDA_VERSION%%.*}
                    CUDA_COMPILER_PACKAGE=$(apt-cache search "^cuda-compiler-${CUDA_MAJOR}-" | awk '{print $1}' | sort -V | tail -n1)
                    echo "try to install $CUDA_COMPILER_PACKAGE (see 'apt search cuda-compiler')"
                    sudo apt install $CUDA_COMPILER_PACKAGE

                    CONTENT=$'export PATH=/usr/local/cuda/bin:$PATH\nexport LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH'
                    grep -q "$CONTENT" ~/.bashrc || echo "$CONTENT" >> ~/.bashrc
                    source ~/.bashrc
                else
                    echo "nvcc is installed"
                fi
            else
                echo "nvidia-smi is not installed, you have to (re)install the nvidia drivers"
                echo "sudo apt remove --purge 'nvidia-*' cuda* && sudo apt update && sudo apt autoremove"
                echo "then rerun the previous code block"
            fi

.. tab:: FreeBSD

    It is not well supproted, you can try to follow `this bsd link <https://gist.github.com/Mostly-BSD/4d3cacc0ee2f045ed8505005fd664c6e>`_.

.. tab:: macOS

    Follow the `NVIDIA CUDA Installation Guide for Mac OS X <https://docs.nvidia.com/cuda/archive/10.2/cuda-installation-guide-mac-os-x/index.html>`_.

.. tab:: Windows

    .. warning:: Windows is crap, so be prepared for a tedious and buggy installation! You should forget Microchiotte-Windaube and go straight to Linux before you pull out all your hair!

    It is important to configure your environment variable to hook cuda to PyTorch.
    Good luck with the `official guide <https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/index.html>`_.

.. _install_amf:

amf (optional)
^^^^^^^^^^^^^^

If you have an AMD GPU, please install ROC then follow the `PyTorch installation guide <https://pytorch.org>`_. Without ROC, the software is not able to use the GPU (CPU only).

.. tab:: Linux

    .. tab:: Debian Ubuntu Mint

        Il faut ajouter https://github.com/CosmicFusion/debian-amdgpu-pro
        To use the free drivers of `debian ati <https://wiki.debian.org/fr/AtiHowTo>`_, execute the following code:

        .. code:: shell

            # lspci -nn | grep -E "VGA|Display|3D"  # to see if AMD gpu is available

            sudo apt install firmware-linux-nonfree firmware-amd-graphics libgl1-mesa-dri libglx-mesa0 mesa-vulkan-drivers xserver-xorg-video-all

            # lsmod | grep amdgpu  # if nothing apears -> probleme

        To use the non-free drivers (required for video encoding but optional for torch), execute the following code.
        It comes from `amdgpu-install package <https://amdgpu-install.readthedocs.io/en/latest/index.html>`_.
        And it doses the installation given in the `AMD documentation <https://instinct.docs.amd.com/projects/amdgpu-docs/en/latest/install/detailed-install/package-manager/package-manager-ubuntu.html>`_.

        .. code:: shell

            # install amdgpu-install
            if ! command -v amdgpu-install --help &> /dev/null; then
                BASE_URL='https://repo.radeon.com/amdgpu-install/latest/ubuntu/noble'
                latest_deb=$(curl -s "$BASE_URL/" | grep -oP 'amdgpu-install_.*?_all\.deb' | sort -V | tail -n 1)
                if [[ -z "$latest_deb" ]]; then
                    echo "failed to find a .deb file in $BASE_URL"
                else
                    echo "last package: $latest_deb"
                    wget --progress=dot:giga "$BASE_URL/$latest_deb" -O "/tmp/$latest_deb"
                    sudo apt install "/tmp/$latest_deb"
                fi
            else
                echo 'the amdgpu-install package is installed'
            fi

            # install drivers
            sudo amdgpu-install --usecase=amf --accept-eula


Installation
------------

Basic Installation
^^^^^^^^^^^^^^^^^^

To install a minimal version of *mendevi*, follow :ref:`simple_installation`.
Overwise, you can follow :ref:`building_from_source` for a full installation.

.. _building_from_source:

Building From Source (For devellopers)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

To install the lastest development version from `GitLab <https://gitlab.inria.fr/rrichard/mendevi>`_ source, clone mendevi using ``git`` and install it using ``uv pip``:

.. code:: shell

    git clone https://gitlab.inria.fr/rrichard/mendevi.git ~/mendevi_git
    cd ~/mendevi_git
    pyenv activate mendevi  # to be sure to be in a virtual env
    uv pip install -e .[doc,test]
    mendevi test  # to test if the installation is ok
.. pip uninstall typing_extensions to debug attribute '__default__' of 'typing.ParamSpec' objects is not writable

You can now also compile documentation locally :ref:`documentation`.

.. _auto_completion:

CLI auto-completion
^^^^^^^^^^^^^^^^^^^

To enable autocompletion in the command line interface, you can follow `this guide <https://click.palletsprojects.com/en/stable/shell-completion/>`_, which involves executing the following code:

.. code:: shell

    _MENDEVI_COMPLETE=bash_source mendevi > ~/.mendevi-complete.bash
    CONTENT=$'. ~/.mendevi-complete.bash'
    grep -q "$CONTENT" ~/.bashrc || echo "$CONTENT" >> ~/.bashrc
    source ~/.bashrc


.. _documentation:

Build Documentation
-------------------

You can compile the documentation yourself after :ref:`full_installation` by simply typing ``mendevi doc`` or:

.. code:: shell

    cd docs/ && make clean && make html && cd -
    firefox docs/build/html/index.html

.. _testing

Testing
-------

To run the unitary tests, just type ``mendevi test``.