Linux Server Build 2024

1. Ubuntu System Function Replacements

Windows Function Ubuntu Replacement Docker Stack

DHCP

Pi-hole

stack-networking

DNS

Pi-hole

stack-networking

LDAP

Open LDAP

stack-ldap

2. Current Backups

2.1. On p-Ubuntu-NAS via rsync

Click here to open
Table 1. Backups
Source Destination

/home/mattosd/repo-docker-master

/mnt/Norco/D-Drive-Backups/P-Ubuntu-NAS

/home/mattosd/repo-docker-master

/mnt/RAIDZStoragePool/backups/P-Ubuntu-NAS

/home/mattosd/repo-docker-master

/media/mattosd/USB-5TB/backups/p-ubuntu-nas

/mnt/v-ubuntu-media-server/docker-media-server

/mnt/RAIDZStoragePool/backups/v-Ubuntu-Media-Server

2.2. On norco via robocopy

  1. To local D$ - External USB 16TB disk

    Click here to open
    Table 2. Local Source
    Source Destination

    \\norco\r$\ServerFolders\CAD

    \\norco\d$\backups\Norco\CAD

    \\norco\r$\Databases

    \\norco\d$\backups\Norco\Databases

    \\norco\r$\Gamz

    \\norco\d$\backups\Norco\Gamz

    \\norco\r$\Library

    \\norco\d$\backups\Norco\Library

    \\norco\r$\Media

    \\norco\d$\backups\Norco\Media

    z:\media

    \\norco\d$\backups\Norco\Media-Plex

    m:\Moms Stuff

    \\norco\d$\backups\Norco\Moms-Stuff

    \\norco\r$\Stuff

    \\norco\d$\backups\Norco\Stuff

    \\norco\r$\Torrents

    \\norco\d$\backups\Norco\Torrents

    \\norco\r$\Websites

    \\norco\d$\backups\Norco\Websites

  2. To local D$ - External USB 16TB disk

    Click here to open
    Table 3. Remote Source
    Source Destination

    \\v-ubuntu-media-server\docker-media-stack

    \\norco\d$\backups\v-Ubuntu-Media-Server\docker-media-stack

  3. To remote p-ubuntu-nas

    Click here to open
    Table 4. Local Source
    Source Destination

    \\norco\r$\ServerFolders\CAD

    \\p-ubuntu-NAS\RAIDZStoragePool\backups\Norco\CAD

    \\norco\r$\Databases

    \\p-ubuntu-NAS\RAIDZStoragePool\backups\Norco\Databases

    \\norco\r$\Gamz

    \\p-ubuntu-NAS\RAIDZStoragePool\backups\Norco\Gamz

    \\norco\r$\Library

    \\p-ubuntu-NAS\RAIDZStoragePool\backups\Norco\Library

    \\norco\r$\Media

    \\p-ubuntu-NAS\RAIDZStoragePool\backups\Norco\Media

    \\norco\m$\Moms Stuff

    \\p-ubuntu-NAS\RAIDZStoragePool\backups\Norco\Mom

    \\norco\r$\Stuff

    \\p-ubuntu-NAS\RAIDZStoragePool\backups\Norco\Stuff

    \\norco\r$\Torrents

    \\p-ubuntu-NAS\RAIDZStoragePool\backups\Norco\Torrents

    \\norco\r$\Websites

    \\p-ubuntu-NAS\RAIDZStoragePool\backups\Norco\Websites

3. Software installs

3.1. Base Software

  1. Follow this doc for base software.

3.2. Desktops

  1. Install desktops via this doc.

4. Shared Folder setup

4.1. Create ZFS Storage Pools

  • Launch Cockpit and use the ZFS plugin to create Storage Pools

4.2. Create Datasets

  1. Create this script as create_datasets.bash

    Details
    script
    #!/usr/bin/bash
    #
    PoolName=TwoDisks
    UserName=mattosd
    SambaGroup=sambashare
    #
    shares=("repo-docker-master" "Backups" "CAD" "Databases" "Gamz" "Library" "Media" "Media-Plex" "Moms-Stuff" "Stuff" "Torrents" "Websites")
    #
    #for shareName in "${shares[@]}"
    #do
            #echo "Destroying Dataset [${PoolName}/${shareName}]..."
            #zfs destroy ${PoolName}/${shareName}
            #echo "Deleting mountpoint [/mnt/${shareName}]..."
            #rm -R /mnt/${shareName}
    #done
    
    echo
    echo
    
    for shareName in "${shares[@]}"
    do
            echo "Creating dataset [${shareName}]..."
            zfs create ${PoolName}/${shareName}
            echo "Changing Mountpoint..."
            zfs set mountpoint=/mnt/${shareName} ${PoolName}/${shareName}
            echo "Setting permissions..."
            chown ${UserName}:${SambaGroup} /mnt/${shareName}
            echo
    done
    #
    #
    #
    #
    PoolName=SingleDisk
    #
    shares=("Media-Plex")
    echo
    echo
    for shareName in "${shares[@]}"
    do
            echo "Creating dataset [${shareName}]..."
            zfs create ${PoolName}/${shareName}
            echo "Changing Mountpoint..."
            zfs set mountpoint=/mnt/${shareName} ${PoolName}/${shareName}
            echo "Setting permissions..."
            chown -R ${UserName}:${SambaGroup} /mnt/${shareName}
            echo
    done
  2. Change permissions

    chmod 777 ./create_datasets.bash
  3. Run

    sudo ./create_datasets.bash

4.3. Create Samba Shares

  1. Create this script as create_samba_shares.bash

    Details
    #!/usr/bin/bash
    #
    UserName=mattosd
    SambaGroup=sambashare
    configFile=/etc/samba/smb.conf
    shares=("repo-docker-master" "Backups" "CAD" "Gamz" "Library" "Media" "Media-Plex" "Moms-Stuff" "Stuff" "Torrents" "Websites")
    #
    for shareName in "${shares[@]}"
    do
            echo "Configuring Samba Share [${shareName}]..."
    
            echo -e "[${shareName}]" >> ${configFile}
            echo -e "	comment = The ${shareName} Folder" >> ${configFile}
            echo -e "	path = /mnt/${shareName}" >> ${configFile}
            echo -e "	browsable = yes" >> ${configFile}
            echo -e "	read only = yes" >> ${configFile}
            echo -e "	guest ok = no" >> ${configFile}
            echo -e "	valid users = ${UserName}" >> ${configFile}
            echo -e "	write list = @${SambaGroup}" >> ${configFile}
            echo -e "	force create mode = 0666" >> ${configFile}
            echo -e "	force directory mode = 0777" >> ${configFile}
            echo -e "" >> ${configFile}
    done
  2. Change permissions

    chmod 777 ./create_samba_shares.bash
  3. Run

    sudo ./create_samba_shares.bash
  4. Now restart Samba

    sudo systemctl restart smbd

4.4. Restore Data

4.4.1. Mounts

  1. Mount //p-ubuntu-nas/RAIDZStoragePool

    1. Create a few mount points first:

      sudo mkdir /mnt/p-ubuntu-nas
      sudo mkdir -p /mnt/Norco/D-Drive
    2. Mount p-ubuntu-nas:RAIDZStoragePool

      sudo mount -t cifs //p-ubuntu-nas/RAIDZStoragePool /mnt/p-ubuntu-nas -o username=root,dir_mode=0777,file_mode=0777
    3. And mount Norco/D$

      sudo mount -t cifs -o username=mattosd@dhante.local //norco/d$ /mnt/Norco/D-Drive

      Won’t need this mount during the real rebuild event since this external 15TB USB drive will be plugged in.

4.4.2. Restores

  1. Use rsync to restore

    Example to restore the repo-docker-master folder
    sudo rsync -vrpaz /mnt/Norco/D-Drive/backups/v-Ubuntu-Media-Server/docker-media-stack/ /mnt/repo-docker-master/stack-media-management
    Example to restore the repo-docker-master folder
    sudo rsync -vrpaz /mnt/p-ubuntu-nas/backups/P-Ubuntu-NAS/repo-docker-master/ /mnt/repo-docker-master
    Example to restore the Torrents folder
    sudo rsync -vrpaz /mnt/p-ubuntu-nas/backups/Norco/Torrents/ /mnt/Torrents
    Example to restore the CAD folder
    sudo rsync -vrpaz /mnt/p-ubuntu-nas/backups/Norco/CAD/ /mnt/CAD
    Example to restore the Databases folder
    sudo rsync -vrpaz /mnt/p-ubuntu-nas/backups/Norco/Databases/ /mnt/Databases
    Example to restore the Gamz folder
    sudo rsync -vrpaz /mnt/p-ubuntu-nas/backups/Norco/Gamz/ /mnt/Gamz
    Example to restore the Library folder
    sudo rsync -vrpaz /mnt/p-ubuntu-nas/backups/Norco/Library/ /mnt/Library
    Example to restore the Media folder
    sudo rsync -vrpaz /mnt/p-ubuntu-nas/backups/Norco/Media/ /mnt/Media
    Example to restore the Moms Stuff folder
    sudo rsync -vrpaz /mnt/p-ubuntu-nas/backups/Norco/Mom Stuff/ /mnt/Moms-Stuff
    Example to restore the Stuff folder
    sudo rsync -vrpaz /mnt/p-ubuntu-nas/backups/Norco/Stuff/ /mnt/Stuff
    Example to restore the Websites folder
    sudo rsync -vrpaz /mnt/p-ubuntu-nas/backups/Norco/Websites/ /mnt/Websites
    Example to restore the Plex2 folder
    sudo rsync -vrpaz /mnt/Norco/D-Drive/backups/Norco/Media-Plex/ /mnt/Media-Plex

    The external 15TB USB drive will be plugged in for this restore during the real event

5. Modifications

5.1. Docker Modifications

5.1.1. Media Management

  1. All media will be local and not remote via shares.

    Update the /mnt/repo-docker-master/stack-media-management/compose.yml file with the following:

    Details
    version: "3.5"
    
    services:
    
    #----------------------------------------------------------------------
    # Define SurfShark VPN service
      surfshark:
        # https://github.com/qdm12/gluetun-wiki/blob/main/setup/providers/surfshark.md
        image: qmcgaw/gluetun
        container_name: My-SurfShark-VPN
        environment:
        - VPN_SERVICE_PROVIDER=surfshark
        - OPENVPN_USER=${SURFSHARK_USER}
        - OPENVPN_PASSWORD=${SURFSHARK_PASSWORD}
        - SERVER_COUNTRIES=United States
        cap_add:
          - NET_ADMIN
        devices:
          - /dev/net/tun
        ports:
          - 8080:8080     # Add the port for qBittorrent
          - 6881:6881     # another qBittorrent port
          - 6881:6881/udp # another qBittorrent port
        restart: unless-stopped
        dns:
          - 1.1.1.1
    
    #----------------------------------------------------------------------
    
      # Define qBittorrent service
      qbittorrent:
        image: lscr.io/linuxserver/qbittorrent:latest
        container_name: My-qBittorrent
        network_mode: service:surfshark   # Use the VPN
        depends_on:
          - surfshark
        environment:
          #- PUID=1000
          #- PGID=1000
          - PUID=0
          - PGID=0
          - TZ=America/New_York
          - WEBUI_PORT=8080
        volumes:
          - ./qBittorrent/config:/config
          - /mnt/Torrents:/downloads
        restart: unless-stopped
    
    #----------------------------------------------------------------------
    
      # Define Homepage dashboard
      # https://gethomepage.dev/en/installation/docker/
      homepage:
        #image: ghcr.io/benphelps/homepage:latest
        image: ghcr.io/gethomepage/homepage:latest
        container_name: My-Homepage
        environment:
          - PUID=0
          - PGID=0
          - TZ=America/New_York
        ports:
          - 3000:3000
        volumes:
          - /var/run/docker.sock:/var/run/docker.sock:ro # (optional) For docker integrations
          - ./Homepage/config:/app/config
          - ./Homepage/images:/app/public/images  #reference your image as /images/myimage.png
          - ./Homepage/icons:/app/public/icons    #reference your icon as /icons/myicon.png
        restart: always
    
    #----------------------------------------------------------------------
    
      # Define Prowlerr service for indexer access
      # https://docs.linuxserver.io/images/docker-prowlarr
      prowlarr:
        image: lscr.io/linuxserver/prowlarr:latest
        container_name: My-Prowlarr
        environment:
          - PUID=0
          - PGID=0
          - TZ=America/New_York
        ports:
          - 9696:9696
        volumes:
          - ./Prowlarr/config:/config
        restart: unless-stopped
    
    #----------------------------------------------------------------------
    
      # Define Radarr service for Movies
      radarr:
        image: linuxserver/radarr:latest
        container_name: My-Radarr
        environment:
          - PUID=0
          - PGID=0
          - TZ=America/New_York
        ports:
          - 7878:7878
        volumes:
          - ./Radarr/config:/config
          - ./Radarr/media:/media # Media can be stored here but should use a larger dedicated Shared drive instead
          - /mnt/Media-Plex:/plex2 # Media.
          - /mnt/Torrents:/downloads # Radarr needs to be able to see files downloaded via qBittorrent
        restart: unless-stopped
    
      # Define Sonarr service for TV Shows
      sonarr:
        image: lscr.io/linuxserver/sonarr:latest
        container_name: My-Sonarr
        environment:
          - PUID=0
          - PGID=0
          - TZ=America/New_York
        ports:
          - 8989:8989
        volumes:
          - ./Sonarr/config:/config
          - ./Sonarr/media:/media # Media can be stored here but should use a larger Windows Shared drive instead
          - /mnt/Media-Plex:/plex2 # Media.
          - /mnt/Torrents:/downloads # Sonarr needs to be able to see files downloaded via qBittorrent
        restart: unless-stopped
    
      # Define Bazarr subtitle service for Movies/TV Shows
      # https://www.bazarr.media/
      # https://wiki.bazarr.media/
      bazarr:
        image: lscr.io/linuxserver/bazarr:latest
        container_name: My-Bazarr
        environment:
          - PUID=0
          - PGID=0
          - TZ=America/New_York
        ports:
          - 6767:6767
        volumes:
          - ./Bazarr/config:/config
          - /mnt/Media-Plex2:/plex2 # Media.
        restart: unless-stopped
    
      # Define Lidarr service for Music
      lidarr:
        image: lscr.io/linuxserver/lidarr:latest
        container_name: My-Lidarr
        environment:
          - PUID=0
          - PGID=0
          - TZ=America/New_York
        ports:
          - 8686:8686
        volumes:
          - ./Lidarr/config:/config
          - ./Lidarr/media:/media # Media can be stored here but should use a larger Windows Shared drive instead
          - /mnt/Media/Music/Lidarr:/data/Musics # Media.
          - /mnt/Torrents:/downloads # Lidarr needs to be able to see files downloaded via qBittorrent
        restart: unless-stopped
    
      # Define Readarr service for Books
      readarr:
        image: lscr.io/linuxserver/readarr:develop
        container_name: My-Readarr
        environment:
          - PUID=0
          - PGID=0
          - TZ=America/New_York
        ports:
          - 8787:8787
        volumes:
          - ./Readarr/config:/config
          - ./Readarr/media:/media # Books can be stored here but should use a larger dedicated Shared drive instead
          - /mnt/Library/Readarr:/data/books # Media.
          - /mnt/Torrents:/downloads # Readarr needs to be able to see files downloaded via qBittorrent
        restart: unless-stopped
    
    #----------------------------------------------------------------------
    
      # Define YouTubeDL
      youtubedl:
        image: tzahi12345/youtubedl-material:latest
        container_name: My-YouTubeDL
        environment:
          - PUID=1000
          - PGID=1000
          - TZ=America/New_York
          #- USER:youtube
        ports:
          - 8084:17442
        volumes:
          - ./YouTubeDL/data/appdata:/app/appdata
          - ./YouTubeDL/data/subscriptions:/app/subscriptions
          - ./YouTubeDL/data/users:/app/users
          - /mnt/Media-Plex/YouTubeDL/audio:/app/audio
          - /mnt/Media-Plex/YouTubeDL/video:/app/video
        restart: unless-stopped

6. Configure DNS and DHCP

  1. See this doc to configure Pi-hole as both the DNS and DHCP server