How to Run Persistent Tasks on Ubuntu 22.04: Complete Guide to Background Process Management

When working with Ubuntu 22.04 servers, one of the most common challenges administrators face is running long-duration tasks that need to continue executing even after disconnecting from the SSH session. Whether you’re performing system maintenance, running data processing scripts, or executing backup operations, understanding how to properly manage persistent tasks is crucial for effective server administration.

In this comprehensive guide, we’ll explore multiple methods to run tasks that survive disconnection, each with their own advantages and use cases. By the end of this article, you’ll have mastered the art of background process management on Ubuntu 22.04.

Why Do Tasks Stop When You Disconnect?

Before diving into solutions, it’s important to understand why tasks typically terminate when you disconnect from an SSH session. When you close your terminal or lose connection, the system sends a hangup signal (SIGHUP) to all processes associated with your session. This behavior dates back to the early days of Unix when terminals were physical devices that could be literally “hung up.”

Modern Linux systems maintain this behavior for security and resource management reasons. However, there are several elegant solutions that allow processes to continue running independently of your session.

Method 1: tmux – The Modern Terminal Multiplexer

tmux (Terminal Multiplexer) is arguably the most powerful and user-friendly solution for managing persistent terminal sessions. It creates virtual terminal sessions that run independently of your SSH connection.

Installing tmux on Ubuntu 22.04

# Update package list
sudo apt update

# Install tmux
sudo apt install tmux

# Verify installation
tmux -V

Basic tmux Usage

Starting a new session and running a long task:

# Create a new named session
tmux new-session -s data-processing

# Inside tmux, run your long-running command
python3 /home/user/scripts/data_analysis.py

# Alternative: Run command directly when creating session
tmux new-session -s backup-job -d 'rsync -av /home/user/documents/ /backup/location/'

Essential tmux Commands

Key Bindings (default prefix is Ctrl+b):

Ctrl+b, d – Detach from session
Ctrl+b, c – Create new window
Ctrl+b, n – Switch to next window
Ctrl+b, p – Switch to previous window
Ctrl+b, % – Split window vertically
Ctrl+b, " – Split window horizontally

Session Management:

# List all sessions
tmux list-sessions
tmux ls

# Attach to specific session
tmux attach-session -t data-processing
tmux a -t data-processing

# Kill a session
tmux kill-session -t session-name

# Rename current session
tmux rename-session new-name

Advanced tmux Configuration

Create a custom tmux configuration file for better usability:

# Create config file
nano ~/.tmux.conf

Add these useful configurations:

# Change prefix key to Ctrl+a (more convenient)
unbind C-b
set-option -g prefix C-a
bind-key C-a send-prefix

# Enable mouse support
set -g mouse on

# Start windows and panes at 1, not 0
set -g base-index 1
setw -g pane-base-index 1

# Reload config file
bind r source-file ~/.tmux.conf \; display-message "Config reloaded!"

# Better window splitting
bind | split-window -h
bind - split-window -v

Reload the configuration:

tmux source-file ~/.tmux.conf

Practical tmux Example: Database Backup

Here’s a real-world example of using tmux for a database backup operation:

# Start a dedicated backup session
tmux new-session -s db-backup

# Inside tmux, run the backup command
mysqldump -u username -p database_name > /backup/db_backup_$(date +%Y%m%d_%H%M%S).sql

# Detach while backup is running
# Press Ctrl+b, then d

# Later, check on the backup progress
tmux attach-session -t db-backup

Method 2: GNU Screen – The Classic Solution

GNU Screen is the predecessor to tmux and remains widely used due to its stability and universal availability.

Installing and Using Screen

# Install screen
sudo apt install screen

# Start a new named session
screen -S log-analysis

# Run your command
tail -f /var/log/apache2/access.log | grep "404"

# Detach from screen session
# Press Ctrl+a, then d

# List active sessions
screen -ls

# Reattach to session
screen -r log-analysis

# Reattach to session if there's only one
screen -r

Screen Configuration

Create a .screenrc file for customization:

nano ~/.screenrc

Add useful configurations:

# Remove startup message
startup_message off

# Set scrollback buffer
defscrollback 10000

# Display a status line
hardstatus alwayslastline
hardstatus string '%{= kG}[%{G}%H%? %1`%?%{g}][%= %{= kw}%-w%{+b yk} %n*%t%?(%u)%? %{-}%+w %=%{g}][%{B}%m/%d %{W}%C%A%{g}]'

# Enable mouse scrolling
termcapinfo xterm* ti@:te@

Screen vs tmux Comparison

Feature	Screen	tmux
Vertical splits	Limited	Full support
Configuration	Simple	More flexible
Session sharing	Basic	Advanced
Scripting	Basic	Extensive
Development	Stable	Active

Method 3: nohup – Simple Background Execution

nohup (no hangup) is perfect for simple tasks that don’t require interaction. It runs commands immune to hangup signals.

Basic nohup Usage

# Run a command in the background with nohup
nohup python3 /home/user/scripts/data_processor.py > /home/user/logs/processing.log 2>&1 &

# Check if the process is still running
ps aux | grep python3

# Monitor the output
tail -f /home/user/logs/processing.log

Advanced nohup Examples

Running multiple commands:

# Create a script for complex operations
cat << 'EOF' > /home/user/scripts/maintenance.sh
#!/bin/bash
echo "Starting maintenance at $(date)"
apt update && apt upgrade -y
echo "System update completed at $(date)"
systemctl restart apache2
echo "Apache restarted at $(date)"
echo "Maintenance completed at $(date)"
EOF

chmod +x /home/user/scripts/maintenance.sh

# Run with nohup
nohup /home/user/scripts/maintenance.sh > /home/user/logs/maintenance.log 2>&1 &

Process monitoring:

# Find your background process
jobs -l
ps aux | grep your_process_name

# Kill a background process if needed
kill -9 PID_NUMBER

Method 4: systemd User Services – Professional Process Management

For tasks that need to run as services or restart automatically, systemd user services provide the most robust solution.

Creating a systemd User Service

# Create the user systemd directory
mkdir -p ~/.config/systemd/user

# Create a service file
nano ~/.config/systemd/user/data-sync.service

Service file content:

[Unit]
Description=Data Synchronization Service
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
Environment=HOME=/home/username
Environment=PATH=/usr/local/bin:/usr/bin:/bin
ExecStart=/usr/bin/python3 /home/username/scripts/sync_data.py
Restart=on-failure
RestartSec=10
StandardOutput=journal
StandardError=journal
WorkingDirectory=/home/username

[Install]
WantedBy=default.target

Managing systemd User Services

# Reload systemd user daemon
systemctl --user daemon-reload

# Enable the service (start on boot)
systemctl --user enable data-sync.service

# Start the service
systemctl --user start data-sync.service

# Check service status
systemctl --user status data-sync.service

# View service logs
journalctl --user -u data-sync.service -f

# Stop the service
systemctl --user stop data-sync.service

# Disable the service
systemctl --user disable data-sync.service

Advanced systemd Service Configuration

For more complex services, you can add additional configuration:

[Unit]
Description=Advanced Data Processing Service
After=network-online.target
Wants=network-online.target
Requires=postgresql.service

[Service]
Type=forking
Environment=HOME=/home/username
Environment=DATABASE_URL=postgresql://localhost/mydb
ExecStartPre=/bin/bash -c 'until pg_isready; do sleep 1; done'
ExecStart=/home/username/scripts/start_processor.sh
ExecReload=/bin/kill -HUP $MAINPID
PIDFile=/tmp/data-processor.pid
Restart=always
RestartSec=5
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=data-processor
User=username
Group=username
WorkingDirectory=/home/username

[Install]
WantedBy=default.target

Method 5: Using disown and bg Commands

For processes already running in your terminal, you can use disown and bg to make them persistent.

# Start a command normally
python3 /home/user/long_running_script.py

# Suspend it with Ctrl+Z

# Put it in background
bg

# Disown the process (make it independent)
disown %1

# Or combine: start command, suspend, background, and disown
python3 script.py
# Press Ctrl+Z
bg && disown

Choosing the Right Method

Decision Matrix

Scenario	Recommended Method	Why
Interactive monitoring needed	tmux or screen	Real-time interaction capability
Simple one-time task	nohup	Lightweight and straightforward
Production service	systemd	Automatic restart, logging, dependencies
Development/testing	tmux	Flexibility and ease of use
Legacy systems	screen	Universal availability
Already running process	disown + bg	Quick conversion to background

Performance Considerations

Resource Usage:

nohup: Minimal overhead
tmux/screen: Small memory footprint per session
systemd: Integrated with system management

Reliability:

systemd: Highest (automatic restart, dependency management)
tmux/screen: High (session persistence)
nohup: Medium (process-dependent)

Best Practices and Security Considerations

Security Best Practices

# Always use specific paths in scripts
/usr/bin/python3 /full/path/to/script.py

# Set proper permissions
chmod 750 /home/user/scripts/
chmod 640 /home/user/logs/

# Use dedicated user accounts for services
sudo useradd -r -s /bin/false service-user

Monitoring and Logging

# Monitor system resources
htop
iotop
nethogs

# Check logs
tail -f /var/log/syslog
journalctl -f

# Monitor specific processes
watch -n 1 'ps aux | grep your_process'

Backup and Recovery

Always ensure your long-running tasks have proper error handling and recovery mechanisms:

#!/bin/bash
# Example robust script
set -euo pipefail

LOG_FILE="/home/user/logs/process.log"
LOCK_FILE="/tmp/process.lock"

# Function to cleanup on exit
cleanup() {
    rm -f "$LOCK_FILE"
}
trap cleanup EXIT

# Prevent multiple instances
if [ -f "$LOCK_FILE" ]; then
    echo "Process already running" | tee -a "$LOG_FILE"
    exit 1
fi

echo $$ > "$LOCK_FILE"

# Your actual work here
echo "Starting process at $(date)" | tee -a "$LOG_FILE"
# ... your commands ...
echo "Process completed at $(date)" | tee -a "$LOG_FILE"

Troubleshooting Common Issues

Process Not Starting

# Check if the executable exists and has proper permissions
ls -la /path/to/your/script
which python3

# Verify environment variables
env | grep PATH

Process Stopping Unexpectedly

# Check system logs
journalctl -xe
dmesg | tail

# Monitor system resources
free -h
df -h

tmux/screen Sessions Not Persisting

# Ensure the session wasn't killed
tmux ls
screen -ls

# Check system limits
ulimit -a

Conclusion

Managing persistent tasks on Ubuntu 22.04 is a fundamental skill for system administrators and developers. Each method we’ve covered serves different use cases:

Use tmux for interactive, long-running tasks that require monitoring
Choose nohup for simple, fire-and-forget operations
Implement systemd services for production workloads requiring reliability
Fall back to screen on older systems or when tmux isn’t available

The key is understanding your specific requirements and choosing the appropriate tool. With these techniques in your toolkit, you’ll be able to efficiently manage background processes and maintain productivity even when working with unreliable network connections.

Remember to always test your process management strategy in a development environment before implementing it in production, and ensure proper logging and monitoring are in place for critical tasks.

Data/ML Engineer Blog