A pipeline component that only runs while your SSH session is open is not deployed - it's a demo. On Ubuntu the reliable way to keep a process alive across logouts, crashes, and reboots is a systemd service, with systemd timers or cron for work that runs on a schedule. The throwaway tools (nohup, tmux) have their place for ad-hoc runs, but production background work belongs to the init system, which restarts the process when it dies and captures its logs.
systemd service units
A service is described by a unit file in /etc/systemd/system/ ending in .service. It tells systemd what to run, which user to run it as, the working directory, environment, and crucially what to do when the process exits. Below is a real unit for a long-running pipeline worker - a Python feed consumer - that restarts automatically and starts at boot:
[Unit]
Description=Feed consumer pipeline worker
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User=dundee
WorkingDirectory=/usr/local/www/pipeline
Environment="PYTHONUNBUFFERED=1"
EnvironmentFile=/usr/local/www/pipeline/.env
ExecStart=/usr/local/www/pipeline/venv/bin/python consumer.py
Restart=always
RestartSec=5
StartLimitIntervalSec=60
StartLimitBurst=5
[Install]
WantedBy=multi-user.targetAfter dropping the file in place you run sudo systemctl daemon-reload to pick it up, sudo systemctl enable --now feed-consumer to start it and mark it for boot, and read its output with journalctl -u feed-consumer -f. The init system now owns the process lifecycle.
Restart policies that matter
The Restart= directive is the difference between a self-healing worker and a silent outage. Restart=always brings the process back no matter how it exited; Restart=on-failure restarts only on a non-zero exit or signal, which is right for a job that's supposed to finish cleanly. RestartSec sets the delay before each restart so a tight crash loop doesn't hammer the CPU. The StartLimitIntervalSec and StartLimitBurst pair acts as a circuit breaker - if the unit restarts five times in sixty seconds, systemd gives up and marks it failed rather than looping forever, which surfaces a genuinely broken deployment instead of hiding it behind endless restarts.
Timers versus cron
For work that runs on a schedule rather than continuously - a nightly Parquet-to-ClickHouse load, an hourly feed pull - you have two options. cron is the classic: one line in a crontab, minimal ceremony, and the right tool for a simple recurring command. A systemd timer is a pair of units (a .timer that triggers a .service) and is heavier to set up, but it integrates with journald logging, can run a missed job after downtime with Persistent=true, supports calendar expressions and randomized delays, and shows up in systemctl list-timers alongside everything else. Choose cron for a one-liner; choose a timer when you want the job managed, logged, and observable like any other service.
nohup and tmux for ad-hoc runs
When you just need a one-off backfill to outlive your terminal, the lightweight options work. nohup python backfill.py > backfill.log 2&>1 & detaches the process from the session and redirects its output to a file. A tmux (or screen) session keeps an interactive shell alive that you can reattach to later with tmux attach, which is handy for watching a long migration. Neither survives a reboot and neither restarts on crash - so they're fine for a supervised one-time task, but anything that needs to be running tomorrow morning belongs in a systemd unit.
Frequently asked questions
Why not just use nohup for production? nohup keeps a process alive past logout but does nothing when the process crashes or the server reboots. systemd restarts it on both, captures logs, and lets you query its status - which is what "always running" actually requires.
When should I use a systemd timer instead of cron? Use a timer when you want the job logged in journald, able to catch up after downtime (Persistent=true), and visible in systemctl list-timers. Use cron for a simple one-line recurring command with no extra requirements.
What does Restart=always do during a crash loop? It restarts the process every time it exits, but StartLimitBurst/StartLimitIntervalSec cap the attempts - exceed them and systemd marks the unit failed instead of looping, so a broken deploy becomes visible rather than silently thrashing.
How do I see a service's logs? journalctl -u <service-name>, add -f to follow live output and --since "1 hour ago" to scope by time. systemd captures stdout and stderr automatically, so no separate log-file plumbing is needed.