Backend Installation Guide
- Six steps: download, create a working directory,
--initand edit the two config files,--setup, get the generated files signed, then--all - The signing round trip blocks the install.
--allstops partway through without the returned.sigfiles - Complete the Pre-Installation Checklist first
Work through the Pre-Installation Checklist before running anything below. A missing DNS record or an untrunked VLAN surfaces partway through the Ansible run, after the installer has already changed the system.
Install
ticrypt-setup.sh is the only supported installation method.
The installer wraps a set of Ansible playbooks. Run it from an Ansible control node that can reach every backend, VM, and Slurm host over SSH.
The package and the working directory are two different places. The package holds the playbooks and is read-only. The working directory is one you create; it holds your inventory.ini, ticrypt.yml, and TLS files. The installer reads those files from the directory you invoke it in, so run every command below from the working directory. Keep it after installation. You need it for upgrades and repairs.
1. Download the package
wget https://storage.googleapis.com/ticrypt/install/ticrypt-setup-0.1.26.tgz
tar -xzf ticrypt-setup-0.1.26.tgz
This extracts to a versioned directory such as ticrypt-setup-0.1.26. It can live anywhere. Substitute its path for <package> below.
https://storage.googleapis.com/ticrypt/install/ticrypt-setup.tgz always points at the latest release. Use the versioned file for production installs, since the version number identifies which installer you ran when you need support.
2. Create the working directory
mkdir -p /opt/ticrypt-deploy
cd /opt/ticrypt-deploy
<package>/scripts/ticrypt-setup.sh --init
--init copies the example inventory.ini and ticrypt.yml into the current directory. Both are commented.
Copy your TLS certificate and private key here as well. ticrypt.yml refers to them by filename relative to this directory, not by absolute path.
3. Edit the configuration
Edit inventory.ini and ticrypt.yml before running any installation step. See Configuration below for what each file controls and which values must be changed.
4. Generate the files Tera Insights signs
This step reads your ticrypt.yml using yq, which must be installed on the control node, and
writes two files into the working directory:
<package>/scripts/ticrypt-setup.sh --setup
| File | Contents |
|---|---|
config.toml | Backend domain, secure network CIDR, and the PBKDF2 iteration count |
deployment.json | Deployment name and logo, base URL, session and key timeouts, self-registration token, and content security policy sources |
Both must be signed by Tera Insights before the frontend will accept them. The command prints the address to send them to.
5. Install the signed files
Send config.toml and deployment.json for signing, and put the returned config.toml.sig and
deployment.json.sig in the working directory alongside the originals.
This is not something you can defer until afterwards. configure-nginx copies config.toml.sig
out of the working directory, and check-net-match fails outright if it is missing, so --all
stops at that point on a first install and leaves the deployment half-configured.
Plan for the round trip. If you would rather not wait, run --start-at 1 and stop before
configure-nginx, then resume once the signed files arrive.
6. Run the installer
# review the steps without running anything
<package>/scripts/ticrypt-setup.sh --list
# run the full sequence
<package>/scripts/ticrypt-setup.sh --all
--all runs these playbooks in order. Each one targets the host groups you defined in inventory.ini.
The number is the position in the sequence, which is what --start-at and --only take:
| # | Playbook | Runs on | What it does |
|---|---|---|---|
| 1 | install-ticrypt-repos | All hosts | Adds the two Tera Insights package repositories. EPEL and MongoDB come later, in install-ticrypt |
| 2 | install-libvirt | VM and Slurm hosts | Installs the Libvirt virtualization stack |
| 3 | configure-vlans-backend | Backend | Creates the VLAN interfaces on the backend. Does nothing unless network_baseNIC is set, and the shipped inventory leaves it empty |
| 4 | install-ticrypt | Backend | Adds the EPEL and MongoDB repositories, installs MongoDB, NGINX, and the ticrypt-backend package, then enables the backend services |
| 5 | install-ovs | VM and Slurm hosts | Installs Open vSwitch |
| 6 | configure-ticrypt | Backend | Generates every service configuration file in /etc/ticrypt from your ticrypt.yml |
| 7 | configure-ovs | VM and Slurm hosts | Creates the Open vSwitch bridges and ports |
| 8 | configure-libvirt | VM and Slurm hosts | Configures the Libvirt networks, storage pools, and access controls |
| 9 | configure-network-backend | Backend | Configures dnsmasq for the virtual networks, and the firewalld zones, policies, and ipsets |
| 10 | configure-nginx | Backend | Installs and configures NGINX with your TLS certificate, and copies the signed config.toml and config.toml.sig into place |
| 11 | check-net-match | Backend | Verifies the network in the signed config.toml matches vm.internal-networks in ticrypt-rest.conf |
| 12 | install-slurm-controller | Backend | Installs Munge and the Slurm controller |
| 13 | install-slurm-worker | Slurm hosts | Installs slurmd on each Slurm host |
The Slurm controller runs on the backend, not on a Slurm host. This is why global.slurm_server_name defaults to backend.
configure-nginx and check-net-match need the signed filesBoth read config.toml.sig from the working directory, and check-net-match fails outright without
it. On a first install --all stops here unless you completed
step 5.
If a playbook fails, fix the cause and resume from that position rather than starting over:
<package>/scripts/ticrypt-setup.sh --start-at 6 # position 6 through the end
<package>/scripts/ticrypt-setup.sh --only 10 # position 10 alone
--only takes a single position; it has no range form. Extra arguments pass through to
ansible-playbook:
<package>/scripts/ticrypt-setup.sh --all --limit backend
If you are not using Slurm, omit the [slurm_hosts] group and install-slurm-worker has no
hosts to act on. install-slurm-controller targets the backend regardless, so avoiding it means
invoking --only once for each of positions 1 through 11 rather than using --all.
Steps run by name
Some playbooks sit outside the numbered sequence and are run with --explicit:
<package>/scripts/ticrypt-setup.sh --explicit system-presetup
| Name | What it does |
|---|---|
system-presetup | Preliminary OS and kernel setup on the backend, VM, and Slurm hosts |
inject-root-key | Places the root SSH key on every host. Prompts for the SSH password (--ask-pass) instead of using a key, so run it first on a new deployment to establish the access every other step needs |
install-tiaudit | Installs the tiAudit logging and auditing service on the hosts in [tiAudit] |
--list advertises that you should not run--list also advertises check-services and check-repo. Those playbooks do not ship in the current release and fail with "playbook not found". Use the Post-Installation Checklist instead.
install-mailbox-sftp and setup-backend-sftp ship in the package's ansible/ directory but are not reachable through ticrypt-setup.sh. Contact Tera Insights before running them directly.
Configuration
--init places both files in your working directory. Edit both before running any installation step.
inventory.ini
Declares which hosts exist and what role each plays.
| Group | Required | Purpose |
|---|---|---|
[backend] | Yes | The backend server. This group must always exist |
[vm_hosts] | Yes | Hosts that run secure VMs. At least one is required |
[slurm_hosts] | No | Hosts that run Slurm jobs. Omit the group if Slurm is not used |
[tiAudit] | No | Host that runs tiAudit. Usually the backend |
[ticrypt-mailbox] | No | Host that runs the mailbox ingress service. Usually the backend |
Per-host variables:
| Variable | Meaning |
|---|---|
ansible_host | IP address or hostname Ansible connects to over SSH |
bridge_secure_ip | The host's address on the secure bridge |
vnc_ip | Address the VNC console binds to on that host. Optional |
A [<group>:vars] block can override four specific values per group, so each group can name its own interfaces when hardware differs between hosts:
| Inventory variable | Overrides |
|---|---|
network_baseNIC | network.baseNIC |
network_secure_nic | network.secure.nic |
network_service_nic | network.service.nic |
network_datain_nic | network.datain.nic |
This is not a general mechanism. ticrypt.yml is passed to Ansible as extra-vars, which outrank
inventory variables, so only these four are read with an inventory fallback. Inventing others, such
as global_sqlPassword, has no effect.
inventory.ini: the file --init writes
# Options for each host
# - ansible_host: IP or hostname to connect to via SSH
# - bridge_secure_ip: IP address of the secure bridge interface on the host
# Override variables for the backend group
# The dot notation becomes _ in variable names.
# E.g. network.baseNIC becomes network_baseNIC
[backend:vars]
# No base NIC
network_baseNIC=""
# NIC to use for management traffic
network_secure_nic=ens3
# NIC to use for service traffic
network_service_nic=end4
# NIC to use for data-in traffic
network_datain_nic=end5
# This must always exist
[backend]
ticrypt_backend ansible_host=127.0.0.1
# Define VM hosts here. At least one must exist.
[vm_hosts]
ticrypt_vm1 ansible_host=10.22.122.2 bridge_secure_ip=192.168.128.2 vnc_ip=127.0.0.1
ticrypt_vm2 ansible_host=10.22.122.3 bridge_secure_ip=192.168.128.3
# Define Slurm hosts here. Can be missing if Slurm is not used.
[slurm_hosts]
ticrypt_slurm1 ansible_host=10.22.122.4 bridge_secure_ip=192.168.128.4
ticrypt_slurm2 ansible_host=10.22.122.5 bridge_secure_ip=192.168.128.5
[tiAudit]
ticrypt_backend
[ticrypt-mailbox]
ticrypt_backend
ticrypt.yml
Supplies every value the playbooks substitute into the service configuration files. The file --init writes is commented throughout. The table below maps its top-level sections.
| Section | Controls |
|---|---|
network | Firewall type, the base NIC, and the virtual networks: secure, service, datain, plus serviceLibvirt, which is the bridge the service network actually uses by default |
global | RPM token, backend domain, deployment name and logo, Libvirt pool and Vault storage paths, TLS filenames, SQL password, the ticrypt service account UID and GID, and the Slurm controller hostname |
features | Deployment-wide switches: key escrow, self-registration, mailboxes, MFA, split credentials, external logging, the account locker and garbage collectors, data-in VMs, and the licensing server |
mongodb | MongoDB credentials. Used only when features.mongoAuthentication is true |
auth | Self-registration token and the message shown to unapproved users, MFA provider details, and what happens on XSS detection |
logger | External log forwarding host, port, and timeouts |
maintenance | How often the account locker and each garbage collector run, retention periods, and the drive trash directory |
proxy | VM connection tunnel port range, session lifetime, whether sessions are single-use, and protocol, which gates whether ticrypt-nft is installed |
vm | Realms, hardware profiles, drive size limits and cache mode, path translation, and the licensing server type |
tiaudit, mailbox, sftp | Hostname and TLS files for each of those services. Only sftp takes a port |
rest, batch | Reserved. Empty in the current release |
Values you must change before the first run
The example file ships with placeholders that will not work as written.
| Value | Why |
|---|---|
global.rpmToken | Obtain from Tera Insights. Without it, step 1 cannot reach the package repositories and nothing installs |
global.backendDomain | Must match the DNS A record and the TLS certificate |
global.ssl_cert, global.ssl_key | Filenames in the working directory, not absolute paths |
global.sqlPassword | Ships as REPLACE_WITH_STRONG_PASSWORD |
mongodb.password | Ships as REPLACE_WITH_STRONG_PASSWORD. Only used when features.mongoAuthentication is true, but set it regardless |
auth.selfRegistration.token | Ships as REPLACE_ME. This token also goes into the signed deployment.json, so set it before step 4 |
auth.mfa.tokenSalt | Ships as REPLACE_ME |
tiaudit.authKey, tiaudit.encryptionKey | Ship as REPLACE_WITH_STRONG_PASSWORD |
Generate each secret with openssl rand -base64 32, or openssl rand -base64 64 for tiaudit.authKey.
Two more values need sizing rather than a generated string:
network.secure: the/17range you reserved in the pre-installation checklist.vm.hardwareProfilesmust match each VM host's actual resources. Reserve at least 2 cores and 8 GiB of RAM for the host OS, and list passthrough devices by their PCI address fromlspci. A profile claiming more than the host has lets tiCrypt schedule VMs the host cannot start.
ticrypt.yml: the sections you edit first
The full file is longer, and the values below are illustrative rather than a copy of what --init
writes. These are the two blocks that stop an installation if they are wrong.
## General setup
global: {
rpmToken: "ask Tera Insights for a token", # RPM token for accessing Tera Insights repositories
backendDomain: "ticrypt.mydomain.edu", # Domain name of the backend server
deploymentName: "My tiCrypt Deployment", # Human-readable name of the deployment
deploymentLogo: "https://example.edu/static/logo136.png",
poolsDirectory: "/storage/libvirt/pools", # Directory where libvirt storage pools are created
storagePath: "/mnt/storage/ticrypt/storage", # Path to the Vault storage. Must be accessible by the ticrypt user.
ssl_cert: "ticrypt.crt", # SSL certificate file name from current directory
ssl_key: "ticrypt.key", # SSL key file name from current directory
sqlPassword: "REPLACE_WITH_STRONG_PASSWORD",
ticryptUser: {
uid: 950,
gid: 950,
home: "/var/lib/ticrypt",
},
libvirtGroup: {
gid: 978,
},
# Slurm is very sensitive to this value
slurm_server_name: "backend", # Hostname of the Slurm controller, usually the backend
}
## Network configuration
network: {
firewall: "firewalld", # Supported values: "firewalld", "iptables", "none"
# Define this and vlans if you want the network nics to be created by ticrypt
baseNIC: "bond0",
# Used for all secure VMs (Slurm included). Must be a private large network
# with at least /17 addresses.
secure: {
bridge: "br-secure",
gateway: "192.168.128.1",
base: "192.168.128.0",
number: 17, # CIDR suffix
dhcpRange: "192.168.129.1,192.168.255.254",
nic: "bond0.1081", # Override per group in inventory.ini if needed
vlan: 1081,
},
# service and datain follow the same shape. /24 is usually sufficient for each.
}
Feature flags worth deciding deliberately
features sets the deployment's security posture. Not every default is right for production.
| Flag | Default | What it means |
|---|---|---|
auth.keyEscrow | true | Enables key escrow. Requires the site key. Without it, a user who loses their password loses their encrypted data permanently |
auth.selfRegistration | true | Lets users create their own accounts. New accounts start disabled and need admin approval, so this is safe to leave on |
auth.mfa | false | Multi-factor authentication. Enable it in production |
auth.splitCredentials | false | Stores the salt and IV used to encrypt each user's key on the backend, protected by MFA. Requires auth.mfa. Enable it wherever MFA is in place |
mongoAuthentication | false | Whether the services authenticate to MongoDB. Enable it unless MongoDB is otherwise isolated |
vm.licensingServer | true | Lets secure VMs reach specific licensed internet endpoints through a controlled path |
maintenance.garbageCollectors.* | true | Reclaim space from deleted files, drives, escrow keys, and directories. Leave enabled |
Post-Installation Checklist
Verify the installation
systemctl status ticrypt-auth ticrypt-batch ticrypt-file-manager ticrypt-logger ticrypt-maintenance ticrypt-notifications ticrypt-proxy ticrypt-rest ticrypt-stats ticrypt-vm)/etc/ticrypt for each of those services, and a realm file exists under /etc/ticrypt/realms for each realm in ticrypt.yml/var/log/ticryptticrypt-services.sh start and ticrypt-services.sh enable act on all backend services at once, and
install-ticrypt runs both. It then moves /etc/ticrypt aside and recreates it empty, so the
services are enabled at boot from that point but do not have their configuration until
configure-ticrypt writes it. Judge them only after the full sequence has run.
Review the service configuration
See NGINX TLS Configuration for certificate verification, Diffie-Hellman parameter generation, and SELinux setup, and Service Configuration for every available option.