In this article, we will be installing ETCD & Postgres.
Install Postgres (Only Postgres Nodes):
We will configure Postgresql later. Stop & disable postgresql service because patroni will handle the lifecycle of postgres and systemctl will not manage the service.
apt update
apt install -y postgresql-common
/usr/share/postgresql-common/pgdg/apt.postgresql.org.sh
apt update
apt install -y postgresql postgresql-contrib
systemctl stop postgresql
systemctl disable postgresql
Install ETCD (Only on ETCD Nodes):
Let's install curl on our etcd nodes.
apt install -y wget curl
You can check the ETCD release from the link below. I will be installing version 3.5.19. You can see the etcd packages from the Assets section. Right click on the link and copy the url address of the package you want to download.
https://github.com/etcd-io/etcd/releases
Download the package to etcd nodes
wget https://github.com/etcd-io/etcd/releases/download/v3.5.19/etcd-v3.5.19-linux-amd64.tar.gz
#Uncompress, rename the folder as etcd
tar xvf etcd-v3.5.19-linux-amd64.tar.gz
mv etcd-v3.5.19-linux-amd64 etcd
In etcd folder, we can see 3 binary files that starts with etcd.
I will move these 3 files to /usr/local/bin/. This way, they will be available globally.
mv etcd/etcd* /usr/local/bin/
#Check etcd version
etcd --version
etcdctl version
etcdutl version
Now, let's create a user for etcd service. This user’s home directory is /var/lib/etcd and this user can not sign in.
useradd --system --home /var/lib/etcd --shell /bin/false etcd
Create the following directories. /etc/etcd for the data . /etc/etcd/ssl for certificates.
mkdir -p /etc/etcd
mkdir -p /etc/etcd/ssl
In previous article, we have already created the certificates and copied them to /var/tmp/ folder. Let's move the certificates to /etc/etcd/ssl/ folder.
mv /var/tmp/etcd*.crt /etc/etcd/ssl/
mv /var/tmp/etcd*.key /etc/etcd/ssl/
mv /var/tmp/ca.crt /etc/etcd/ssl/
Set the permisssions and ownership
chown -R etcd:etcd /etc/etcd
chmod 600 /etc/etcd/ssl/etcd*.key
chmod 644 /etc/etcd/ssl/etcd*.crt /etc/etcd/ssl/ca.crt
I will create an environment file for ETCD. When ETCD service starts, it will use the parameters that are defined in env file. Initial Cluster state is "New". We will change it to "Existing" later on.
ETCD01:
nano /etc/etcd/etcd.envETCD_NAME="etcd01"
ETCD_DATA_DIR="/var/lib/etcd"
ETCD_INITIAL_CLUSTER="etcd01=https://192.168.204.13:2380,etcd02=https://192.168.204.14:2380,etcd03=https://192.168.204.15:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.204.13:2380"
ETCD_LISTEN_PEER_URLS="https://0.0.0.0:2380"
ETCD_LISTEN_CLIENT_URLS="https://0.0.0.0:2379"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.204.13:2379"
ETCD_CLIENT_CERT_AUTH="true"
ETCD_TRUSTED_CA_FILE="/etc/etcd/ssl/ca.crt"
ETCD_CERT_FILE="/etc/etcd/ssl/etcd01.crt"
ETCD_KEY_FILE="/etc/etcd/ssl/etcd01.key"
ETCD_PEER_CLIENT_CERT_AUTH="true"
ETCD_PEER_TRUSTED_CA_FILE="/etc/etcd/ssl/ca.crt"
ETCD_PEER_CERT_FILE="/etc/etcd/ssl/etcd01.crt"
ETCD_PEER_KEY_FILE="/etc/etcd/ssl/etcd01.key"
ETCD02:
ETCD_NAME="etcd02"
ETCD_DATA_DIR="/var/lib/etcd"
ETCD_INITIAL_CLUSTER="etcd01=https://192.168.204.13:2380,etcd02=https://192.168.204.14:2380,etcd03=https://192.168.204.15:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.204.14:2380"
ETCD_LISTEN_PEER_URLS="https://0.0.0.0:2380"
ETCD_LISTEN_CLIENT_URLS="https://0.0.0.0:2379"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.204.14:2379"
ETCD_CLIENT_CERT_AUTH="true"
ETCD_TRUSTED_CA_FILE="/etc/etcd/ssl/ca.crt"
ETCD_CERT_FILE="/etc/etcd/ssl/etcd02.crt"
ETCD_KEY_FILE="/etc/etcd/ssl/etcd02.key"
ETCD_PEER_CLIENT_CERT_AUTH="true"
ETCD_PEER_TRUSTED_CA_FILE="/etc/etcd/ssl/ca.crt"
ETCD_PEER_CERT_FILE="/etc/etcd/ssl/etcd02.crt"
ETCD_PEER_KEY_FILE="/etc/etcd/ssl/etcd02.key"
ETCD03:
ETCD_NAME="etcd03"
ETCD_DATA_DIR="/var/lib/etcd"
ETCD_INITIAL_CLUSTER="etcd01=https://192.168.204.13:2380,etcd02=https://192.168.204.14:2380,etcd03=https://192.168.204.15:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.204.15:2380"
ETCD_LISTEN_PEER_URLS="https://0.0.0.0:2380"
ETCD_LISTEN_CLIENT_URLS="https://0.0.0.0:2379"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.204.15:2379"
ETCD_CLIENT_CERT_AUTH="true"
ETCD_TRUSTED_CA_FILE="/etc/etcd/ssl/ca.crt"
ETCD_CERT_FILE="/etc/etcd/ssl/etcd03.crt"
ETCD_KEY_FILE="/etc/etcd/ssl/etcd03.key"
ETCD_PEER_CLIENT_CERT_AUTH="true"
ETCD_PEER_TRUSTED_CA_FILE="/etc/etcd/ssl/ca.crt"
ETCD_PEER_CERT_FILE="/etc/etcd/ssl/etcd03.crt"
ETCD_PEER_KEY_FILE="/etc/etcd/ssl/etcd03.key"
Now let’s create a service for etcd on all 3 nodes
nano /etc/systemd/system/etcd.service[Unit]
Description=etcd key-value store
Documentation=https://github.com/etcd-io/etcd
After=network-online.target
Wants=network-online.target
[Service]
Type=notify
WorkingDirectory=/var/lib/etcd
EnvironmentFile=/etc/etcd/etcd.env
ExecStart=/usr/local/bin/etcd
Restart=always
RestartSec=10s
LimitNOFILE=40000
User=etcd
Group=etcd
[Install]
WantedBy=multi-user.target
We need to create a directory for etcd ETCD_DATA_DIR defined in environment file.
mkdir -p /var/lib/etcd
chown -R etcd:etcd /var/lib/etcd
Now we can start ETCD Service on ETCD Nodes
systemctl daemon-reload
systemctl enable etcd
systemctl start etcd
systemctl status etcd
We can check the logs
journalctl -xeu etcd.service
Check Health & Members:
etcdctl \
--endpoints=https://127.0.0.1:2379 \
--cacert=/etc/etcd/ssl/ca.crt \
--cert=/etc/etcd/ssl/etcd01.crt \
--key=/etc/etcd/ssl/etcd01.key \
endpoint health
etcdctl \
--endpoints=https://127.0.0.1:2379 \
--cacert=/etc/etcd/ssl/ca.crt \
--cert=/etc/etcd/ssl/etcd01.crt \
--key=/etc/etcd/ssl/etcd01.key \
member list
Check the Leader
etcdctl \
--endpoints=https://192.168.204.13:2379,https://192.168.204.14:2379,https://192.168.204.15:2379 \
--cacert=/etc/etcd/ssl/ca.crt \
--cert=/etc/etcd/ssl/etcd01.crt \
--key=/etc/etcd/ssl/etcd01.key \
endpoint status --write-out=table