Lab: Install MicroShift from RPM Packages

Estimated reading time: 13 minutes.

Objective: Install MicroShift in a RHEL VM that is not connected to the internet.

Before You Begin

You need a test machine with RHEL and unrestricted sudo, on which you will install MicroShift.

You also need a package server machine already configured to serve DNF repositories for RHEL, Fast Datapath for RHEL, and Red Hat OpenShift Container Platform.

Finally, you need a mirror registry machine with RHEL and unrestricted sudo, already populated with the container iamges required bu MicroShoft and sample applications.

Make sure that your package server machine and mirror registry machine are properly configured and verified by following the instructions from the previous lab of this course.

These instructions were tested on RHEL 9.5 but should work with minimal or no change on newer and older RHEL 9.x releases.

If you are using the course classroom, log in to the workstation VM as the user student with password student, and start SSH sessions from workstation VM to the serverb VM, which is your test machine, using the same user. If not, please adapt the instructions to your test environment.

Perform all steps in this lab on your test machine.

Instructions

Check that the required package repositories from RHEL and OpenShift are available.

On the classroom environment, you will find configuration files which point to the package server machine and the names of its DNF repositories match the names of Red Hat DNF repositories.

$ ls /etc/yum.repos.d
fast-datapath-for-rhel-9-x86_64-rpms.repo  rhel-9.5-for-x86_64-baseos-rpms.repo
rhel-9.5-for-x86_64-appstream-rpms.repo    rhocp-4.17-for-rhel-9-x86_64-rpms.repo
$ cat /etc/yum.repos.d/rhocp-4.17-for-rhel-9-x86_64-rpms.repo
[rhocp-4.17-for-rhel-9-x86_64-rpms]
baseurl = http://content.example.com/rhde/rpms/rhocp-4.17-for-rhel-9-x86_64-rpms
enabled = true
...

If you were connected to the Internet, or to a Red Hat Satellite instance, and had registered your test machine with a valid Red Hat subscription, you would notice something similar to:

$ sudo dnf repolist
Updating Subscription Management repositories.
repo id                                                         repo name
fast-datapath-for-rhel-9-x86_64-rpms                            Fast Datapath for RHEL 9 x86_64 (RPMs)
rhel-9-for-x86_64-appstream-rpms                                Red Hat Enterprise Linux 9 for x86_64 - AppStream (RPMs)
rhel-9-for-x86_64-baseos-rpms                                   Red Hat Enterprise Linux 9 for x86_64 - BaseOS (RPMs)
rhocp-4.17-for-rhel-9-x86_64-rpms                               Red Hat OpenShift Container Platform 4.17 for RHEL 9 x86_64 (RPMs)

Please do not register any VM on your virtual labs. Conserve the limited internet bandwidth that is shared by all learners using ROLE.

As an additional validation, search for packages from the OpenShift and Fast Datapath repositories.

$ dnf search microshift
...
microshift.x86_64 : MicroShift service
microshift-greenboot.noarch : Greenboot components for MicroShift
...
$ dnf search ovn24.09-central
...
ovn24.09-central.x86_64 : Open Virtual Network support

Verify that your test machine can access a mirror container image registry that is pre-populated with the container images required by MicroShift.

Install the container-tools package, if it is not already installed on your test machine.

This is not strictly required to run MicroShift; however, it provides troubleshooting tools, which can be valuable in a development environment.
```
$ sudo dnf install container-tools
...
Complete!
```
Add the CA key of the mirror registry to the set of trusted CAs. The public CA key is available from the web server machine, which in the classroom environment is also the mirror registry machine.
```
$ wget -q http://servera.lab.example.com/quay-rootCA.pem
$ sudo cp quay-rootCA.pem /etc/pki/ca-trust/source/anchors
$ sudo update-ca-trust
```

Create a pull secret, which enables access to the MicroShift container images.

In a deployment connected to the internet, you would get a pull secret from your Red Hat account from the Red Hat Hybrid Cloud Console.

Please, do not use your pull secret with VMs on your virtual labs. Please stick to the scenario of an air-gaped deployment of MicroShift, assuming that an IT Operations team already configured all local network infrastructure required for deploying Red Hat Device Edge.

For the classroom environment, the pull secret for the mirror registry is also available from GitHub

$ podman login -u microshift -p redhat123 servera.lab.example.com:8443
Login Succeeded!
$ cp $XDG_RUNTIME_DIR/containers/auth.json mirror-pull-secret

Check that you can use the pull secret to inspect container images on the mirror registry. It should work with TLS validation enabled, which is the default for all container tools.
```
$ skopeo inspect --authfile mirror-pull-secret -f '{{.RepoTags}}' docker://servera.lab.example.com:8443/ubi9/ubi
[latest]
```
The command would work with your actual Red Hat pull secret if you change the registry part of the container image name to redhat.registry.io; however, it would display a long list of tags.

Verify that your test machine is configured to support the installation of MicroShift.

MicroShift expects that SELinux is enabled, in enforcing mode, and that Firewalld is enabled. Please do not disable any of them; there’s no need!

$ getenforce
Enforcing
$ systemctl is-enabled firewalld
enabled
$ sudo firewall-cmd --list-all
[sudo] password for student:
public (active)
  target: default
  icmp-block-inversion: no
  interfaces: eth0 eth1
  sources:
  services: cockpit dhcpv6-client pulseaudio ssh
  ports: 8888/tcp
  protocols:
  forward: no
  masquerade: no
  forward-ports:
  source-ports:
  icmp-blocks:
  rich rules:

Enable more services and trusted zones or interfaces as you need. Later in this course, you’ll learn what to add to enable remote access to MicroShift and Kubernetes applications on it.

Check that you have two CPU cores, or two virtual CPUs, and at least 2GB of memory. If you are using the classroom environment, you have more memory than required on the serverb VM.

$ nproc
2
$ free -h
               total        used        free      shared  buff/cache   available
Mem:           3.6Gi       803Mi       2.6Gi        27Mi       456Mi       2.8Gi
Swap:             0B          0B          0B

Check that you have some GB of free disk space on whatever file system holds /var/lib/containers so you can store container images and ephemeral storage layers from CRI-O containers.

In the classroom environment, it is on the root partition of the serverb VM.
```
$ df -h /var/lib/containers
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda4       9.4G  2.6G  6.8G  28% /
```
Check that you have a volume group named rhel with a few GB of unallocated space, so you can create Persistent Volumes (PVs) for your Kubernetes applications.
```
$ sudo vgs
  VG   #PV #LV #SN Attr   VSize   VFree
  rhel   1   0   0 wz--n- <10.00g <10.00g
```

Check that your test machine is not using the network ranges used by the Kubernetes virtual networks or the OpenShift ingress IP address, which are 10.42.0.0/16 to 10.44.0.0/16 and 169.254.169.1.

$ ip -br addr show
lo               UNKNOWN        127.0.0.1/8 ::1/128
eth0             UP             172.25.250.11/24 fe80::5054:ff:fe00:fa0b/64
eth1             UP             10.80.0.156/16 fe80::2d83:182f:c10a:438b/64
$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         172.25.250.254  0.0.0.0         UG    100    0        0 eth0
10.80.0.0       0.0.0.0         255.255.0.0     U     101    0        0 eth1
169.254.169.254 10.80.0.10      255.255.255.255 UGH   101    0        0 eth1
172.25.250.0    0.0.0.0         255.255.255.0   U     100    0        0 eth0

Install the MicroShift packages.

You need only one package, called microshift, but pay attention to its dependency list. Notice that it includes packages from OpenShift, such as cri-o; packages from RHEL for Edge, like greenboot; and also packages from Fast Datapath for RHEL 9, which provide Open Virtual Networking (OVN).

$ sudo dnf install microshift
...
Installing:
 microshift                            x86_64      4.17.3-202410241116.p0.gec0b5ea.assembly.4.17.3.el9        rhocp-4.17-for-rhel-9-x86_64-rpms          53 M
Installing dependencies:
 NetworkManager-ovs                    x86_64      1:1.46.0-4.el9_4                                           rhel-9.4-for-x86_64-appstream-rpms         65 k
 conntrack-tools                       x86_64      1.4.7-2.el9                                                rhel-9.4-for-x86_64-appstream-rpms        239 k
 cri-o                                 x86_64      1.30.6-6.rhaos4.17.git6ac6e96.el9                          rhocp-4.17-for-rhel-9-x86_64-rpms          18 M
 cri-tools                             x86_64      1.30.0-5.el9                                               rhocp-4.17-for-rhel-9-x86_64-rpms          10 M
 greenboot                             x86_64      0.15.4-1.el9                                               rhel-9.4-for-x86_64-appstream-rpms         41 k
 libnetfilter_cthelper                 x86_64      1.0.0-22.el9                                               rhel-9.4-for-x86_64-appstream-rpms         26 k
 libnetfilter_cttimeout                x86_64      1.0.0-19.el9                                               rhel-9.4-for-x86_64-appstream-rpms         25 k
 libnetfilter_queue                    x86_64      1.0.5-1.el9                                                rhel-9.4-for-x86_64-appstream-rpms         31 k
 microshift-greenboot                  noarch      4.17.3-202410241116.p0.gec0b5ea.assembly.4.17.3.el9        rhocp-4.17-for-rhel-9-x86_64-rpms          21 k
 microshift-networking                 x86_64      4.17.3-202410241116.p0.gec0b5ea.assembly.4.17.3.el9        rhocp-4.17-for-rhel-9-x86_64-rpms          21 k
 microshift-selinux                    noarch      4.17.3-202410241116.p0.gec0b5ea.assembly.4.17.3.el9        rhocp-4.17-for-rhel-9-x86_64-rpms          26 k
 openshift-clients                     x86_64      4.17.0-202410161505.p0.g897ef0b.assembly.stream.el9        rhocp-4.17-for-rhel-9-x86_64-rpms          54 M
 openvswitch-selinux-extra-policy      noarch      1.0-36.el9fdp                                              fast-datapath-for-rhel-9-x86_64-rpms       11 k
 openvswitch3.4                        x86_64      3.4.0-1.el9fdp                                             fast-datapath-for-rhel-9-x86_64-rpms      6.8 M
 rpm-ostree                            x86_64      2024.3-1.el9                                               rhel-9.4-for-x86_64-appstream-rpms        4.1 M
 rpm-ostree-libs                       x86_64      2024.3-1.el9                                               rhel-9.4-for-x86_64-appstream-rpms         25 k
 runc                                  x86_64      4:1.1.14-2.rhaos4.17.el9                                   rhocp-4.17-for-rhel-9-x86_64-rpms         3.2 M
 unbound-libs                          x86_64      1.16.2-3.el9_3.3                                           rhel-9.4-for-x86_64-appstream-rpms        553 k

Transaction Summary
==============================================================================================================================================================
Install  19 Packages
...
Complete!

Do not try to start MicroShift yet!

Configure MicroShift to use your mirror registry.

Check the presence of the MicroShift configuration directory. You do not need to make any changes, and the files there just inform about their default configuration values.
```
$ ls /etc/microshift
config.yaml.default  lvmd.yaml.default  manifests  manifests.d  ovn.yaml.default
```
Copy the pull secret to the CRI-O configuration directory, so MicroShift uses it for pulling container images for any pod, in any namespace.
```
$ sudo cp mirror-pull-secret /etc/crio/openshift-pull-secret
$ sudo chmod 600 /etc/crio/openshift-pull-secret
```

Download a CRI-O registry configuration, from the course samples repository, that redirects container images from Red Hat Registries and Quay.io to the mirror registry.

$ wget -q https://raw.githubusercontent.com/RedHatQuickCourses/rhde-build-samples/refs/heads/main/microshift/999-microshift-mirror.conf
$ head -n 5 999-microshift-mirror.conf
[[registry]]
    prefix = ""
    location = "servera.lab.example.com:8443" 
    mirror-by-digest-only = true
    insecure = false
...

This registry configuration only affects MicroShift release iamges. You may want to change it to include images from add-on operators and your applications. In this course, we will refer to application images by the mirror registry.

Copy the registry configuration to the container tools registries configuration directory.

$ sudo cp 999-microshift-mirror.conf /etc/containers/registries.conf.d/999-microshift-mirror.conf

Download a CRI-O image policy, from the course samples repository, that allows running any container image, from anywhere, without checking image signatures.

$ wget -q https://raw.githubusercontent.com/RedHatQuickCourses/rhde-build-samples/refs/heads/main/microshift/containers-policy.json.nosigs
$ cat containers-policy.json.nosigs
{
	"default": [
    	{
        	"type": "insecureAcceptAnything"
    	}
	],
	"transports":
    	{
        	"docker-daemon":
            	{
                	"": [{"type":"insecureAcceptAnything"}]
            	}
    	}
}

Copy the image policy to the container tools registries configuration directory.
```
$ sudo cp containers-policy.json.nosigs /etc/containers/policy.json
```
If you skip that step, some Red Hat add-on operator images will fail to pull from the mirror registry.

Configure the system firewall and start MicroShift.

Allow network traffic on the Kubernetes virtual network.

This is not yet allowing network traffic from outside of your test machine to MicroShift and its applications. It is only allowing network traffic between Kubernetes pods and between the test machine and MicroShift.

$ sudo firewall-cmd --permanent --zone=trusted --add-source=10.42.0.0/16
success
$ sudo firewall-cmd --permanent --zone=trusted --add-source=169.254.169.1
success
$ sudo firewall-cmd --reload

Enable and start MicroShift.

Be patient, it will take a few moments for MicroShift to generate Kubernetes configuration files, pull all its containers, and starts it daemons and pods. Note that it will not be ready yet when the systemctl command returns.

$ sudo systemctl enable microshift --now
Created symlink /etc/systemd/system/multi-user.target.wants/microshift.service → /usr/lib/systemd/system/microshift.service.
Created symlink /etc/systemd/system/multi-user.target.wants/microshift-cleanup-kubelet.service → /usr/lib/systemd/system/microshift-cleanup-kubelet.service.

Check that the microshift service, which includes the Kubernetes core daemons, has started, and shows no errors on its logs. Be aware that having a healthy MicroShift service does not mean it’s fully ready and available.

$ sudo systemctl status microshift
● microshift.service - MicroShift
     Loaded: loaded (/usr/lib/systemd/system/microshift.service; enabled; preset: disabled)
     Active: active (running) since Thu 2024-11-21 22:30:19 UTC; 4s ago
...

Verify that MicroShift is fully initialized.

Copy the autogenerated kubeconfig file to your home directory and change its ownership so you can read it from your unprivileged user. Do not let that file stay read-write; ensure it remains unchanged so you can use it in an emergency.
```
$ sudo cp /var/lib/microshift/resources/kubeadmin/kubeconfig ~/local-admin
$ sudo chown student:student ~/local-admin
$ chmod a-w ~/local-admin
```

Check that all pods running on MicroShift are running and all their containers are ready. It may take a while, so repeat the following command until all pods are fully ready and running.

$ oc --kubeconfig ~/local-admin get pod -A
NAMESPACE              	NAME                                   	READY   STATUS	RESTARTS    	AGE
kube-system            	csi-snapshot-controller-69ddff88c8-hr9dt   1/1 	Running   0           	6m37s
kube-system            	csi-snapshot-webhook-74dc497864-hfdw7  	1/1 	Running   0           	6m41s
openshift-dns          	dns-default-9z4ph                      	2/2 	Running   0           	5m55s
openshift-dns          	node-resolver-prvjx                    	1/1 	Running   0           	6m39s
openshift-ingress      	router-default-575b4fc7-tg5lz          	1/1 	Running   0           	6m37s
openshift-ovn-kubernetes   ovnkube-master-wj4gv                   	4/4 	Running   1 (5m57s ago)   6m39s
openshift-ovn-kubernetes   ovnkube-node-fdpm8                     	1/1 	Running   1 (5m58s ago)   6m39s
openshift-service-ca   	service-ca-9db855698-zd7vg             	1/1 	Running   0           	6m36s
openshift-storage      	lvms-operator-7f544467bc-cqf2n         	1/1 	Running   0           	6m40s
openshift-storage      	vg-manager-tjrs2                       	1/1 	Running   0           	5m25s

As an additional validation, verify that your Kubernetes cluster contains one node that takes the roles of both control plane and worker node.

$ oc --kubeconfig ~/local-admin get node
NAME 	STATUS   ROLES                     	AGE 	VERSION
rhelvm   Ready	control-plane,master,worker   7m31s   v1.30.5

Check the presence of the virtual network bridges and interfaces used by Kubernetes virtual networks.

$ ip -br addr show
lo               UNKNOWN        127.0.0.1/8 ::1/128
eth0             UP             172.25.250.11/24 fe80::5054:ff:fe00:fa0b/64
eth1             UP             10.80.0.156/16 fe80::2d83:182f:c10a:438b/64
ovs-system       DOWN
br-ex            UNKNOWN        10.44.0.0/32 169.254.169.2/29
br-int           UNKNOWN
ovn-k8s-mp0      UNKNOWN        10.42.0.2/24 fe80::50:70ff:feaa:25e2/64
0ef8475e7471f6c@if2 UP             fe80::f860:c6ff:fe76:39ce/64
23e82273294fb87@if2 UP             fe80::24b6:c5ff:fe6b:3086/64
...

The output will be longer as you create more pods, because each one gets a virtual network interface. Notice the two kernel bridges, named br-ex and br-int, and the OVN switch, named ovn-k8s-mp0.

Check the presence of the routing rules, which enable communication between your real networks and Kubernetes virtual networks.

$ sudo route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         172.25.250.254  0.0.0.0         UG    100    0        0 eth0
10.42.0.0       0.0.0.0         255.255.255.0   U     0      0        0 ovn-k8s-mp0
10.42.0.0       10.42.0.1       255.255.0.0     UG    0      0        0 ovn-k8s-mp0
10.43.0.0       169.254.169.4   255.255.0.0     UG    0      0        0 br-ex
10.80.0.0       0.0.0.0         255.255.0.0     U     101    0        0 eth1
169.254.169.0   0.0.0.0         255.255.255.248 U     0      0        0 br-ex
169.254.169.1   0.0.0.0         255.255.255.255 UH    0      0        0 br-ex
169.254.169.3   10.42.0.1       255.255.255.255 UGH   0      0        0 ovn-k8s-mp0
169.254.169.254 10.80.0.10      255.255.255.255 UGH   101    0        0 eth1
172.25.250.0    0.0.0.0         255.255.255.0   U     100    0        0 eth0

Finally, check that you can create new pods.

$ oc --kubeconfig ~/local-admin run shell -it --restart Never --image servera.lab.example.com:8443/ubi9/ubi -- rpm -q redhat-release
redhat-release-9.5-0.6.el9.x86_64
$ oc --kubeconfig ~/local-admin delete pod shell
pod "shell" deleted

You now have a fully functional deployment of MicroShift in a RHEL machine. If you find that any of the MicroShift pods failed to start, the most likely cause is an error in your registry mirror and image policy configurations.

The way you configured MicroShift, it does not contain a pull secret to download container images from Red Hat, even if it has access to the internet. It can only pull containers from a mirror registry, which is usually what organizations with strict security policies would demand.

Next Steps

You probably wouldn’t want to perform development from the machine running MicroShift, so in the next activity, you will configure MicroShift for remote access and also an unprivileged user, without full Kubernetes administration rights.