Lab: Access MicroShift as a Developer or Platform Engineer

Estimated reading time: 10 minutes.

Objective

Configure remote access to MicroShift as privileged and unprivileged users.

Before You Begin

You need a test machine where you installed, configured, and verified MicroShift, by following the instructions from the previous lab.

You also need a development machine from which you will access the MicroShift instance on your test machine.

Finally, you need a mirror registry machine with RHEL and unrestricted sudo, already populated with the container iamges required bu MicroShoft and sample applications.

Make sure that your mirror registry machine was configured and verified by following the instructions from a previous lab of this course.

These instructions were tested on RHEL 9.5 but should work with minimal or no change on newer and older RHEL 9.x releases.

If you are using the course classroom, log in to the workstation VM, which is your development machine, as the user student with password student, and start SSH sessions from workstation VM to the serverb VM, which is your test machine, using the same user. If not, please adapt the instructions to your test environment.

You perform the initial steps of this lab on your test machine and the final steps on your development machine.

Instructions

  1. Log in to your test machine and verify that MicroShift is up and running.

    1. Check that the MicroShift service is active.

      $ systemctl is-active microshift
active

    2. Check that all pods are running, and that all their containers are ready and running.

      $ oc --kubeconfig ~/local-admin get pod -A
NAMESPACE              	NAME                                   	READY   STATUS	RESTARTS    	AGE
kube-system            	csi-snapshot-controller-69ddff88c8-hr9dt   1/1 	Running   0           	6m37s
kube-system            	csi-snapshot-webhook-74dc497864-hfdw7  	1/1 	Running   0           	6m41s
openshift-dns          	dns-default-9z4ph                      	2/2 	Running   0           	5m55s
openshift-dns          	node-resolver-prvjx                    	1/1 	Running   0           	6m39s
openshift-ingress      	router-default-575b4fc7-tg5lz          	1/1 	Running   0           	6m37s
openshift-ovn-kubernetes   ovnkube-master-wj4gv                   	4/4 	Running   1 (5m57s ago)   6m39s
openshift-ovn-kubernetes   ovnkube-node-fdpm8                     	1/1 	Running   1 (5m58s ago)   6m39s
openshift-service-ca   	service-ca-9db855698-zd7vg             	1/1 	Running   0           	6m36s
openshift-storage      	lvms-operator-7f544467bc-cqf2n         	1/1 	Running   0           	6m40s
openshift-storage      	vg-manager-tjrs2                       	1/1 	Running   0           	5m25s
      Remember that you copied the autogenerated kubeconfig file to ~/local-admin in the previous lab.

  2. Configure access to a developer user by creating a namespace and a service account with limited read-write access to that namespace.

    1. Create a namespace named userns and a service account named user in that namespace, and then grant to the service account rights to create application resources on the namespace.

      $ export KUBECONFIG=~/local-admin
$ oc whoami
system:admin
$ oc create namespace userns
namespace/myproj created
$ oc create sa user -n userns
serviceaccount/user created
$ oc create rolebinding user --clusterrole edit --serviceaccount userns:user -n userns

    2. Use Kubernetes impersonation to verify that the service account has the intended rights and nothing more. First, check that the service account has access to the namespace but not to other namespaces, or to non-namespaced resources.

      $ oc --as system:serviceaccount:userns:user get sa -n userns
NAME      SECRETS   AGE
default   0         8m48s
user      0         6m58s
$ oc --as system:serviceaccount:userns:user get sa -n default
Error from server (Forbidden): serviceaccounts is forbidden: User "system:serviceaccount:userns:user" cannot list resource "serviceaccounts" in API group "" in the namespace "default"
$ oc --as system:serviceaccount:userns:user get node
Error from server (Forbidden): nodes is forbidden: User "system:serviceaccount:userns:user" cannot list resource "nodes" in API group "" at the cluster scope

    3. Check that the service account can manage common application resources but not policy resources.

      $ oc --as system:serviceaccount:userns:user create configmap test1 --from-literal one=1 -n userns
configmap/test1 created
$ oc --as system:serviceaccount:userns:user get configmap -n userns
NAME                       DATA   AGE
kube-root-ca.crt           1      15m
openshift-service-ca.crt   1      15m
test1                      1      11s
$ oc --as system:serviceaccount:userns:user create quota test --hard pods=2 -n userns
error: failed to create quota: resourcequotas is forbidden: User "system:serviceaccount:userns:user" cannot create resource "resourcequotas" in API group "" in the namespace "userns"
      To grant rights to manage policy resources, such as quotas and network policies, to a namespace administrator, platform engineer, or site reliability engineer role, you must grant the admin cluster role instead of the edit cluster role.

  3. Create another namespace and grant limited read-only access to the service account.

    Remember that managing namespaces requires cluster administration rights in Kubernetes and MicroShift does NOT include self-service capabilities from OpenShift.

    1. Create a second namespace and grant only the view role to the service account.

      $ oc create namespace project
namespace/project created
$ oc create rolebinding user --clusterrole view --serviceaccount userns:user -n project

    2. Verify that the service account can view but not manage application resources from the second namespace.

      $ oc create configmap test2 --from-literal two=2 -n project
configmap/test2 created
$ oc --as system:serviceaccount:userns:user get configmap -n project
NAME                       DATA   AGE
kube-root-ca.crt           1      89s
openshift-service-ca.crt   1      89s
test2                      1      35s
$ oc --as system:serviceaccount:userns:user create configmap test3 --from-literal three=3 -n project
error: failed to create configmap: configmaps is forbidden: User "system:serviceaccount:userns:user" cannot create resource "configmaps" in API group "" in the namespace "project"

  4. Create a kubeconfig file with a token from the service account.

    1. Create a secret for the service account.

      Recent Kubernetes and OpenShift releases do NOT automatically create secret tokens for service accounts anymore, and the oc create command does not support creating that kind of token. You must create the secret from a YAML file.

      You can download an example for such a secret from the course samples repository in GitHub.

      $ cat <<EOF >user-token.yaml
apiVersion: v1
kind: Secret
metadata:
  name: user-token
  annotations:
    kubernetes.io/service-account.name: "user"
type: kubernetes.io/service-account-token
EOF
$ oc apply -f user-token.yaml -n userns
secret/user-token created

    2. Extract the token from the secret to a temporary file.

      $ mkdir temp-token
$ oc extract secret/user-token --keys token --to temp-token -n userns
token

    3. Copy the autogenerated kubeconfig file for remote access to MicroShift. You will use it as a basis for the kubeconfig for your unprivileged service account.

      $ unset KUBECONFIG
$ sudo cp /var/lib/microshift/resources/kubeadmin/serverb/kubeconfig ~/remote-admin
$ sudo chown student:student ~/remote-admin
$ chmod a-w ~/remote-admin
$ oc --kubeconfig ~/remote-admin whoami
system:admin
$ oc --kubeconfig ~/remote-admin get node
NAME      STATUS   ROLES                         AGE   VERSION
serverb   Ready    control-plane,master,worker   23h   v1.30.5

    4. Create a copy of the kubeconfig file for remote access and change its user credentials and namespace.

      $ cp remote-admin remote-user
$ chmod u+w remote-user
$ oc --kubeconfig ~/remote-user config delete-user user
deleted user user from /home/student/remote-user
$ oc --kubeconfig ~/remote-user config set-credentials user --token $(cat temp-token/token)
User "user" set.
$ oc --kubeconfig ~/remote-user config set-context microshift --namespace userns --user user --cluster microshift
Context "microshift" modified.

    5. Check that the new kubeconfig authenticates as the service account and has access to the first namespace created during this lab.

      $ oc --kubeconfig ~/remote-user whoami
system:serviceaccount:userns:user
$ oc --kubeconfig ~/remote-user get configmap
NAME                       DATA   AGE
kube-root-ca.crt           1      107m
openshift-service-ca.crt   1      107m
test1                      1      92m

  5. Now configure access to a cluster administrator user. Instead of sharing the autogenerated kubeconfi fileg, create another kubeconfig file, as you did for a developer user, and grant the service account rights to impersonate a user with cluster administration rights.

    1. Create a cluster role binding that grants the cluster-admin role to a nonexistent user.

      $ oc --kubeconfig ~/local-admin create clusterrolebinding cluster-admin-user --clusterrole cluster-admin --user admin
clusterrolebinding.rbac.authorization.k8s.io/cluster-admin-user created
      Names of users and groups in Kubernetes RBAC resources are just string values. Kubernetes has no resources to represent users and groups, and uses whatever values are encoded as part of client certificates or bearer tokens.

    2. Create a cluster role that grants the impersonate verb for the nonexistent user.

      $ oc --kubeconfig ~/local-admin create clusterrole sudo-admin --resource users --resource-name admin --verb impersonate
clusterrole.rbac.authorization.k8s.io/sudo-admin created

    3. Create a cluster role binding that grants the service account access to the new cluster role.

      $ oc --kubeconfig ~/local-admin create clusterrolebinding sudo-user --clusterrole sudo-admin --serviceaccount userns:user
clusterrolebinding.rbac.authorization.k8s.io/sudo-user created

    4. Check that the service account can impersonate the nonexistent user to manage resources that would require cluster administration rights.

      $ oc --kubeconfig ~/remote-user whoami
system:serviceaccount:userns:user
$ oc --kubeconfig ~/remote-user get nodes
Error from server (Forbidden): nodes is forbidden: User "system:serviceaccount:userns:user" cannot list resource "nodes" in API group "" at the cluster scope
$ oc --kubeconfig ~/remote-user --as admin whoami
admin
$ oc --kubeconfig ~/remote-user --as admin get nodes
NAME      STATUS   ROLES                         AGE   VERSION
serverb   Ready    control-plane,master,worker   25h   v1.30.5

  6. Now that you have your kubeconfig files ready and tested, prepare your test machine to accept remote requests to the Kubernetes API server of MicroShift.

    1. Notice that the autogenerated kubeconfig files are configured with different API URLs.

      $ oc --kubeconfig ~/remote-admin whoami --show-server
https://serverb:6443
$ oc --kubeconfig ~/local-admin whoami --show-server
https://localhost:6443

    2. Allow remote access to MicroShift on the system firewall.

      $ sudo firewall-cmd --permanent --zone=public --add-port=6443/tcp
success
$ sudo firewall-cmd --reload
success

  7. Switch to your development machine to test remote access to the MicroShift instance.

    1. Install the OpenShift CLI.

      $ sudo dnf install openshift-clients

    2. Copy the kubeconfig files for the cluster administrator and unprivileged user.

      $ scp -q serverb:~/remote-admin .
$ chmod a-w ~/remote-admin
$ scp -q serverb:~/remote-user .
      Red Hat recommends that you save the kubeconfig file for system:admin in protected storage and use it only for emergencies. For day-to-day work, use a kubeconfig for a service account and impersonation.

    3. Check that you can access MicroShift remotely by using the kubeconfig for a cluster administrator.

      $ oc --kubeconfig ~/remote-admin get node
NAME      STATUS   ROLES                         AGE   VERSION
serverb   Ready    control-plane,master,worker   24h   v1.30.5

    4. Check that you can access MicroShift remotely by using the kubeconfig for an unprivileged user.

      $ oc --kubeconfig ~/remote-user get configmap
NAME                       DATA   AGE
kube-root-ca.crt           1      115m
openshift-service-ca.crt   1      115m
test1                      1      100m
$ oc --kubeconfig ~/remote-user get configmap -n project
NAME                       DATA   AGE
kube-root-ca.crt           1      99m
openshift-service-ca.crt   1      99m
test2                      1      98m

    5. Finally, check that Kubernetes impersonation also works for remote access.

      $ oc --kubeconfig ~/remote-user --as admin get node
NAME      STATUS   ROLES                         AGE   VERSION
serverb   Ready    control-plane,master,worker   25h   v1.30.5

You now have a kubeconfig file for an unprivileged user, using a service account token, which you can use to deploy applications to pre-provisioned namespaces on MicroShift, and have also granted impersonation rights to the service account so it can perform Kubernetes cluster administration tasks on a MicroShift instance.

Next Steps

The next activity uses the kubeconfig file created here, for an unprivileged user, to deploy test applications on MicroShift.