Tools of the trade

There are many tools available to speed up debugging, issue resolution and problem identification when analyzing issues with an OpenShift cluster.

Tools we will use

During this lab, we will run through a number of exercises that show the power and effective use of the following 7 tools.

omc

It’s just like oc, but on a must-gather!

You can find and download omc from github at: omc releases

For more in-depth details on how to collect a must-gather and for alternative must-gather commands for other OpenShift components, you can refer to the Gathering data about your cluster section of our doc.

kubectl-dev_tool

Inspect your audit logs with ease!

You can install the kubectl-dev_tool from github using go install against the: cluster debug tools repository.

Example: go install github.com/openshift/cluster-debug-tools/cmd/kubectl-dev_tool@latest

In order to use the kubectl-dev_tool you need to collect the audit logs from the problematic cluster. You can find more information on using the gather_audit_logs sub-command with must-gather in the gather_audit_logs section of our docs.

koff

Read objects from an etcd snapshot.

You can find and download koff from github at: koff releases

In order to use the koff tool, you must collect an etcd snapshot from the target environment. Most customers are already taking daily or more frequent etcd snapshots` for disaster recovery purposes, so if you are looking to analyze a problem from a specific time period, request a specific etcd snapshot.

If you need to collect a fresh etcd snapshot, you can follow the Backing up etcd data section of our docs.

The archive produced by the cluster-backup.sh script will include a snapshot_<date>.db file you can use with koff. You can zip and upload the archive to the customer portal for further analysis.

Snapshot saved at /home/core/assets/backup/snapshot_2021-06-25_190035.db

yq and jq

Process and parse YAML and JSON

gron

Making JSON greppable

You can find and download gron from github at: gron releases

etcd-ocp-diag

Find etcd issue in your must-gather data

You can find and download etcd-ocp-diag from github at: etcd-ocp-diag-script

ocp_insights

Parse and view your Insights data from the Insights Operator

You can find and download ocp_insights from github at: ocp_insights