Kind installation for Kubernetes

For installation of base tools and other setup details, see:

• https://nostra.github.io/neon/#/1/1
• https://nostra.github.io/gitops-fluxcd2/

Using Colima on Mac

We want to increase the number of file handles:

colima ssh

Edit /etc/sysctl.conf and add:

sudo -i 
cat <<EOF >> /etc/sysctl.conf
fs.inotify.max_user_watches = 1048576
fs.inotify.max_user_instances = 512
EOF

Then systctl -p

Base setup / setup of your local kubernetes

Create cluster:

kind create cluster --config kind-api-cluster.yaml --name=k8sdebug

Test cluster: (##1)

kubectl cluster-info --context kind-k8sdebug

kubectl wait pod \
--all \
--for=condition=Ready -A

Create a key that you will register as a deploy key in github:

ssh-keygen -f ~/.ssh/k8sdebug -N "" -C "Key for k8sdebug setup. This key was generated by $USER"

Copy the key and create it as a deploy key with write rights in https://github.com/nostra/k8sdebug/settings/keys

cat $HOME/.ssh/k8sdebug.pub | pbcopy

Bootstrapping Flux

Bootstrapping flux:

cd ..
kubectl create -k flux/system
kubectl wait pod \
--all \
--for=condition=Ready -n flux-system --timeout 300s

Create the local cluster:

cd ..
kubectl create -k flux/kind/

Create secret in the flux system

The secret gets created so you can pull (and later write) to the git repository:

flux create secret git flux-cluster \
--url=ssh://git@github.com/nostra/k8sdebug \
--private-key-file=$HOME/.ssh/k8sdebug --namespace=flux-system \
--export | kubectl create -f -

Create a pull token to fetch images from github packages

First just check that you have the namespace "apps":

kubectl get ns apps

If you don't have it, check running pods, and status of flux. (See debug section at the bottom of this file for details.)

Ref: authenticating-with-a-personal-access-token-classic

• open: https://github.com/settings/tokens
• Choose "Generate new token" and classic token
• Fill in info
• Choose write:packages and delete:packages (for good measure)
• Click "Generate token"
• write something like

export CR_PAT="..."
export USERNAME="your github handle"
echo $CR_PAT | docker login ghcr.io -u $USERNAME --password-stdin
docker pull ghcr.io/k8sdebug/k8sdebug-app:0.0.12

If you with this manage to pull an image, you successfully have a token in CR_PAT variable.

export CR_PAT="..."
export USERNAME="your github handle"
kubectl -n apps \
  create secret docker-registry gh-packages-auth --docker-server=ghcr.io --docker-username=$USERNAME \
  --docker-password="$CR_PAT" --dry-run=client -o yaml > k8sdebug-pull-secret.yaml
kubectl create -f k8sdebug-pull-secret.yaml
rm k8sdebug-pull-secret.yaml

(Take a moment ot look at the yaml file before applying and deleting it, in order to verify contents.)

Optional: Enable mcalert

In order to get a nifty display of cluster status on the menubar, install https://github.com/nostra/mcalert The github entry page will explain how you compile and install it.

Create the file .mcalert.properties in your $HOME and put the following into it:

mcalert.prometheus.endpoints.local-prometheus.ignore-alerts=CPUThrottlingHigh,KubeControllerManagerDown,KubeControllerManagerDown,KubeSchedulerDown,NodeClockNotSynchronising
mcalert.prometheus.endpoints.local-prometheus.watchdog-alerts=disabled
mcalert.prometheus.endpoints.local-prometheus.uri=http://prometheus.local.gd:31090/api/v1/alerts

mcalert.prometheus.endpoints.local-prometheus-auth.ignore-alerts=CPUThrottlingHigh,KubeControllerManagerDown,KubeControllerManagerDown,KubeSchedulerDown,NodeClockNotSynchronising
mcalert.prometheus.endpoints.local-prometheus-auth.watchdog-alerts=disabled
mcalert.prometheus.endpoints.local-prometheus-auth.uri=http://prometheus-auth.local.gd:31090/api/v1/alerts
mcalert.prometheus.endpoints.local-prometheus-auth.header[0].name=Authorization
mcalert.prometheus.endpoints.local-prometheus-auth.header[0].content=Basic bmVvbjpzZXNhbQ==

Commands / cheat sheet

Command	Description
kubectl get pods -A	Get pods in all namespaces. All should be running ok
flux get kustomization -A	Get Kustomizations for all namespaces. They should all not be suspended,
flux get sources git	Find status of sources, i.e. the remote git repostory
flux reconcile source git k8sdebug	Force reconcilliation of git
flux suspend kustomization -n flux-system flux-cluster	Stop reconcillation. Start it again with "resume"
kubectl config set-context --current --namespace=apps	Set active namespace to apps
kubectl edit deployment k8sdebug	Adjust a running deployment. Flux need to be suspended
kubectl rollout undo deployment k8sdebug	Revert to previous setting after an edit

If trouble with nginx-proxy and prometheus

kubectl port-forward -n monitoring svc/prometheus-k8s 9090:9090

Then: http://localhost:9090/alerts