Changes

Jump to navigation Jump to search

Initializing the Kubernetes cluster

1,281 bytes added, 6 years ago
m
Kubernetes and pre-requisites (every node)
== Kubernetes and pre-requisites (every node) ==
Install Kubernetes on Ubuntu 18.04. Assuming version 1.14.2 3 is pulled, check how to fix version. On new systems, copy over the install script from the master node.
<syntaxhighlight lang="bash">
"storage-driver": "overlay2"
}
</syntaxhighlight>
 
On nodes with an nVidia GPU, add the following:
 
<syntaxhighlight lang="bash">
"default-runtime": "nvidia",
"default-shm-size": "1g",
"runtimes": {
"nvidia": {
"path": "nvidia-container-runtime",
"runtimeArgs": []
}
}
</syntaxhighlight>
</syntaxhighlight>
Check /etc/fstab if swap is still configured there, delete if this is the case.
 
== Create cluster configuration scripts ==
 
OBSOLETE, DOES NOT SEEM TO WORK IN NEW KUBERNETES.
 
<syntaxhighlight lang="bash">
> cd init/templates
# edit cluster information in the following config file
> nano make_init_config.sh
> touch /home/kubernetes/.rnd
> ./make_init_config.sh
</syntaxhighlight>
 
This will generate the init config from the config template and store it in /home/kubernetes/clusters/ccu.
== Spin up the master node ==
* == Update kubelet configuration for master node==
Edit /etc/kubernetes/manifests/kube-controller-manager.yaml:
</syntaxhighlight>
Copy certs/ca.crt (certificate for ccu.uni-konstanz.de) to /usr/share/ca-certificates/ca-dex.pem. Edit /etc/kubernetes/manifests/kube-controller-managerapiserver.yaml:
<syntaxhighlight lang="bash">
spec:
containers:
- command:
# add these five
- --oidc-issuer-url=https://ccu.uni-konstanz.de:32000/dex
- --oidc-client-id=loginapp
- --oidc-ca-file=/usr/share/ca-certificates/ca-dex.pem
- --oidc-username-claim=name
- --oidc-groups-claim=groups
</syntaxhighlight>
== Daemonsets on Master node ==
 
=== Flannel daemonset (pod network for communication) ===
<syntaxhighlight lang="bash">
> cd init
> ./start_pod_network.sh
</syntaxhighlight>
* Flannel daemonset (pod network for communication)
* === nVidia daemonset=== <syntaxhighlight lang="bash">> cd init> ./deploy_nvidia_device_plugin.sh</syntaxhighlight> The daemonset should be active on any node with an nVidia GPU.
== Authentication systems ==
The master node should now login to the docker registry of the cluster. <syntaxhighlight lang="bash">> docker login https://ccu.uni-konstanz.de:5000Username: bastian.goldlueckePassword:</syntaxhighlight> Also, we need to provide the read-only secret for the docker registry in every namespace. TODO: howto.  Finally, we need to set up all the rules for rbac. <syntaxhighlight lang="bash">> cd rbac# generate namespaces for user groups> ./generate_namespaces.sh# label all compute nodes for which namespace they serve# (after they are up, needs to be redone when new nodes are added)> ./label_nodes.sh# set up access rights for namespaces> kubectl apply -f rbac.yaml# set up rights for which namespaces can access which compute node> kubectl apply -f node_to_groups.yaml</syntaxhighlight> = DEX with LDAP = Persistent volumes == === Local persistent volumes ===
TODOCheck directory local_storage:* clone the git repository for the provisioner using clone_provisioner.sh (delete first if already here).* install helm: outdatedinstall_helm.sh, switched to containerized DEXget_helm.sh. Check what still needs to be doneDo NOT run helm init (unsafe and soon obsolete).* set up and run provisioner:
Set up according to [https://github<syntaxhighlight lang="bash">> cd install> generate_config.sh> kubectl apply -f install_storageclass.yaml> kubectl apply -f install_service.com/krishnapmv/k8syaml> kubectl apply -ldap this tutorial]f provisioner_generated.yamlwith customized install scripts in kubernetes/init/dex</syntaxhighlight>
# Create secrets for TLS connectionsAfter local persistent volumes on the nodes have been generated in /mnt/kubernetes, use certs for ccu.uni-konstanz.de## Modify ca-cm.yml to contain correct ca.## Run upload_ccu_tls.sh# Spin they should show up login application service.## Modify loginapp-cm.yml: server configunder## Modify loginapp-ing-srv.yml: service data, mapping of ports to outside world## Modify loginapp-deploy.yml: ID secret for TLS## Run start-login-service.sh# Spin up dex## Modify dex-cm.yml: server data and LDAP configuration## Modify dex-ing-srv.yml: service data, mapping of ports to outside world<syntaxhighlight lang="bash">## Modify dex-deploy.yml: ID secret for TLS> kubectl get pv## Run start-dex-service.sh</syntaxhighlight>

Navigation menu