ccu, Administrators
684
edits
Changes
Jump to navigation
Jump to search
m
→Acquiring GPUs with more than 20 GB
By default, Kubernetes schedules GPU pods only on the smallest class of GPU with 20 GB of memory. The way how this is achieved is that nodes with higher grade GPUs are assigned a "node taint", which makes the node only available to pods which specify that they are "tolerant" against the taint.
So if your tasks for example requires a GPU with at least *exactly* 32 GB, you have to
# make the pod tolerate the taint "gpumem-32" (see table below).
# make the pod require the node label "gpumem-32".
# make the pod tolerate the taint "gpumem-32" *and* "gpumem-40".
# make the pod require the node label "gpumem-32" *or* "gpumem-40".
Example:
If you need a GPU with *at least* 32 GB, but also would be happy with 40, you have to
# make the pod tolerate the taint "gpumem-32" *and* "gpumem-40".
# make the pod require the node label "gpumem-32" *or* "gpumem-40".
Example:
== List of compute nodes ==
