Can I change the limits of compute resources of unscheduled pods, with a custom scheduler in kubernetes - kubernetes-go-client

Question

I am new to kubernetes and I am working on compute resource management of a kubernetes cluster. For this reason, I downloaded a toy scheduler (https://github.com/kelseyhightower/scheduler) in go. I know that once you set compute resource requests to pods you cannot change them. However, suppose that I have not set the resource requirements of the pod in the yaml file: e.g. nginx.yaml:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 2
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
schedulerName: hightower
containers:
- name: nginx
image: nginx
ports:
- containerPort: 8080
protocol: TCP
can I apply resource requests for each pod that the custom scheduler tries to schedule?

Answer 1

According to official Kubernetes Documentation, in order to manage compute resources in the cluster, you have to specify resource types like the maximum amount of CPU and Memory accordingly in the Pod or Deployment (if you consider making ReplicaSets) manifest file. Therefore, when you create Pod, Scheduler selects an appropriate Node for Pod to run on it, assuming that requested resources will not consume most of the Node capabilities.
Custom schedulers can extend the functionality and flexibility for the native Kubernetes scheduler; however, they can't change approach how to provision and manage resource requests or resource limits in Kubernetes cluster.

Answer 2

Thank you for the response, I agree that an implementation of a custom scheduler basically concerns how you bind the pods to nodes, however I thought that I could also enforce resources limits to pods (if there exist none), before the binding and when pods are waiting to be scheduled. It turns out that the only thing I could do is use deployments instead of pods and change the pod template of the deployment to apply also compute resource limits. This way is not as "clean" as I would like because the pending pods are destroyed and kubernetes spawns new ones that include the resource limits. If anybody is interested that is how I did it
imports ...
...
kubeconfig := clientcmd.NewNonInteractiveDeferredLoadingClientConfig(
clientcmd.NewDefaultClientConfigLoadingRules(),
&clientcmd.ConfigOverrides{},
)
namespace, _, err := kubeconfig.Namespace()
if err != nil {
panic(err.Error())
}
restconfig, err := kubeconfig.ClientConfig()
if err != nil {
panic(err)
}
clientset, err := kubernetes.NewForConfig(restconfig)
if err != nil {
panic(err)
}
...
deploymentsClient := clientset.AppsV1().Deployments(namespace)
retryErr := retry.RetryOnConflict(retry.DefaultRetry, func() error {
d, getErr := deploymentsClient.Get("deployment name",
metav1.GetOptions{})
if getErr != nil {
panic(getErr)
}
d.Spec.Template.Spec.Containers[0].Resources.Requests =
make(map[v1core.ResourceName]resource.Quantity)
d.Spec.Template.Spec.Containers[0].Resources.Requests[v1core.ResourceCPU] =
*resource.NewQuantity("# of cores", resource.BinarySI)
_, updateErr := deploymentsClient.Update(d)
return updateErr
})
...