How to create an IAM role to run workloads on Amazon EMR on EKS?
The official documentation remains very vague on this point in particular
This is multistep process. We have EKS cluster on which we run EMR jobs. So we need to setup IAM + EKS role mapping configurations. AWS Documentation is detailed which creates confusion sometimes.
Following are high level steps which might help you to refer AWS documentation again and understand.
Note: These steps assumes you already have created EKS.
Create Namespace to run your spark jobs.
Create RBAC Role and Role binding for your cluster configuration. (This is EKS level role based access mechanism.)
Edit aws-auth to update roleARN for AWSServiceRoleForAmazonEMRContainers. (EKS Authorisation)
Create EMR virtual cluster and assigned it to EKS namespace created above.
Create trust policy to access EKS cluster for EMR containers.
Create Job execution role and associate above policy to it.
Submit your emr on eks job.
Related
I'm using amazon EKS fargate. I can see container logs using fluentbit side car etc no problem at all. But those logs ONLY show what is happening inside the container AFTER it has started up
I enabled aws eks cluster logging fully
Now I would like to see logs in cloudwatch which is equivalent of
kubectl describe pod
command
I have searched the ENTIRE cloudwatch clustername log group and am not able to find logs like
"pulling image into container"
"efs not mounted"
etc
I want to see logs in cloudwatch prior to the actual container creation stage
IS it possible at all using eks fargate ?
Thanks a bunch
You can use Container Insights which can collect metrics by using performance log events using the embedded metric format. The logs are stored in CloudWatch Logs. CloudWatch generates several metrics automatically from the logs which you can view in the CloudWatch console.
In Amazon EKS and Kubernetes, Container Insights uses a containerized version of the CloudWatch agent to discover all of the running containers in a cluster. It then collects performance data at every layer of the performance stack.
https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-view-metrics.html
I read that we should use assume-role to create AWS EKS cluster. From the documention, I only find the use of EKS service role but I don't see how to create a cluster with a role. Do I miss anything?
I was planning to use Jenkins(CI) ----> Spinnaker(CD) integration for AWS EKS.
Does Spinnaker support multi-cluster deployments?
For example:
I will have 4 clusters in different accounts
and I want to have 1 Spinnaker deployed to one of the clusters and manage other 3 as well.
is it possible to do?
Yes is possible. The suggested way is to have Spinnaker Running in an AWS Account called Spinnaker or CD and in a specific namespace called spin.
A great guide to follow for Spinnaker in EKS is Continuous Delivery using Spinnaker on Amazon EKS
I am trying to run a pod on fargate from EKS that needs to access s3 via boto3 python client and I cant figure out why this is happening. It works just fine when scheduled on a eks ec2 node.
botocore.exceptions.NoCredentialsError: Unable to locate credentials
I have a properly setup fargate profile and followed this guide.
Does anyone know why aws credentials are not within the context of this pod? Does this have anything to do with the pod execution role?
I have a properly setup fargate profile and followed this guide.
That's a great start and it will ensure your pods are scheduled on Fargate rather than EC2.
Does anyone know why aws credentials are not within the context of this pod? Does this have anything to do with the pod execution role?
Without knowing what exactly you defined it's impossible to troubleshoot but yes, it's worth checking the pod execution role for starters.
However, given that you want to access an S3 bucket from your pod you need to make sure the pod's service account uses the respective policy. Last year we introduced IRSA, allowing you to assign least privileges on the pod level and given you're on Fargate this is the way to go. So, please peruse and apply IRSA as per doc and report back if anything is not working as expect.
Deployed Spinnaker in AWS to run a test in the same account. However unable to configure server groups. If I click create the task is queued with the account configured via hal on the CLI. Anyway to troubleshoot this, the logs are looking light.
Storage backend needs to be configured correctly.
https://www.spinnaker.io/setup/install/storage/