Is there any way I can see FluentBit logs for EKS Fargate? I'd like to see the errors that are raised by the plugins.
The EKS Fargate logging manual provides a way to see if the ConfigMap is valid. The ConfigMap entry I'm using is valid, but there seem to be some issues in the plugin because the logs aren't created in Cloudwatch and I don't know why.
Turns out AWS provides a way - we need to put the flag flb_log_cw: "true" under data in the ConfigMap (ref), and that would output the FluentBit logs to Cloudwatch logs.
Related
I would like to create an ECS task with Fargate, and have that upload a file to S3 using the AWS CLI (among other things). I know that it's possible to create task roles, which can provide the task with permissions on AWS services/resources. Similarly, in OpsWorks, the AWS SDK is able to query instance metadata to obtain temporary credentials for its instance profile. I also found these docs suggesting that something similar is possible with the AWS CLI on EC2 instances.
Is there an equivalent for Fargate—i.e., can the AWS CLI, running in a Fargate container, query the metadata service for temporary credentials? If not, what's a good way to authenticate so that I can upload a file to S3? Should I just create a user for this task and pass in AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY as environment variables?
(I know it's possible to have an ECS task backed by EC2, but this task is short-lived and run maybe monthly; it seemed a good fit for Fargate.)
"I know that it's possible to create task roles, which can provide the
task with permissions on AWS services/resources."
"Is there an equivalent for Fargate"
You already know the answer. The ECS task role isn't specific to EC2 deployments, it works with Fargate deployments as well.
You can get the task metadata, including IAM access keys, through the ECS metadata service. But you don't need to worry about that, because the AWS CLI, and any AWS SDK, will automatically pull that information when it is running inside an ECS task.
I am trying to create eks anywhere local cluster on centos machine. However, the cluster creation is stuck and I don't see any more output on the screen. I have enabled debug to more output. Please see screenshot of the logs advise if I am missing anything. I have been following below link to create eks anywhere local cluster.
Link: https://aws.amazon.com/blogs/aws/amazon-eks-anywhere-now-generally-available-to-create-and-manage-kubernetes-clusters-on-premises/
Here is the screenshot of the logs:
logs screenshot
Here is the screenshot of the cluster create yaml file
cluster create yaml file
We've a PHP application that is pushed to ECR Fargate and we've configured an ECS task definition for it, and it works fine as a container in ECS.
I've configured aws-logs for the application and it sends the app logs normally to cloudwatch, but I'm wondering how to send the logs in a file inside the container in
"/var/www/html/app/var/dev.log"
to the same log group that I configured when created the task definition.
I found the answer on the following link:
https://aws.amazon.com/blogs/devops/send-ecs-container-logs-to-cloudwatch-logs-for-centralized-monitoring/
Just needed to install both syslog and awslogs on the php image, then use supervisord to start them with the container along with our php app. From Task definition side, create a volume and a mount point.
I have locked myself out after modifying the config map. Is there any way around this?
This happened after i modified the config map using
kubectl edit -n kube-system configmap/aws-auth
Now i am getting an error using the IAM role that was used to create the cluster
Error from server (Forbidden): pods is forbidden: User "USERNAME" cannot list resource "pods" in API group "" in the namespace "default"
By default, EKS Cluster creator (IAM role/user) get full accessto EKS Cluster (irrespective of aws-auth configMap)
Run aws sts get-caller-identity and validate if Arn from response is the IAM role/user that created the EKS Cluster.
If you are locked out with no access for Cluster Creator, reach out to AWS Premium Support using the same account as EKS Cluster. They can help fix it (hope).
Worst case, have to create a new Cluster.
I have done the below steps.
Created an EKS Cluster
Installed aws-iam-authenticator client binary
Execute "aws eks update-kubeconfig --name <cluster_name>"
Execute "kubectl get svc"
I am able to view the services available in my cluster. When I see ~/.kube/config file it is using an external command called "aws-iam-authenticator".
My understanding is that "aws-iam-authenticator" uses my ~/.aws/credentials and retrieves the token from AWS(aws-iam-authenticator token -i cluster-1) and uses that token for "kubectl get svc" command. Is my understanding correct?
If my understanding correct, where does heptio comes into picture in this flow? Does Heptio Authenticator be deployed automatically when creating the EKS Cluster?
Basically, Heptio authenticator = aws-iam-authenticator.
You can check the details on here. If your aws-iam-authenticator is working fine, then you don't need to care about heptio additionally. They just renamed it.