When we are running a serverless app, say a beanstalk, a danymoDB and
a SNS, there will be three XRay daemons in each JVM, right?
If so,how could XRay trace a request from the beginning to the end?
The trace ID will go with http request header or sth like that?
According to the docs, "The AWS X-Ray daemon is a software application that listens for traffic on UDP port 2000, gathers raw segment data, and relays it to the AWS X-Ray API. The daemon works in conjunction with the AWS X-Ray SDKs and must be running so that data sent by the SDKs can reach the X-Ray service."
It's important to note that the X-Ray SDK produces what's called a remote subsegment to mimic the result of the downstream in a client-sided manner. For a service like DynamoDB, you would see this. For something like SNS, the trace header information is propagated across through the Http headers. The X-Ray Daemon is used to forward segments generated when a service receives an upstream request; DynamoDB doesn't do this yet, and SNS forwards it through the trace header mentioned.
The Daemon isn't part of the JVM; it's an external process that's ran in an instance that forwards the trace data to the service. Technically, it can be run by a single instance, the same instance, or all instances.
Related
I looking for a solution to send the logs from AWS-Xray to AWS CloudWatch to help me doing an aggregation and metrics.
I was checking if we can do this directly using AWS X-Ray daemon, it seems there is no way to do this form the X-Ray daemon.
I can see that the only solution to do so using Get the trace summary from Xray using AWS XRAY SDK API and share to other streams like CloudWatch.
Is there a solution to conduct this using a config in AWS X-Ray daemon to send the logs directly to CloudWatch log group?
Unfortunately xray daemon only support X-Ray endpoint by PutTraceSegments API, it cannot emit metrics or logs to CloudWatch.
Alternatively, you can choose ADOT collector which is a all-in-one agent.
https://aws-otel.github.io/docs/getting-started/collector
I'm following this tutorial: https://github.com/yurishkuro/opentracing-tutorial/tree/master/java/src/main/java/lesson03. What need to be set so that services running in different hosts can send the data to the same backend?
You have two options:
Run Jaeger agent (or OpenTelemetry Collector) on each host that runs your applications and let the agent forward trace data to a central Jaeger collector. The Jaeger client can continue emitting data via UDP port in this case.
Configure the Jaeger client with an HTTP endpoint of the Jaeger collector.
For (2), you can pass the environment variable to you applications:
JAEGER_ENDPOINT=http://jaeger-collector:14268/api/traces
Additional references:
https://www.jaegertracing.io/docs/latest/client-features/
https://github.com/jaegertracing/jaeger-client-java/tree/master/jaeger-core
My understanding on aws xray is, xray is similar to dynatrace and I am trying to use xray for monitoring apache performance. I do not see any document related to xray with apache except below.
https://mvnrepository.com/artifact/com.amazonaws/aws-xray-recorder-sdk-apache-http
Can anyone please suggest if it is possible to use aws xray with apache and if yes can you also point some document related to it. Thanks.
I assume that by "apache" you mean the Apache Tomcat servlet container, since you are referring to a maven artifact which is a Java build tool.
Disclamer: I don't know what "dynatrace" is and I don't know which logging you specifically want.
But as far as the Apache Tomcat servlet container and X-Ray goes - here is the link to get started:
http://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-java.html
Start by adding AWSXRayServletFilter as a servlet filter to trace incoming requests. A servlet filter creates a segment While the segment is open you can use the SDK client's methods to add information to the segment and create subsegments to trace downstream calls. The SDK also automatically records exceptions that your application throws while the segment is open.
As for the mentioned maven artifact:
aws-xray-recorder-sdk-apache-http – Instruments outbound HTTP calls made with Apache HTTP clients
So, you'll need this if, let's say, a client makes a request to your Tomcat server and your Tomcat server makes a request to another server thus acting as a client in this case.
On openstack cloud plantform ，If I want to close a instance on compute node, what does the openstack do? can you tell me the process?
I assume by close you mean "terminate".
When terminating an instance the running virtual machine with an instance id of X is shut down and removed from the physical host it exists on.
The nova client query for this would be:
nova delete <instance-id> or something to that effect.
When you make that query the python-novaclient is interfacing with its own internal API to reach out to the nova-api RESTful API. It authenticates itself with an auth token in the http header of its query. Then nova-api interprets the instance termination request. It will verify any ACLs it needs to against keystone. And then it will perform necessary methods to shut down and remove the instance freeing up resources for future instances. It will then return a result.
Going deeper the scheduler will send out requests over the messaging system as a result of the nova-api queries. Those messages will be received by the targeted physical hosts. There nova-compute will interpret the request to delete the instance and it will perform its own local necessary tasks. Usually this involves interfacing with libvirt to shut down and free the instance resource. After this is completed or failed it will respond to the messaging bus the status. And the API will eventually get that message back and send it on to the user who initially requested the action.
Explanation:
I have one executable Jar deployed on one EC2 instance which can be run manually to listen on port 80 for proxy traffic
I have one Spring application on another EC2 instance which hits a website on third party server
Connection between these two machines:
Spring application setup i.e. B tells third party server to open a website and use A as a proxy, this leads to generation of logs of network calls on A.
What I want to do is: for every request I send from B to third party server I want network logs that are being generated on A to be transferred to B
What I tried:
One way is to rotate logs on A and write to S3 and then application and pick it from S3 and process them
ssh into A and grep the log file, but this stops the JAR to listen to the new traffic and it gets stuck
What I am looking for:
A realtime solution, as soon as logs show up on A I want them to be ported to B without stopping A on its listening job
I am not sure what OS you are running, but if you are running a nix variant, you can install syslog-ng instead of syslog, or rsyslog, which is capable of logging local and external events. In this case I would set up a central logging server, that listens for logs from server a, and server b.
Another alternative is syslog-ng is not what you're looking for, you could install splunk on a server, and have it pick up the logs from splunk reporters on each server you want to centrally log.
Hope this helps.
As Kevin mentioned , you could set up a Splunk Indexer on an EC2 instance and use this to aggregate the collection of logs from A and B and any other sources , and then use the Splunk Search language to search over this log data in "near realtime", correlate events together across your various systems , create custom dashboards, setup proactive alerting etc...
http://www.splunk.com/
As far as the mechanisms for getting this data from your systems to the Splunk Indexer :
1) Use a Splunk Universal Forwarder to monitor log output and forward it to your Splunk Indexer , http://www.splunk.com/download/universalforwarder
2) As your systems are Java based , SplunkJavaLogging has log4j/logback/jdk appenders that you can seamlessly wire in to your logging config to forward log events to your Splunk Indexer : https://github.com/damiendallimore/SplunkJavaLogging
3) Use the Splunk Java SDK , http://dev.splunk.com/view/java-sdk/SP-CAAAECN , to input log events into your Splunk Indexer via HTTP REST or Raw TCP