How to get data (read) from AWS managed ElasticSearch using java - amazon-elasticsearch

In AWS IOT, I created a thing (device), and published messages to a topic, say trucks/truck1234. I created an Elasticsearch domain and stored those messages to analyse further using Kibana. Up to now, all fine.
Now i want to read the published message data from my application with Java. I’ve followed the steps in this link:
https://aws.amazon.com/blogs/mobile/analyze-device-generated-data-with-aws-iot-and-amazon-elasticsearch-service/
Can anyone please suggest how to achieve it?

Related

Migrating from Astronomer airflow setup to MWAA

Currently we are using Apache airflow platform provided by Astronomer with Kubernetes executer and we are using PostgreSQL as a backend DB. Now we want move to MWAA. I want some help if anyone has done the same.
How to migrate existing DAGs and Metadata to MWAA ?
What are the things we need to take care before migration ?
Is there anyway we can migrate everything from current setup to MWAA ?
Please help me if anyone has any idea about this.
Here's the official migration guide for MWAA from AWS: Migrating to a new Amazon MWAA environment. The documentation covers how to migrate metadata and key considerations.
Please give it a read through. If you have a more specific question, feel free to post a new question.

How do I create SQL connection to my app and Upload it to google cloud

Thanks for getting back at me. Sorry for the late reply, it was bed-time this time. I need to connect the Cloud SQL database that I have created to my application that is in App Engine. I tried to follow the online tutorials but when I do apply such info I would get then gcloud app deploy it return a connection error. Please help. Also clarify here: When I execute the gcloud app deploy command I suppose it takes my local file to Google Cloud where I would see the entire folder and files of my project on the project I was deploying but I am seeing the old version of my project while presentation has changed to the latest version. Also last one how can I link domain nam from http://domain.google.com to my app in http://cloud.google.com . Please help I am dying with stress I have been trying in here
Given that you haven't provided any information as to what settings you are using, or what error has been provided it is impossible to know what kind of problem you are running into.
I suggest taking a look at the "Connecting to App Engine" page here. It should answer a lot of your questions around connecting from an App Engine app.
I see two questions here.
1.
I need to connect the Cloud SQL database that I have created to my
application that is in App Engine. I tried to follow the online
tutorials but when I do apply such info I would get then gcloud app
deploy it return a connection error. Please help. Also clarify here:
When I execute the gcloud app deploy command I suppose it takes my
local file to Google Cloud where I would see the entire folder and
files of my project on the project I was deploying but I am seeing the
old version of my project while presentation has changed to the latest
version.
I see your problem here to be with CloudSQL and GAE connectivity. Depending on whether you use GAE Standard or Flex and CloudSQL MySQL or POSTGRES the steps varies. Documentation is quite clear in here though.
2.
Also last one how can I link domain nam from http://domain.google.com
to my app in http://cloud.google.com . Please help I am dying with
stress I have been trying in here
This is going to be super simple, goto GCP cloud console, Navigate to GAE-->Settings-->Custom Domain and click on add custom domain "Enter the domain name you want to link" When you click continue you will be shown the steps for verifying the domain owneship and to point the DNS to the GAE.
Documented properly by GCP folks at https://cloud.google.com/appengine/docs/standard/python/mapping-custom-domains
If you are using GAE Standard or Flex, a possible result of command gcloud app deploy :
An app.yaml (or appengine-web.xml) file is required to deploy this directory as an App Engine App, check next links:
https://cloud.google.com/appengine/docs/flexible/python/configuring-your-app-with-app-yaml
https://cloud.google.com/appengine/docs/flexible/python/writing-application-logs
Mysql and Postgres connection:
https://cloud.google.com/sql/docs/mysql/connect-app-engine
https://cloud.google.com/sql/docs/postgres/connect-app-engine
Sometimes it easy share the app.yaml for replicate the app correctly.

Connect to AWS S3 without API

I've looked everywhere on the Interweb but couldn't find a satisfying answer...
Does anybody know what "protocol" the AWS S3 speaks?
Our idea is to write a Function for a PLC (no chance to use the provided API) to communicate directly with AWS S3.
For Example PLC to "AWS IoT" works in MQTT/HTTP - how can I skip "AWS IoT"?
I know there is the possibility to put an IoT device inbetween - but we are evaluating our possibilities right now.
Thank you in advance
All of the AWS services have a documented REST API - the S3 one is here. In addition, all of their libraries are open source so you could likely get some ideas from them too.

Interacting with Amazon Elasticsearch Service using JAVA SDK

I am having trouble finding examples of how to interact with an amazon elasticsearch cluster using the amazon sdk. Can someone point me to examples? I've found javadocs for things like AWSCredentials and the AWSElasticsearchClient, just looking for concrete examples of how to glue everything together. I'm looking for how to do things like:
-create an index
-index documents
-delete documents
Just the basic operations. I assume that there is an 'amazon' way of creating a client and interacting with the cluster since they provide an sdk and a credentials object for signing requests.
Did a little more digging and I guess those SDK classes are more for doing 'infrastructure' type operations against your elasticsearch instance, like spinning up a new node, etc, not for creating a client to do search/index type operations. I guess I can just use a Jest Client instead.

Red5 stream audio from Azure Storage or Amazon S3

I'm wondering if its possible to stream audio in Red5 from files stored in Azure? I am aware of how to manipulate the playback path via a custom file name generator IStreamFilenameGenerator, our legacy Red5 webapp uses it. It would seem to me though that this path needs to be on the local red5 server, is this correct?
I studied the example showing how to use Amazon S3 for file persistence and playback (https://goo.gl/7IIP28) and while the file recording + upload makes perfect sense, I'm just not seeing how the playback file name that is returned is streaming from S3. Tracing the StringBuilder appends/inserts, it looks like the filename is going to end up to be something like {BucketLocation}/{SessionID}/{FileKey} ... this lead me to believe that bucket.getLocation() on Line 111 was returning an HTTP/S endpoint URL, and Red5 would somehow be able to use it. I wrote a console app to test what bucket.getLocation() returned, and it only returns null for US servers, and EU for Europe. So, I'm not even sure where/how this accesses S3 for direct playback. Am I missing something?
Again, my goal is to access files stored in Azure, but I figured the above Amazon S3 example would have given me a hint.
I totally understand that you cannot record directly to Azure or S3, the store locally + upload makes sense. What I am failing to see is how to stream directly from a blob cloud storage. If anyone has suggestions, I would greatly appreciate it.
Have you tried using Azure Media Services? I believe looking at their documentation will be a good start for your scenario.