Migrating from Astronomer airflow setup to MWAA - mwaa

Currently we are using Apache airflow platform provided by Astronomer with Kubernetes executer and we are using PostgreSQL as a backend DB. Now we want move to MWAA. I want some help if anyone has done the same.
How to migrate existing DAGs and Metadata to MWAA ?
What are the things we need to take care before migration ?
Is there anyway we can migrate everything from current setup to MWAA ?
Please help me if anyone has any idea about this.

Here's the official migration guide for MWAA from AWS: Migrating to a new Amazon MWAA environment. The documentation covers how to migrate metadata and key considerations.
Please give it a read through. If you have a more specific question, feel free to post a new question.

Related

Azure SQL Elastic Jobs & Data Tier Application (DACPAC)

I'm trying to get a DACPAC containing schema changes deployed to a target group in an elastic jobs setup in Azure. I've done the tutorial and my test jobs are executing properly. Now I want to run a DACPAC against my target group but I'm starting to realize that the documentation is for an older PowerShell module, ElasticDatabaseJobs, specifically the Set-AzureSqlJobContentDefinition command. I don't see any analogous command in the Az.Sql documentation, nor do I see any mention of it on the T-SQL tutorial.
What is the current way to associate a DACPAC file with an Elastic Job? Is this being phased out? Should I use the old PowerShell module? Am I missing something? Help!
Current Azure hosted version of Elastic Database Jobs(https://learn.microsoft.com/en-us/azure/sql-database/sql-database-job-automation-overview#elastic-database-jobs-preview) does not support DACPAC. It is not a tested/supported scenario. Please use Powershell/T-SQL scripts for your jobs.
thanks
-- Srini Acharya

Migrating Paperclip database to Active Storage Amazon S3

We are in the process of migrating paperclip to active storage.But we want to move Paperclip storage from database to Active Storage Amazon S3. Can anyone point me to a documentation for this? I can't seem to find one.
Maybe this article can help you. Code is missing due to Github username update but the main idea is there.
Or simply the documentation of Paperclip. As you wish.

How to use SSL in MariaDB datasource with quarkus?

I would like to connect to MariaDB database via SSL in Quarkus application. However, I cannot find a way how to define the SSL-related information in Quarkus application.
How to provide the certificate which is needed in database connection in Quarkus application?
Is it even possible?
If not, I assume that many would be interested in that feature.
I searched information from https://quarkus.io/guides/datasource but did not find anything regarding this.
MariaDB reference: https://mariadb.com/kb/en/library/using-tls-ssl-with-mariadb-java-connector/
There's no reason for it to not work. Just include what you need in your JDBC url.
Be aware though that if you are using native images, you should read this guide that will guide you through configuring everything properly: https://quarkus.io/guides/native-and-ssl .

How do I create SQL connection to my app and Upload it to google cloud

Thanks for getting back at me. Sorry for the late reply, it was bed-time this time. I need to connect the Cloud SQL database that I have created to my application that is in App Engine. I tried to follow the online tutorials but when I do apply such info I would get then gcloud app deploy it return a connection error. Please help. Also clarify here: When I execute the gcloud app deploy command I suppose it takes my local file to Google Cloud where I would see the entire folder and files of my project on the project I was deploying but I am seeing the old version of my project while presentation has changed to the latest version. Also last one how can I link domain nam from http://domain.google.com to my app in http://cloud.google.com . Please help I am dying with stress I have been trying in here
Given that you haven't provided any information as to what settings you are using, or what error has been provided it is impossible to know what kind of problem you are running into.
I suggest taking a look at the "Connecting to App Engine" page here. It should answer a lot of your questions around connecting from an App Engine app.
I see two questions here.
1.
I need to connect the Cloud SQL database that I have created to my
application that is in App Engine. I tried to follow the online
tutorials but when I do apply such info I would get then gcloud app
deploy it return a connection error. Please help. Also clarify here:
When I execute the gcloud app deploy command I suppose it takes my
local file to Google Cloud where I would see the entire folder and
files of my project on the project I was deploying but I am seeing the
old version of my project while presentation has changed to the latest
version.
I see your problem here to be with CloudSQL and GAE connectivity. Depending on whether you use GAE Standard or Flex and CloudSQL MySQL or POSTGRES the steps varies. Documentation is quite clear in here though.
2.
Also last one how can I link domain nam from http://domain.google.com
to my app in http://cloud.google.com . Please help I am dying with
stress I have been trying in here
This is going to be super simple, goto GCP cloud console, Navigate to GAE-->Settings-->Custom Domain and click on add custom domain "Enter the domain name you want to link" When you click continue you will be shown the steps for verifying the domain owneship and to point the DNS to the GAE.
Documented properly by GCP folks at https://cloud.google.com/appengine/docs/standard/python/mapping-custom-domains
If you are using GAE Standard or Flex, a possible result of command gcloud app deploy :
An app.yaml (or appengine-web.xml) file is required to deploy this directory as an App Engine App, check next links:
https://cloud.google.com/appengine/docs/flexible/python/configuring-your-app-with-app-yaml
https://cloud.google.com/appengine/docs/flexible/python/writing-application-logs
Mysql and Postgres connection:
https://cloud.google.com/sql/docs/mysql/connect-app-engine
https://cloud.google.com/sql/docs/postgres/connect-app-engine
Sometimes it easy share the app.yaml for replicate the app correctly.

How to get data (read) from AWS managed ElasticSearch using java

In AWS IOT, I created a thing (device), and published messages to a topic, say trucks/truck1234. I created an Elasticsearch domain and stored those messages to analyse further using Kibana. Up to now, all fine.
Now i want to read the published message data from my application with Java. I’ve followed the steps in this link:
https://aws.amazon.com/blogs/mobile/analyze-device-generated-data-with-aws-iot-and-amazon-elasticsearch-service/
Can anyone please suggest how to achieve it?