Hello: I do have Django based application deployed in AWS EC2 instance. The application accesses a hospital database hosted on the premises for query and update through as set of APIs written in Django application. However, we cant have production IP, database UserID, database Pwd in our EC2 application due to security reasons. What are the options available for this problem. One option that is thought is set-up a reverse proxy server on a different machine at on-premise environment and access the prod database through it. I wanted to know,
If there are other better practiced solutions available
A pointer to an example configuration of ReverseProxy maintaining database UserID, databasePwd etc in the proxy and running the API through it.
We are still trying setting-up an UBUNTU based Ngnx server. However, still not sure where to maintain the DBUserID, DBPwd etc.
Any help or guiding pointer will be greatly appreciated. Thanks
I haven't tried this in django, here is what I consider as a neat solution:
Assign an IAM role to the EC2 instance. Assign permission to the role so that it can read from the Systems manager parameter store
Store the credentials (Username, Password, Host, etc) in the aws System manager (SSM) parameter store
Add a boot script to the ec2 and query the systems manager for the parameter. The script will run once when the ec2 instance boots, read the values from SSM and set the values as environment variables
In the django setting file where the db credentials are set, i would read the credential from the environment variables for e.g os.environ['DATABASE_PASSWORD']
References:
EC2 Boot Script
Systems manager parameter store
How to set environment variables in python
Related
My Airflow application is running in AWS EC2 instance which has IAM role as well. Currently I am creating Airflow S3 connection using hardcoded access and secret key. But I want my application to pickup this AWS credentials from this instance itself.
How to achieve this?
We have a similar setup, our Airflow instance run inside containers deployed inside an EC2 machine. We set up the policies to access S3 on the EC2 machine instance profile. You don't need to pick up the credentials in the EC2 machine, because the machine has an instance profile that should have all the permissions that you need. From the Airflow side, we only use aws_default connection, in the extra parameter we only setup the default region, but there aren't any credentials.
Here a details article about Intance Profiles: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html
The question is answered but for future reference, it is possible to do it without relying on aws_default and just doing it via Environment Variables. Here is an example to write logs to s3 using an AWS connection to benefit form IAM:
AIRFLOW_CONN_AWS_LOG="aws://"
AIRFLOW__CORE__REMOTE_LOG_CONN_ID=aws_log
AIRFLOW__CORE__REMOTE_LOGGING=true
AIRFLOW__CORE__REMOTE_BASE_LOG_FOLDER="s3://path to bucket"
I'm specifically trying to do this with Apache Storm (1.0.2), but it's relevant to any service that is secured with Kerberos. I'm trying to run a secured Storm cluster in Docker. There are a number of out-of-the-box docker images out there for Storm, and they work great unsecured. I'm using https://github.com/Baqend/docker-storm. I also have Storm running securely on RHEL VM's.
However, my understanding is that Kerberos ties hostnames to principals, so if I'm making service foobar available to clients, I need to create a principal of foobar/hostname#REALM. Then a client service might connect to hostname with principal foobar, Kerberos will look up foobar/hostname#REALM in its database, find that it's there (because we created a principal with exactly that name), and everything will work.
In my case, it's described here: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_installing_manually_book/content/configure_kerberos_for_storm.html. The nimbus authenticates as storm/<nimbus host>#REALM, and the supervisors and outside clients authenticate as storm/REALM. Everything works.
But here in 2017, we have containers and hostnames are no longer static. So how would I Kerberize a service that runs in Docker Data Center (or Kubernetes, etc)? I have to attach an unknown hostname to the server authentication. I imagine I could create a principal for all possible hostnames and dynamically pick the right one at startup based on where the container lives, but that's kludgy.
Am I misunderstanding how Kerberos works? Is there a solution here that I don't see? I see multiple examples online of people running Storm in Docker, but I can't imagine that nobody's clusters are secure.
I don't know Apache Storm or Docker, but based on previous workings with JBOSS in a cluster in which an inbound client could be connecting to any one of a possible number of different hosts, then you would simply assign a virtual name to the entire pool at the load balancer and kerberize the service according to the virtual name instead of individual host name at the host level. So if you're making service foobar available to clients, you need to create a service principal (SPN) of foobar/virtualhostname#REALM in your Directory to kerberize the service with. You assign that SPN to a user account (not a computer account) to give it the flexibility to work with any Kerberized service which uses that SPN. If you are using Active Directory, you must create a keytab with the SPN inside of it, and place the keytab on each host running the kerberized service instance foobar/virtualhostname#REALM.
Azure VM, Cloud service or Web job?
I have a configurable console application which runs continuosly. Currently it is running on a VM and consumes lot of memory (it is basically doing data mining).
The current requirement is to have multiple instances of this application with different set of configuration which can be changed by specific users.
So where should I host this application such that the configuration can be modified using some front end which provides access managements(like Sharepoint),ability to stop it/restart (like WCF service) without logging on the VM?
I am open to any suggestions/ideas. Thanks
I don't think there's any sold answer to this question as there is the preference variable but for what it's worth, if it were up to me I would deploy it against individual azure VM's for each specific set of users. That way if the server resources went up because of config changes the user group made it is isolated to that group, and with azure, will scale automatically to meet the resource demand. Then just build a little .net web app to allow user to authenticate and change configuration settings.
You could expose an "admin" endpoint for your service (obviously you need authentication here!) that:
1. can return the current configuration
2. accept new configuration
3. restart the service (if needed). Stopping the service will be harder, since that leaves the question on how to start it again.
Then you need to write your own (or use a 3-party (like sharepoint or a CMS)) application that will handle your users and under the hood consume your "admin" endpoint.
Edit: The hosting part: If I understand you correctly your app is just an console application today, and you don't know how to host it? Well, there are many answers to that question. If you have a operations department go talk to them, if you are on your own play around and see what fits you and your environment best!
My tip: go for a http/https protocol/interface - just because there are many web host out there, and you can easy find tools for that protocol. if you are on the .NET platform check out Web.API or OWASP
Azure now has Machine learning to process data mining.
You should check if it's suit to you.
Otherwise, you can use Webjob:
Allow you to have multiple instances of your long time running job (Webjon scaling out).
AppSettings can be change from the Azure Portal or using the Azure Management API
I have currently installed gsutil on a server to access my GCS buckets. I followed the instructions under the section 'How to convert gsutil to use OAuth 2.0' from https://cloud.google.com/storage/docs/gsutil_install
The intermediate steps in the instructions require that a URL is copy pasted in the browser to generate a code that you have to enter again on the terminal. You also need to enter proxy server details (if any).
I am looking for ways to automate this set up and configuration process for gsutil.
Any ideas/references/suggestions/comments are welcome.
Thanks.
Can you say more about what you're trying to do? Are you looking to create distinct credentials for each of a set of users, or are you trying to set up gsutil running on multiple machines all as part of an application that authenticates as that application to Google Cloud Storage?
For the former you need users to set up their own credentials. The web-based dialog for OK'ing the creation of OAuth2 credentials was designed to make it unlikely that a customer could grant long lasting credentials without being aware that they are doing so (for security reasons).
For the latter you should use a service account (see https://cloud.google.com/storage/docs/authentication#service_accounts). You create those credentials once and then deploy them on your production machines along with gsutil - which is a valid security approach because all instances of those machines are authenticating on behalf of an application, not distinct users.
So I'm setting up a dedicated server using Debian 5 Lenny. I will be using some Atlassian Tools (JIRA, Confluence, Bamboo, and Fisheye). I want to use a local LDAP server to store information for the users that will be accessing these software titles, so that they can use one set of credentials to log in.
I also want webmail users to be configured using LDAP.
However, this is a small operation. Three people. That's why all of the software, including the ldap server, will all be on the same machine.
That said, is it safe to use LDAP to store user credentials (including passwords) in LDAP without using Kerberos? I'm confused as to when Kerberos should be used.
Hypothetically, let's say I had two servers on a subnet. Server A received requests from the outside world, for atlassian tools. Server a communicates to ldap server (internally) on server b. In that case, would I use kerberos?
When do I use Kerberos? When do I not?
I am not setting anything like "Active Directory" up. No Samba either. Users do not need to login to a domain (with access to files on the domain), they just need to login to webapps. But if I was doing LDAP on it's own dedicated machine, then I might want Kerberos?
:confuzzled: :(
-Sam
The simplest possible answer is yes, it is possible to store user names, user ids, and passwords without using Kerberos, and in fact directory services accessed via LDAP are an excellent tool for storing this sort of authentication and authorization information.
Update:
In my opinion, if you do choose an open source server, you will find OpenDS to be superior to OpenLDAP or Apache.
Basically, if you have Kerberos, you do not need any directory server. If you aren't in a corporate environment and are looking for an identity management store, you should definitively go for a directory server like OpenLDAP or Apache Directory. Kerberos require running a correctly set up DNS and NTP server. This might be way to much. Even if you do, those lazy morons from Atlassian still did not implement Kerberos support into their products. You can't even go with that.
I just noticed that there are only three of you, maybe a simple database setup with MySQL would suffice instead of running a full-blown directory server?