I am running Cloudera 5.15, with Kerboros enabled on the cluster. Sentry is installed to configure user access to various tables/databases ...etc.
Everything is installed and working fine for Hive, but not for Impala.
I'm using Hue web UI for issuing hive/impala queries. (I'm getting same results using beeline and impala-shell though)
From Hue/Hive:
show current roles;
return --> "professors_role"
From Hue/Impala
show current roles;
return --> no results
When I issue "select current_user()" from hive and impala query editors I'm getting different results.
From hive "select current_user()" returns "hive"
from impala "select current_user()" returns "professor1"
I'm thinking perhaps this is the kulprit, but i'm not sure how to fix? Maybe I've missed a configuration setting in impala somewhere?
Everything works fine in hive - logging in as different users shows me different databases and tables as I would expect based on their assigned roles. Users logged into Impala can't see anything.
Any help is greatly appreciated.
Thanks
I've had this problem, but I fix it now.
You can check which machine you install the impala statestore, and you must have the same impala user/group on the linux system.
Related
I'm new to Hortonworks HDP; I have the following questions:
There are some users that if I don't create them, then even admin can't perform. For example, unless I create a user called hdfs in ambari, I won't be able to do a lot of the file/folder operations on HDFS. Should I create such users? Is this how others manage the cluster?
In Hive interface, I have to click on the 'Execute' button each time I want the query to be executed. Is there a keyboard shortcut for execute? For example in Oracle SQL developer, you press Ctrl+Enter to execute the query. That's what I'm looking for.
Ambari creates required users automatically. But you can set up LDAP if you have such need.
Ranger is also available in hortonworks, Using the Apache Ranger console, security can easily manage using policies for access to files, folders, databases, tables, or column. These policies can be set for individual users or groups and then enforced consistently across HDP stack.
I've got SparkController 2.0.0 running on a HDP 2.4.3 with Spark 1.6.2
In the configuration I have these parameters configured:
sap.hana.es.enable.cache=true
sap.hana.es.cache.max.capacity=500
sap.hana.hadoop.datastore=Hive
I've got HANA 1.00.122 connected to that Spark Controller, set enable_remote_cache parameter to true in indexserver.ini, and imported one of exposed Hive tables as a virtual table in HANA.
Then I ran select-statements against that virtual table, but every time I see that no cache is created (nothing in the Storage tab of Spark UI), nor it is hit (query runtime doesn't drop, and I see the job going through the same stages every time).
Using the hint "with hint (USE_REMOTE_CACHE)" doesn't help either.
Are there any other settings I forgot to make?
In order to enable remote caching for federated queries to Hive from HANA you must also set the HANA parameter enable_remote_cache = true
For more info see the bottom of this page:
https://help.sap.com/viewer/6437091bdb1145d9be06aeec79f06363/2.0.1.0/en-US/1fcb5331b54e4aae82c0340a8a9231b4.html
Accordingly to SAP, the HANA version for caching to work should be 2.0+.
I'm trying to create tables in hive in cloudera quickstartvm 5.5, but it says "Logging intializing" and takes long time, even after long time it doesn't create tables in hive, also cannot import tables to hive.. I have seen lot of questions in stackoverflow on the same issue, but can't find answer. If someone know or got answer for this please share the info. Thanks in advance..
I tried the below from Cloudera community blog
After restarting Hive Service, I tried creating table using Hue worked.. After that create tables in hive started working.. Not sure about the magic behind..
I was new to Cloudera VM and did the following steps. Stopped Hue followed by Hive and started Hive & Hue. One extra thing which I did is to follow the quick start steps in Hue which installed couple of sample tables/databases in Hive. Hope this helps for others who are new to Cloudera VM and facing the same issue.
I know this has been a topic that has been discussed in AWS forums before
and SO How to setup sessionState with SQL Server mode on Amazon RDS
As mentioned on the above thread and on a couple of stack overflow articles it would seem like there is a way to get a SQL session using AWS RDS. I used the above stackoverflow as guidance to try and set up my ASPState database. Using the pastebin script in the above stackoverflow article for a "jobless" InstallSqlState.sql still caused me issues as it tries to use "tempdb" and "master". I don't have access to these databases neither can I grant myself permissions to do so.
However had a working site that used the sessions table on an EC2 server. As per the above stackoverflow article I used the SQL import/export tool.
The database seemed to copy over okay, tables and stored procedures all seem to be present and correct.
In my web.config I have:
<sessionState mode="SQLServer" allowCustomSqlDatabase="true" cookieless="false" timeout="45" sqlConnectionString="data source=RDSIP;initial catalog=ASPState;user id=myuser;password=mypassword" />
However when I run my site I get the error:
Invalid object name 'tempdb.dbo.ASPStateTempApplications'.
Is there anyone who has managed to achieve a session state using SQL on AWS RDS or can point me to a resource that can explain the steps I need to take?
After some digging around I realised that the stored procedures being generated by the pastebin script are still making reference to the tempdb. By doing a simple find replace of [tempdb] to [ASPState] and then re-running the script recreated SP with the correct DB name.
I also changed the "USE" statements to databases I had permissions for.
I've solved my issue create session db on Amazon RDS.
1 Step:- Create ASPState DB on AWS RDS and database and schema migration using Following Tool SQLAzureMW v5.15.6 Release Binary for SQL Server 2014.
2 Step:- change DB Name on sessionState mode="SQLServer" allowCustomSqlDatabase="true" sqlConnectionString="data source=amazon-server-name;initial catalog=ASPState;persist security info=True;user id=userid;password=password" cookieless="false" timeout="100">
please feel free to contact us about any of your queries
Thanks
Amit Verma
I solved the issue by generating scripts for the whole schema (Tasks -> Generate Scripts) from a locally create session database, and then running the same on the RDS instance on a newly created session database.
For the job, I created it manually by copying the exact steps from my local instance.
The setup is running fine till now.
I have to run a simulation with several postgresql databases spread on different machines which all of them are running linux.
I successfully compiled and built postgresql from the source code and I can also run the server, but when I try to create a new db with this command:
./postgresql/bin/createdb db1
I get this error:
createdb: could not connect to database postgres: FATAL: role "giulio" does not exist
where giulio is my username to access all the machines.
On some machine it works while on other it does not. I really cannot figure out the root of the problem. I suppose it is something related with the access control of postgres.
I did several research on google but I was not able to found and to solve the problem.
Does anyone know how to get this work?
Thanks,
-Giulio
PostgreSQL has its own users and roles that are separate from that of your OS. Generally there is a dedicated super user, postgres. For user management info, look here:
http://www.postgresql.org/docs/9.1/interactive/user-manag.html
When executing postgres commands, you need to specify the user with the -U flag (unless you are already logged in as an existing db user). When you called the createdb script, because you didn't use the -U flag, the server assumed that the uid of the caller (giulo) should be used, but you didn't add a user "giulio" to the db, and hence the error message.
So execute the command as
./postgresql/bin/createdb -U postgres db1
and it should work. Then, later on, you may want to create other users and roles in your db rather than doing everything as the superuser.
I would assume that on the machines where the user "giulio" is already known, you executed initdb with exactly this user making him the DB superuser. A quote from inidb(1) (emphasis mine):
--username=username
Selects the user name of the database superuser. This defaults
to the name of the effective user running initdb. It is really
not important what the superuser's name is, but one might choose
to keep the customary name postgres, even if the operating sys‐
tem user's name is different.
On the other machines I assume you did execute initdb with another user, hopefully using postgres.
In order to get back on the standard track I propose, that you delete the database cluster on the machines where "giulio" is the superuser and setup a new database cluster using the standard postgres user. Then add another user "giulio". This will avoid more confusion down the road as some scripts/programs expect a superuser account named postgres.
My answer was more simple...I realized I needed to just run the following:
brew install postgresql
then to check if it worked I ran:
which createdb
to check the file path and if it worked, and it had ! Hope this helps.