I've downloaded free geoiplite databases from link.
I am going to use it in hive-geo-ip-udf.
Update:
SELECT geoip(host,'COUNTRY_CODE','/home/dhruv/GeoLite2-Country.mmdb')
from table_name;
For 64th entry i am getting FAILED: UnsupportedOperationException null.
There was bug in library. It throws exception in case it didn't not find any location for ip. It was fixed now by returning empty string.
https://github.com/Spuul/hive-udfs/issues/2
Related
When I issue this query on solr(separate db) then it is working. But when I am accessing cassandra data through Solr query(I am using DSE) then it returns nothing. And it is giving some error related UserCacheField. So How to enable UserCacheField in a solr query?
Update
My Query is
select * FROM trackfleet_db.location WHERE
solr_query='{"facet":{"pivot":"date,latitude,longitude"},"q":":"}' ;
And I am getting following error
InvalidRequest: Error from server: code=2200 [Invalid query]
message="Field cache is disabled, set the field=date to be docValues=true
and reindex. Or if the field cache will not exceed the heap usage,
then place useFieldCache=true in the request parameters."
The best approach should be enabling of docValues on given field (date) & reindex the data.
But it looks like that you have this field defined with date type, that (per documentation) doesn't support docValues, so you may need to change type for this field to timestamp (I'm not sure that you can use copy field with different type).
Is there a way to get around this error.
"CQO: Internal Error - Optimizer internal error. Assert:
a_drgcidChild->CLength() == UlSafeCLength(popMS->Pdrgcid()) in
rlstreamset.cpp:499"
Facing this issue while loading data from partitioned U-SQL table.
#myData =
SELECT *
FROM dbo.MyTable;
If you encounter any system error message (or something that says Internal Error), please open a support ticket with us and/or send me your job link (if it happens on the cluster) or a self-contained smallest repro (if it is happening with local run) to usql at microsoft dot com.
Thanks
Michael
UPDATE: This issue has been fixed and will be made available in the next refresh. If you are blocked, please contact me for a private runtime.
Somebody tell me I'm not crazy. I have SAS on a server, and I'm running the following code:
data wtf;
a=".123456 1 1";
b=input(a,anydtdtm.);
run;
If I run this on my local computer, no problem. If I run this on the server, I get:
ERROR: An exception has been encountered.
Please contact technical support and provide them with the following traceback information:
The SAS task name is [DATASTEP]
ERROR: Read Access Violation DATASTEP
Exception occurred at (04E0AB8C)
Task Traceback
Address Frame (DBGHELP API Version 4.0 rev 5)
0000000004E0AB8C 0000000009C4EC20 sasxdtu:tkvercn1+0x9B4C
0000000004E030D9 0000000009C4F100 sasxdtu:tkvercn1+0x2099
0000000005FF14BE 0000000009C4F108 uwianydt:tkvercn1+0x47E
0000000002438026 0000000009C4F178 tkmk:tkBoot+0x162E6
Does anyone else get this error???
This is an internal bug that cannot be resolved by the user. You'll need to send this information, your environment description, and the exact steps to recreate the bug over to SAS Technical Support to open up an investigation and determine a workaround.
If your server is a database not made up of .sas7bdat files, it might be due to the SAS/ACCESS engine attempting to translate the function into a way that the server's language can understand, but is unable to do so properly; that is, it might think it's doing it correctly, but it's not. There are special cases where this can occur, and you may have discovered one.
If you are in fact querying some other database, try adding this before running the data step:
options sastrace=',,,d' sastraceloc=saslog;
This will show all of the steps as SAS sends data & functions to and from the server, and may help give some insight.
I am getting the same error on Linux system running SAS 9.4
AUTOMATIC SYSSCP LIN X64
AUTOMATIC SYSSCPL Linux
AUTOMATIC SYSVER 9.4
AUTOMATIC SYSVLONG 9.04.01M3P062415
AUTOMATIC SYSVLONG4 9.04.01M3P06242015
Until SAS can fix the informat you probably need to add additional testing in your code to exclude strange values like that.
While refreshing Webi report I am getting an error:
A database error occured. The database error text is: (CS) "Unexpected behavior" . (WIS 10901)
All the objects are parsing in the universe and Server is also responding. What can be the possible reason?
We are also able to run query in the database using database client tool.
If the error message appears after the a long time it might just be a timeout issue.
Else, you could try to import a version of the report that works in CMS to your local drive, rename it and run again.
It can be caused by some special character in the data combined with the fact that the server language settings do not foresee such character and therefore Business Objects cannot parse it for presentation.
If that is the case you might need to configure an environment variable of the server (like NLS_LANG) setting it to a value such that those special characters in your data can be handled by Business Objects.
In my situation, the error appera when some objet from the data base has changed or does not exists anymore. So we need to delete this object in the Universe or be sure that the field exists in the data base with the same name and type.
I had same problem with my reports. After couple hour of "investigation", I found.
I create Object in my universe, and set inappropriate type of object data Number, when value in database have type Character.
It throw me oracle Error (ORA-01722), and Bussiness Object error (WIS 10901), though SQL copied from report creator interface, executed directly on database return proper data.
Is there a programmatic way to validate HiveQL statements for errors like basic syntax mistakes? I'd like to check statements before sending them off to Elastic Map Reduce in order to save debugging time.
Yes there is!
It's pretty easy actually.
Steps:
1. Get a hive thrift client in your language.
I'm in ruby so I use this wrapper - https://github.com/forward/rbhive (gem install rbhive)
If you're not in ruby, you can download the hive source and run thrift on the included thrift configuration files to generate client code in most languages.
2. Connect to hive on port 10001 and run a describe query
In ruby this looks like this:
RBHive.connect(host, port) do |connection|
connection.fetch("describe select * from categories limit 10")
end
If the query is invalid the client will throw an exception with details of why the syntax is invalid. Describe will return you a query tree if the syntax IS valid (which you can ignore in this case)
Hope that helps.
"describe select * from categories limit 10" didn't work for me.
Maybe this is related to the Hive version one is using.
I'm using Hive 0.8.1.4
After doing some research I found a similar solution to the one Matthew Rathbone provided:
Hive provides an EXPLAIN command that shows the execution plan for a query. The syntax for this statement is as follows:
EXPLAIN [EXTENDED] query
So for everyone who's also using rbhive:
RBHive.connect(host, port) do |c|
c.execute("explain select * from categories limit 10")
end
Note that you have to substitute c.fetch with c.execute, since explain won't return any results if it succeeds => rbhive will throw an exception no matter if your syntax is correct or not.
execute will throw an exception if you've got an syntax error or if the table / column you are querying doesn't exist. If everything is fine, no exception is thrown but also you'll receive no results, which is not an evil thing
In the latest version hive 2.0 comes with hplsql tool which allows us to validate hive commands without actually running them.
Configuration:
add the below XML in hive/conf folder and restart hive
https://github.com/apache/hive/blob/master/hplsql/src/main/resources/hplsql-site.xml
To Run the hplsql and validate the query , please use the below command:
To validate Singe Query
hplsql -offline -trace -e 'select * from sample'
(or)
To Validate Entire File
hplsql -offline -trace -f samplehql.sql
If the query syntax is correct , the response from hplsql would be something like this:
Ln:1 SELECT // type
Ln:1 select * from sample // command
Ln:1 Not executed - offline mode set // execution status
if the query Syntax is wrong , the syntax issue in the query will be reported
If the hive version is older, we need to manually place the hplsql jars inside the hive/lib and proceed.