I created a External Table like this:
CREATE External TABLE IF NOT EXISTS words (word string, timest string,
url string, occs string, nos string, hiveall string, occall string) STORED
BY org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES
('hbase.columns.mapping' =':key, count:timest, count:url, count:occs,
count:nos, other:hiveall, other:occall ')
Is there any way to create the columnfamilys dynamically? so that i have for example something like this:
1397897857000 column=word:occall, timestamp=1449778100184, value=value1
1397897857000 column=otherword:occall, timestamp=1449778100184, value=value2
I thought about something like this but from hive, this code here is from hbase :
Configuration config = HBaseConfiguration.create();
HBaseAdmin admin = new HBaseAdmin(conf);
String table = "myTable";
admin.disableTable(table);
HColumnDescriptor cf1 = ...;
admin.addColumn(table, cf1); // adding new ColumnFamily
HColumnDescriptor cf2 = ...;
admin.modifyColumn(table, cf2); // modifying existing ColumnFamily
admin.enableTable(table);
from here:
http://hbase.apache.org/0.94/book/schema.html
Or does somebody has another idea for my Problem:
I have multiple data from a word count job. This data contains the url, where the word was read from, a timestamp ,when the word was read, the occurance of how often it was found in the url, and some information about a category( there are news, social and all) with the occurance. The main problem is that multiple words can occur at the same timestamp, which will override a existing one. I need the rowkey to be the timestamp to make some querys against it ( like what was most used word in the last 2 weeks).
Column families can't be changed after creation like this. In your scenario, you should create different column qualifiers instead of different column families.
Fix a column family and use word coming as qualifier name. So, it will not override when different words come at the same timestamp.
Related
I am trying to find a way to capture relevant errors from oracle alertlog. I have one table (ORA_BLACKLIST) with column values as below (these are the values which I want to ignore from
V$DIAG_ALERT_EXT)
Below are sample data in ORA_BLACKLIST table. This table can grow based on additional error to ignore from alertlog.
ORA-07445%[kkqctdrvJPPD
ORA-07445%[kxsPurgeCursor
ORA-01013%
ORA-27037%
ORA-01110
ORA-2154
V$DIAG_ALERT_EXT contains a MESSAGE_TEXT column which contains sample text like below.
ORA-01013: user requested cancel of current operation
ORA-07445: exception encountered: core dump [kxtogboh()+22] [SIGSEGV] [ADDR:0x87] [PC:0x12292A56]
ORA-07445: exception encountered: core dump [java_util_HashMap__get()] [SIGSEGV]
ORA-00600: internal error code arguments: [qercoRopRowsets:anumrows]
I want to write a query something like below to ignore the black listed errors and only capture relevant info like below.
select
dae.instance_id,
dae.container_name,
err_count,
dae.message_level
from
ORA_BLACKLIST ob,
V$DIAG_ALERT_EXT dae
where
group by .....;
Can someone suggest a way or sample code to achieve it?
I should have provided the exact contents of blacklist table. It currently contains some regex (perl) and I want to convert it to oracle like regex and compare with v$diag_alert_ext message_text column. Below are sample perl regex in my blacklist table.
ORA-0(,|$| )
ORA-48913
ORA-00060
ORA-609(,|$| )
ORA-65011
ORA-65020 ORA-31(,|$| )
ORA-7452 ORA-959(,|$| )
ORA-3136(,|)|$| )
ORA-07445.[kkqctdrvJPPD
ORA-07445.[kxsPurgeCursor –
Your blacklist table looks like like patterns, not regular expressions.
You can write a query like this:
select dae.* -- or whatever columns you want
from V$DIAG_ALERT_EXT dae
where not exists (select 1
from ORA_BLACKLIST ob
where dae.message_text like ob.<column name>
);
This will not have particularly good performance if the tables are large.
I'm attempting to write a program that will grab the content from fields from a table both specified by the user on the selection screen.
For example, the user could specify the fields equnr, b_werk, b_lager from the table eqbs.
I've been able to accomplish this like so:
" Determine list of fields provided by user
DATA(lv_fields) = COND string(
WHEN p_key3 IS NOT INITIAL AND p_string IS NOT INITIAL THEN
|{ p_key1 }, { p_key2 }, { p_key3 }, { p_string }|
WHEN p_key2 IS NOT INITIAL AND p_string IS NOT INITIAL THEN
|{ p_key1 }, { p_key2 }, { p_string }|
WHEN p_key2 IS NOT INITIAL AND p_string IS NOT INITIAL THEN
|{ p_key1 }, { p_string }| ).
DATA: lv_field_tab TYPE TABLE OF line.
APPEND lv_fields TO lv_field_tab.
" Determine table specified by user and prepare for Open SQL query
DATA t_ref TYPE REF TO data.
FIELD-SYMBOLS: <t> TYPE any,
<comp> TYPE any.
CREATE DATA t_ref TYPE (p_table).
ASSIGN t_ref->* TO <t>.
ASSIGN COMPONENT lv_fields OF STRUCTURE <t> TO <comp>.
" Prepare result container
DATA: lt_zca_str_to_char TYPE TABLE OF zca_str_to_char,
ls_zca_str_to_char TYPE zca_str_to_char.
SELECT (lv_field_tab) FROM (p_table) INTO (#ls_zca_str_to_char-key1, #ls_zca_str_to_char-key2, #ls_zca_str_to_char-key3, #ls_zca_str_to_char-string).
APPEND ls_zca_str_to_char TO lt_zca_str_to_char.
ENDSELECT.
This will correctly populate lt_zca_str_to_char with data from the table specified by the user.
However, this implies that the user is always providing p_key1, p_key2, and p_key3. I could perform a different selection statement based on how many key fields the user provides, but what's the fun in that?
I set out to solve this like this:
DATA(lv_results) = COND string(
WHEN p_key3 IS NOT INITIAL AND p_string IS NOT INITIAL THEN
|(#ls_zca_str_to_char-key1, #ls_zca_str_to_char-key2, #ls_zca_str_to_char-key3, #ls_zca_str_to_char-string)|
WHEN p_key2 IS NOT INITIAL AND p_string IS NOT INITIAL THEN
|(#ls_zca_str_to_char-key1, #ls_zca_str_to_char-key2, #ls_zca_str_to_char-string)|
WHEN p_key2 IS NOT INITIAL AND p_string IS NOT INITIAL THEN
|(#ls_zca_str_to_char-key1, #ls_zca_str_to_char-string)| ).
SELECT (lv_field_tab) FROM (p_table) INTO (#lv_results).
APPEND ls_zca_str_to_char TO lt_zca_str_to_char.
ENDSELECT.
This will activate, and when I get to my Open SQL query (from a Z table, only filling out the first two of three possible key fields), the values are the following:
lv_field_tab = GUID, TEXT_ID, TEXT_DATA (Good)
p_table = ZCR_TRANS_TEXT (Good)
lv_results = (#ls_zca_str_to_char-key1, #ls_zca_str_to_char-key2, #ls_zca_str_to_char-string) (Good, 3 = 3!)
But, since I'm assuming the compiler is seeing (#lv_results) as one single variable, the program dumps with the following error:
The current ABAP program attempted to execute an Open SQL statement
containing a dynamic entry. The parser returned the following error:
"The field list and the INTO list must have the same number of
elements."
Is it possible for me to use the new Open SQL syntax to accomplish my dynamic INTO clause in harmony with my dynamic field list?
The brackets on the INTO do not do what you expect, from the ABAP help:
... INTO (#dobj1, #dobj2, ... )
Effect
If the results set consists of multiple columns or aggregate expressions specified explicitly in the SELECT list, a list of elementary data objects dobj1, dobj2, ... (in parentheses and separated by commas) can be specified after INTO.
In your case you only have one value in there so you can only select one column and the data will be passed in the variable LV_RESULT. Not what you are looking for. Since you want to fill the fields of an existing structure the INTO CORRESPONDING FIELDS OF construct will work here. And you can use TABLE to make your command more efficient as well. This leads to:
SELECT (lv_field_tab) FROM (p_table)
INTO CORRESPONDING FIELDS OF TABLE #lt_zca_str_to_char.
As said previously, you may use INTO CORRESPONDING FIELDS OF ..., but it's not mandatory, it's only for simplifying the code.
So, instead of using CORRESPONDING FIELDS, you may create a structure dynamically (RTTC) with its components corresponding to the columns in LV_FIELD_TAB, and you may then use:
SELECT (lv_field_tab) FROM (p_table) INTO #<structure> ... ENDSELECT.
But of course, as explained by Gert Beukema, you should better do only one SELECT, by creating an internal table dynamically with the same logic as for the structure above, and you may then use:
SELECT (lv_field_tab) FROM (p_table) INTO TABLE #<internal table> ...
Refer to the many examples in the web how to create data objects dynamically with RTTC.
Do not use a fields list for your INTO clause.
Try with
INTO CORRESPONDING FIELDS OF TABLE
must be a FIELD-SYMBOL type any table, and the rest of the logic is up to you (to put the proper information from your generic and almost-empty to your specific destination one).
TABLES: VBRK.
DATA: BEGIN OF it_test,
BUKRS LIKE VBRK-BUKRS,
FKDAT LIKE VBRK-FKDAT,
END OF it_test.
DATA: wa_test LIKE it_test.
SELECT * FROM VBRK INTO CORRESPONDING FIELD OF wa_test.
IF wa_test-BUKRS = 'xxxx'.
wa_test-BUKRS = 'XXXXX' "Problem occurs here as the BUKRS allow 4 value
APPEND wa_test TO it_test.
ENDIF.
Then I want to map the internal table to output as ALV table. Is they any way to change the field length afterwards?
Apart from multiple issues in your code, you can't. If you need something similar to that, add an additional field to the structure with whatever size you require and copy the values over.
If the objective is to output something to the screen that is different(or differently formatted) that what is stored internally(or in the database), then the use of a data element with a conversion exit maybe the way to go.
For an example, look at the key fields of table PRPS.
Expanding the answer of vwegert:
The MOVE-CORRESPONDINGcommand (and SELECT ... INTO CORRESPONDING FIELDS) don't need the same field type. The content is converted. So you could define a 5-character field in your internal structure and copy the BUKRS-value into this 5-character field:
TABLES: VBRK.
DATA: BEGIN OF it_test,
BUKRS(5), "longer version of VBRK-BUKRS,
FKDAT LIKE VBRK-FKDAT,
END OF it_test.
DATA: tt_test TYPE STANDARD TABLE OF it_test.
* I would strongly recommend to set a filter!
SELECT * FROM VBRK INTO CORRESPONDING FIELD OF it_test.
IF it_test-BUKRS = 'xxxx'.
it_test-BUKRS = 'XXXXX'.
APPEND it_test to tt_test.
ENDIF.
ENDSELECT.
A pitfall: When you use it with ALV you will loose the field description. (on the other side, the field description of the original field will not fit any longer the new field.)
I have a table that has a column called RAW DATA of type NVARCHAR MAX, which is a dump from a web service. Here is a sample of 1 data line:
<CourtRecordEventCaseHist>
<eventDate>2008-02-11T06:00:00Z</eventDate>
<eventDate_TZ>-0600</eventDate_TZ>
<histSeqNo>4</histSeqNo>
<countyNo>1</countyNo>
<caseNo>xxxxxx</caseNo>
<eventType>WCCS</eventType>
<descr>Warrant/Capias/Commitment served</descr>
<tag/>
<ctofcNameL/>
<ctofcNameF/>
<ctofcNameM/>
<ctofcSuffix/>
<sealCtofcNameL/>
<sealCtofcNameF/>
<sealCtofcNameM/>
<sealCtofcSuffix/>
<sealCtofcTypeCodeDescr/>
<courtRptrNameL/>
<courtRptrNameF/>
<courtRptrNameM/>
<courtRptrSuffix/>
<dktTxt>Signature bond set</dktTxt>
<eventAmt>0.00</eventAmt>
<isMoneyEnabled>false</isMoneyEnabled>
<courtRecordEventPartyList>
<partyNameF>Name</partyNameF>
<partyNameM>A.</partyNameM>
<partyNameL>xxxx</partyNameL>
<partySuffix/>
<isAddrSealed>false</isAddrSealed>
<isSeal>false</isSeal>
</courtRecordEventPartyList>
</CourtRecordEventCaseHist>
It was suppose to go in a table, with the node names representing the column names. The table it's going to is created, I just need to exract the data from this row to the table. I have 100's of thousands records like this. I was going to copy to a xml file, then import. But there is so much data, I would rather try and do the work within the DB.
Any ideas?
First, create the table with all the required columns.
Then, use your favorite scripting language to load the table! Mine being groovy, here is what I'd do:
def sql = Sql.newInstance(/* SQL connection here*/)
sql.eachRow("select RAW_DATA from TABLE_NAME") { row ->
String xmlData = row."RAW_DATA"
def root = new XmlSlurper().parseText(xmlData)
def date = root.eventDate
def histSeqNo = root.histSeqNo
//Pull out all the data and insert into new table!
}
I did find an answer to this, I'm sure there is more than one way of doing this. But this is what I got to work. Thanks for everyone's help.
SELECT
pref.value('(caseNo/text())[1]', 'varchar(20)') as CaseNumber,
pref.value('(countyNo/text())[1]', 'int') as CountyNumber
FROM
dbo.CaseHistoryRawData_10 CROSS APPLY
RawData.nodes('//CourtRecordEventCaseHist') AS CourtRec(pref)
We would like to store both rails query string and table name in db and retrieve them for execution at run time. Here is the scenario:
Retrieve active customer records from customers table. Let's say we have 2 variable defined as:
table_name = 'Customer'
query_string = ':active => true'
In rails, the query could be:
records = Customer.where(:active => true)
Now with table name and query string stored in variables table_name and query_string, is it possible to assemble a query string with 2 variables like:
records = table_name.where(query_string) ?
Thanks for the help.
You could do this, but it's not generally recommended to evaluate a string as a hash. Also, table_name is an unfortunate name for the variable, because you actually are storing the class name (table would be 'customers'). In any event, what you are missing is the eval of these strings:
records = class_name.constantize.where(instance_eval(query_string))
Note that running instance_eval on a user-inputted string can be disasterous for security and the well-being of your application. Use with care, and stick to building an actual hash.
The definition of instance_eval is: Evaluates a string containing Ruby source code, or the given block, within the context of the receiver (obj).
Another way to eval a query string is to include where in the string, like:
table_name = 'Customer'
query_string = 'where(:active => true)'
Then the record could be retrieved by:
records = table_name.constantize.instance_eval(query_string)
By putting where into the string, we can use the full power of instance_eval instead of just returning the source code to where as in the question above.