Add field/string length to logstash event - lucene

I'm trying to add a string length field to an index. Ideally, I'd like to use the kibana script feature as I can 'add' this field later but I keep getting a null_pointer_exception with the following code... I'm trying to sort in a visualization based on the fields length.
doc['field'].value ? doc['field'].length() : 0
Is this correct?
I thought it was because my field isn't always set (sparse data), but I added the ?:0 to combat that (which didn't work)
Any ideas?

You can define an scripted field in Kibana, of type int, language painless, and try this:
return (doc['field'].value != null? doc['field'].value.length(): 0);

Related

SQL Server - Error: converting charcter string in "uniqueidentifier"

I am new to SQL Server and am trying to implement the steps from this tutorial: https://www.youtube.com/watch?v=ElGSvn3OCK4 (At around minute 12).
I want to implement a semantic search. Therefore I want to set a Search Property List. Here is the code (which is from the tutorial):
ALTER SEARCH PROPERTY LIST DocumentPropertiesTest
ADD 'Title'
WITH (PROPERTY_SET_GUID = 'F29F85E0-1068-AB91-08002B27B309', PROPERTY_INT_ID = 2,
PROPERTY_DESCRIPTION = 'System.Title = Title of the item' );
GO
I am getting an error that says that the conversion in uniqueidentifier failed. Can anyone explain what this means in this example? Thanks a lot!
A UNIQUEIDENTIFIER field must have a valid GUID
Your string 'F29F85E0-1068-AB91-08002B27B309' isn't a GUID.
You can use something like this to validate.
A GUID has another 4 digit block that you seem to be missing.
For example:
'F29F85E0-1068-0000-AB91-08002B27B309' note the 0000 around the middle.

How to retrieve a specific field from a list output?

I don't have any developer rights in my SAP-System but I found a way to write some ABAP-Code in a tiny "User-Exit" box (I don't know if that's what you call it) inside a report.
I'm trying to submit a HR-Report and plug it's outcoming PERNR into that same report again.
There's a syntax-error that is telling me that t_list doesn't have a component with the Name PERNR.
What do I have to do in order to get this to work?
DATA: t_list TYPE TABLE OF abaplist WITH HEADER LINE,
seltab TYPE TABLE OF rsparams,
selline LIKE LINE OF seltab.
*I found out that the name of the selection field in the Report-GUI is "PNPPERNR" and tested it
selline-selname = 'PNPPERNR'.
selline-sign = 'I'.
selline-option = 'EQ'.
SUBMIT Y5000112
USING SELECTION-SET 'V1_TEST'
EXPORTING LIST TO MEMORY
AND RETURN.
CALL FUNCTION 'LIST_FROM_MEMORY'
TABLES
listobject = t_list
EXCEPTIONS
not_found = 1
OTHERS = 2.
IF sy-subrc <> 0.
WRITE 'Unable to get list from memory'.
ELSE.
LOOP AT t_list.
*The Problem is here: how do I get the pnppernr out of t_list, it's the first column of the report output
selline-low = t_list-pernr.
append selline to seltab.
ENDLOOP.
SUBMIT Y5000112
WITH SELECTION-TABLE seltab
USING SELECTION-SET 'V2_TEST'
AND RETURN.
ENDIF.
Use the function module LIST_TO_ASCI to decode the contents of t_list into something readable. This answer contains some sample code including the data types required. At this point, the data you're looking for will probably occur at the same column range in the output. Use the standard substring access methods - e. g. line+42(21) to obtain the part of the line you need.
The vwegert's answer is more than useful! In my previous answer I forgot to mention LIST_TO_ASCI FM :)
The only thing I can add is that parsing of result lines has no universal solution and greatly depends on its structure. Usually it is done like:
LOOP AT t_list.
SPLIT t_list AT '|' INTO <required_structure>.
selline-low = <required_structure>-pernr.
APPEND selline TO seltab.
ENDLOOP.
where <`required_structure> is your Y5000112 output structure. But this may be not so simple and may require additional manipulations.

Peoplesoft CreateRowset with related display record

According to the Peoplebook here, CreateRowset function has the parameters {FIELD.fieldname, RECORD.recname} which is used to specify the related display record.
I had tried to use it like the following (just for example):
&rs1 = CreateRowset(Record.User, Field.UserId, Record.UserName);
&rs1.Fill();
For &k = 1 To &rs1.ActiveRowCount
MessageBox(0, "", 999999, 99999, &rs1(&k).UserName.Name.Value);
End-for;
(Record.User contains only UserId(key), Password.
Record.UserName contains UserId(key), Name.)
I cannot get the Value of UserName.Name, do I misunderstand the usage of this parameter?
Fill is the problem. From the doco:
Note: Fill reads only the primary database record. It does not read
any related records, nor any subordinate rowset records.
Having said that, it is the only way I know to bulk-populate a standalone rowset from the database, so I can't easily see a use for the field in the rowset.
Simplest solution is just to create a view, but that gets old very soon if you have to do it a lot. Alternative is to just loop through the rowset yourself loading the related fields. Something like:
For &k = 1 To &rs1.ActiveRowCount
&rs1(&k).UserName.UserId.value = &rs1(&k).User.UserId.value;
&rs1(&k).UserName.SelectByKey();
End-for;

REGEX_EXTRACT error in PIG

I have a CSV file with 3 columns: tweetid , tweet, and Userid. However within the tweet column there are comma separated values.
i.e. of 1 row of data:
`396124437168537600`,"I really wish I didn't give up everything I did for you, I'm so mad at my self for even letting it get as far as it did.",savava143
I want to extract all 3 fields individually, but REGEX_EXTRACT is giving me an error with this code:
a = LOAD tweets USING PigStorage(',') AS (f1,f2,f3);
b = FILTER a BY REGEX_EXTRACT(f1,'(.*)\\"(.*)',1);
The error is:
error: Filter's condition must evaluate to boolean.
In the use case shared, reading the data using PigStrorage(',') will result in missing savava143 (last field value)
A = LOAD '/Users/muralirao/learning/pig/a.csv' USING PigStorage(',') AS (f1,f2,f3);
DUMP A;
Output : A : Observe that the last field value is missing.
(396124437168537600,"I really wish I didn't give up everything I did for you, I'm so mad at my self for even letting it get as far as it did.")
For the use case shared, to extract all the values from CSV file with field values having ',' we can use either CSVExcelStorage or CSVLoader.
Approach 1 : Using CSVExcelStorage
Ref : http://pig.apache.org/docs/r0.12.0/api/org/apache/pig/piggybank/storage/CSVExcelStorage.html
Input : a.csv
396124437168537600,"I really wish I didn't give up everything I did for you, I'm so mad at my self for even letting it get as far as it did.",savava143
Pig Script :
REGISTER piggybank.jar;
A = LOAD 'a.csv' USING org.apache.pig.piggybank.storage.CSVExcelStorage() AS (f1,f2,f3);
DUMP A;
Output : A
(396124437168537600,I really wish I didn't give up everything I did for you, I'm so mad at my self for even letting it get as far as it did.,savava143)
Approach 2 : Using CSVLoader
Ref : http://pig.apache.org/docs/r0.9.1/api/org/apache/pig/piggybank/storage/CSVLoader.html
Below script makes use of CSVLoader(), DUMP A will result in the same output seen earlier.
A = LOAD 'a.csv' USING org.apache.pig.piggybank.storage.CSVLoader() AS (f1,f2,f3);
The error is that you do not want to FILTER based on a regex but GENERATE new fields based on a regex. To filter, you need to know if the line have to be filtered, hence the boolean requirement.
Therefore, you have to use :
b = FOREACH a GENERATE REGEX_EXTRACT(FIELD, REGEX, HOW_MANY_GROUPS_TO_RETURN);
However, as #Murali Rao said, your values are not just coma separated but CSV (think how you will handle a coma in tweet : it is not a field separator, just some content).

Orientdb sql auto increment: id is always null (sql batch, update increment, variables)

I was looking at another question in stackoverflow regarding auto increment fields in orientdb, where one of the answers was to create our own vertex with counter field.
However, when I'm trying to execute the following code (both java api and console script batch), It is not working.
Do note however that the id is returned good (did some debug attempts, returning the id variable only), and the vertex is created.
However, the vertex id is always null (unless I set it explicit, that is).
The script:
script sql
LET id = UPDATE CCounter INCREMENT value=1 RETURN AFTER $current WHERE name='session'
LET csession = CREATE VERTEX CDate SET id=$id.result, meet_date='2015-01-01 15:23:00'
end
I tried playing around with $id and $current , but nothing seems to work.
Currently I am doing it in a 2-transaction mode; one to get the id, and another to create the vertex. I really hope there is a better way though.
P.S.
I am using version 2.0-M2
You should execute
LET csession = CREATE VERTEX CDate SET id=$id.value, meet_date='2015-01-01 15:23:00'
Note the $id.value in place of $id.result.
As stated in a comment to Lvca, the answer was:
(1) Change the field name. Id seems to be reserved (ish). It's probably possible to still bypass it and use a field named 'id', but I didn't want to mess around with it.
(2) From some reason, the result was a collection (shown as '[id]'). It took me some time to figure it out, but I just had to choose the first value from it.
(3) Also, there are 2 'values' here. One of the field ($current.value), and the second one(not sure where it's coming from).
Final solution that works:
script sql
LET id = UPDATE CCounter INCREMENT value = 1 RETURN AFTER $current.value WHERE name='session'
LET csession = CREATE VERTEX CDate SET data_id=$id[0].value, meet_date='2015-01-01 15:23:00'
end