Passing pig macro parameters - apache-pig

I have a pig macro
define chop_massive (my, maxev) returns grouped, massive {
gr = foreach (group $my by id) generate group as id, $my;
split gr into
massive if COUNT($my) > $maxev,
grouped otherwise;
$grouped = grouped;
$massive = foreach massive generate id, COUNT($my) as $my;
};
my problem is with passing maxev parameter.
when I use a constant literal (e.g., 100L) all is fine.
however, I want to use $MAX_EVENTS specified using -param on the command line.
I tried
A, massive = chop_massive(A, $MAX_EVENTS);
and got
mismatched input '100L' expecting set null
I tried
A, massive = chop_massive(A, ($MAX_EVENTS));
and got
mismatched input '(' expecting set null
I tried
A, massive = chop_massive(A, '$MAX_EVENTS');
and got
Macro doesn't support user defined schema that contains name that conflicts with alias name: A
I tried
massive if COUNT($my) > $MAX_EVENTS,
inside the macro and got
Macro inline failed for macro 'chop_massive'. Reason: Undefined parameter : MAX_EVENTS
(this error is at least clear and reasonable).
So what should I do?

While I don't see it documented anywhere, it wouldn't surprise me if expressions are not supported as macro arguments. Why not just pass $EVENTS_PER_DAY and $DAYS as parameters to the macro (three parameters instead of two)?

Related

Pass a variable into a SQL query in Excel Power Query

Long story short, I have a sql query pulling data from a database into a spreadsheet. I want to be able to change the Plant filter inside the query via Power Query, I have tried using a custom function and adding that into the query where the variable is set (see below)
Sql.Database("server", "database", [Query="DECLARE #FilterOnPlant AS varchar(3)#(lf)SET #FilterOnPlant = '" + filterOnPlant + "'#(lf)#(lf)SELECT
In doing this I keep getting the following error:
Expression.Error: We cannot apply operator + to types Text and Table.
Details:
Operator=+
Left=DECLARE #FilterOnPlant AS varchar(3)
SET #FilterOnPlant = '
Right=[Table]
I figure it has something to do with the '+' operator used but can't find an alternative.
I have tried using:
'&'
'+'
'and'
but they all yield the same error (with the exception of 'and' that error states that it's not being used in a logical statement which makes sense)
For further info:
The custom function to get the value I want is:
filterOnPlant = Excel.CurrentWorkbook(){[Name="Active_Plant"]}[Content]
The intention is to get the Value from a cell Named "Active_Plant"
EDIT:
after further testing I have found the issue lies with variable filterOnPlant, when I exchange that for a string value then the query works as expected. If anyone knows how I can get CELL().Value of a named range in power query would be super helpful
Solved!
In my filterOnPlant function I was missing {0}[Column1]
Original: filterOnPlant = Excel.CurrentWorkbook(){[Name="Active_Plant"]}[Content]
New (Working): filterOnPlant = Excel.CurrentWorkbook(){[Name="Active_Plant"]}[Content]{0}[Column1]

Looking for a way to get a detailed description of ZTERM

I am currently trying to program a function module which should, in theory, output a custom table like T052, but with an additional field Z_TEXTLONG, which explains the details of the chosen ZTERM, akin to the text in FI_F4_ZTERM's popup. Here's what I tried:
LOOP AT T_ZBEDS ASSIGNING FIELD-SYMBOL(<line>).
CALL FUNCTION 'FI_F4_ZTERM'
EXPORTING
I_KOART = 'K'
I_ZTERM = <line>-zterm
I_XSHOW = ''
I_ZTYPE = ''
I_NO_POPUP = 'X'
IMPORTING
E_ZTERM = v_text
EXCEPTIONS
NOTHING_FOUND = 1
OTHERS = 2.
WRITE v_text TO <line>-Z_TEXTLONG.
From what I gathered, this does not work due to FI_F4_ZTERM writing the list it returns into E_ZTERM, not a single value, which would be what I need. I am a bit lost as to what I should do next. I tried looking into how exactly FI_F4_ZTERM generates these texts or where it calls them from, but I was not successful. Currently, I am trying to maybe get this text from V_T052, but that does not work either. I would be thankful for any suggestions.
Try function FI_TEXT_ZTERM, it has single import parameter I_T052
why not just call CALL FUNCTION 'FI_F4_ZTERM'
with I_NO_POPUP = 'X'
importing
ET_ZTERM = lt_zterm.
you get the list of payment terms and their texts.
Then in the loop read it from the table of payment terms.
LOOP AT T_ZBEDS ASSIGNING FIELD-SYMBOL(<line>).
read table lt_zterm assigning <term>
with key ZTERM = <line>-zterm.
ENDLOOP.

How to read automatically-created variants

I need to read automatically-created variants, to get the selection screen parameters and other selection criteria. The names of such variants begin with symbol & (for example, &0000000000425). Such variants are created once you schedule any background job from se80/se38 or any transaction without choosing any existing variant (from the selection screen, menu Program -> Execute in background).
The function module RS_VARIANT_CONTENTS works fine for normal variants (which can be seen via se80/se38), but not for the automatically-created ones (that begin with &). I looked into the FM and found that the VARI table was read by the next code:
IMPORT %_VARI40C TO P_VARI
%_VARI40 TO L_VARI_40
%_VARI TO L_VARI
%_VARIVDAT TO P_VARIVDAT
* %_VARIDYN40 TO P_VARIDYN
%_VARIVDAT_DYN40 TO P_VDATDYN
DYNS_FIELDS TO OLD_DYNSFIELDS
DYNS_TEXPRI TO OLD_TEXPRI
DYNS_EXPR TO OLD_EXPR
DYNS_FIELD_TAB TO DYNS_FIELDS
DYNS_TEXPR TO DYN_SEL-TEXPR
FROM DATABASE VARI(VB) CLIENT L_CLIENT ID P_RKEY
ignoring structure boundaries
IGNORING CONVERSION ERRORS.
However the import returns nothing for the & variants. It looks like the values for & variants are stored within a format which is different from the format used by FM RS_VARIANT_CONTENTS.
Is there any way to find a proper format/data structure for & variants values?
Update: I created ZBC_TEST program and scheduled it as a job. I see a record in VARI table:
MANDT RELID REPORT VARIANT SRTF2
200 VB ZBC_TEST &0000000000425 0
So, the &0000000000425 variant exists in VARI table. VARI-CLUSTD field is not empty for the record. I use this code:
CALL FUNCTION 'RS_VARIANT_CONTENTS'
EXPORTING
REPORT = 'ZBC_TEST'
VARIANT = '&0000000000425'
MOVE_OR_WRITE = 'W'
IMPORTING
SP = lv_sp
TABLES
VALUTAB = lt_valtab.
The FM has been performed without any exception and sy-subrc=0, but lt_valtab table is empty...
The function module is working correctly: Since your program does not have any parameters, the returned value set is empty.

REGEX_EXTRACT error in PIG

I have a CSV file with 3 columns: tweetid , tweet, and Userid. However within the tweet column there are comma separated values.
i.e. of 1 row of data:
`396124437168537600`,"I really wish I didn't give up everything I did for you, I'm so mad at my self for even letting it get as far as it did.",savava143
I want to extract all 3 fields individually, but REGEX_EXTRACT is giving me an error with this code:
a = LOAD tweets USING PigStorage(',') AS (f1,f2,f3);
b = FILTER a BY REGEX_EXTRACT(f1,'(.*)\\"(.*)',1);
The error is:
error: Filter's condition must evaluate to boolean.
In the use case shared, reading the data using PigStrorage(',') will result in missing savava143 (last field value)
A = LOAD '/Users/muralirao/learning/pig/a.csv' USING PigStorage(',') AS (f1,f2,f3);
DUMP A;
Output : A : Observe that the last field value is missing.
(396124437168537600,"I really wish I didn't give up everything I did for you, I'm so mad at my self for even letting it get as far as it did.")
For the use case shared, to extract all the values from CSV file with field values having ',' we can use either CSVExcelStorage or CSVLoader.
Approach 1 : Using CSVExcelStorage
Ref : http://pig.apache.org/docs/r0.12.0/api/org/apache/pig/piggybank/storage/CSVExcelStorage.html
Input : a.csv
396124437168537600,"I really wish I didn't give up everything I did for you, I'm so mad at my self for even letting it get as far as it did.",savava143
Pig Script :
REGISTER piggybank.jar;
A = LOAD 'a.csv' USING org.apache.pig.piggybank.storage.CSVExcelStorage() AS (f1,f2,f3);
DUMP A;
Output : A
(396124437168537600,I really wish I didn't give up everything I did for you, I'm so mad at my self for even letting it get as far as it did.,savava143)
Approach 2 : Using CSVLoader
Ref : http://pig.apache.org/docs/r0.9.1/api/org/apache/pig/piggybank/storage/CSVLoader.html
Below script makes use of CSVLoader(), DUMP A will result in the same output seen earlier.
A = LOAD 'a.csv' USING org.apache.pig.piggybank.storage.CSVLoader() AS (f1,f2,f3);
The error is that you do not want to FILTER based on a regex but GENERATE new fields based on a regex. To filter, you need to know if the line have to be filtered, hence the boolean requirement.
Therefore, you have to use :
b = FOREACH a GENERATE REGEX_EXTRACT(FIELD, REGEX, HOW_MANY_GROUPS_TO_RETURN);
However, as #Murali Rao said, your values are not just coma separated but CSV (think how you will handle a coma in tweet : it is not a field separator, just some content).

Zoho Creator making a custom function for a report

Trying to wrap my head around zoho creator, its not as simple as they make it out to be for building apps… I have an inventory database, and i have four fields that I call to fill a field called Inventory Number (Inv_Num1) –
First Name (First_Name)
Last Name (Last_Name)
Year (Year)
Number (Number)
I have a Custom Function script that I call through a Custom Action in the form report. What I am trying to do is upload a CSV file with 900 entries. Of course, not all of those have those values (first/last/number) so I need to bulk edit all of them. However when I do the bulk edit, the Inv_Num1 field is not updated with the new values. I use the custom action to populate the Inv_Num1 field with the values of the other 4 fields.
Heres is my script:
void onetime.UpdateInv()
{
for each Inventory_Record in Management
{
FN = Inventory_Record.First_Name.subString(0,1);
LN = Inventory_Record.Last_Name.subString(0,1);
YR = Inventory_Record.Year.subString(2,4);
NO = Inventory_Record.Number;
outputstr = FN + LN + YR + NO;
Inventory_Record.Inv_Num1 = outputstr;
}
}
I get this error back when I try to run this function
Error.
Error in executing UpdateInv workflow.
Error in executing For Each Record task.
Error in executing Set Variable task. Unable to update template variable FN.
Error evaluating STRING expression :
Even though there is a First Name for example, it still thinks there is none. This only happens on the fields I changed with Bulk Edit. If I do each one by hand, then the custom action works—but of course then the Inv_Num1 is already updated through my edit on success functions and makes the whole thing moot.
this may be one year late, you might have found the solution but just to highlight, the error u were facing was just due to the null value in first name.
you just have put a null check on each field and u r good to go.
you can generate the inv_number on the time of bulk uploading also by adding null check in the same code on and placing the code on Add> On Submt.( just the part inside the loop )
the Better option would be using a formula field, you just have to put this formula in that formula field and you'll get your inventory_number autogenerated , you can rename the formula_field to Inv Number or whaterver u want.
Since you are using substring directly in year Field, I am assuming the
year field as string.else you would have to user Year.tostring().substring(2,4) & instead of if(Year=="","",...) you have to put if(Year==null , null,...);
so here's the formula
if(First_Name=="","",First_Name.subString(0,1))+if(Last_Name =="","",Last_Name.subString(0,1)) + if(Year=="","",Year.subString(2,4)+Number
Let me know ur response if u implement this.
Without knowing the datatype its difficult to fix, but making the assumption that your Inventory_Record.number is a numeric data item you are adding a string to a number:
The "+" is used for string Concatenation - Joiner but it also adds two numbers together so think "a" + "b" = "ab" for strings but for numbers 1 + 2 = 3.
All good, but when you do "a" + 2 the system doesn't know whether to add or concatenate so gives an error.