I have a dataframe with column abc having values like below
[{note=Part 3 of 4; Total = $11,000, cost=2750, startDate=2021-11-01T05:00:00Z+0000}]
Now I want to extract data based on name,for example i want to extract cost and start date and create a new column.
Asking it to be working on name because the order of these values might change.
I have tried below line of code but due to change in the data order I am getting wrong data.
df_mod = df_mod.withColumn('cost', split(df_mod['costs'], ',').getItem(1)) \
.withColumn('costStartdate', split(df_mod['costs'], ',').getItem(2))
That's because your data is not comma-separated, it just looks like that. You'll want to use regexp_extract to find the correct content.
Related
I have following code:
df.orderBy(expr("COUNTRY_NAME").desc, expr("count").asc).show()
I expect count column to be arranged in ascending order for a given COUNTRY_NAME. But I see something like this:
Last value of 12 is not as per the expectation.
Why is it so?
If you output df.printSchema(), you'll see that your "count" column is of the string datatype, resulting in the undesired alphanumeric sort.
In pyspark, you can use the following to accomplish what you are looking for:
df = df.withColumn('count',df['count'].cast('int'))
df.orderBy(['COUNTRY_NAME'],ascending=False).orderBy(['count'],ascending=True).show()
You should create and apply your schema when the data is read in - if possible.
Hi I want to take value cuisine from table a in column a with value - ak://food/category?pageTitle=Minuman&cuisine=MINUMAN&sortBy=1. The result I want is MINUMAN. How to take value cuisine?
Use below as an example
select regexp_extract('ak://food/category?pageTitle=Minuman&cuisine=MINUMAN&sortBy=1', r'\bcuisine=(\w+)\b')
with output
i have this kind of data
-
B-3-I11
B-3-I12
BI1-I190
BI1-I191
BI1-I192L
BI1-I194A
BI1-I195L
BI1-I198R
BI1-I199L
BI1-I200Ac
BI1-I201L
conasde
Installation
Madqw
Medsfg
Woasd
this is the data I have .. now I want only those which start from B and have some numeric character in data..how I get in qlikview script
how to extract only those data ..
To filter to those that start with B you'd do
where left(Field,1)=B
Then to filter on those with numbers, you could add
and len(keepchar(Field,'1234567890'))>0
So that would give something like this:
LOAD Field
From Table
Where left(Field,1)=B
AND len(keepchar(Field,'1234567890'))>0
(where Field is the name of the field your data is in and Table is the name of the table your data is in)
Or, if you want to keep all the data but create a new field you would do:
LOAD
Field,
if(left(Field,1)=B AND len(keepchar(Field,'1234567890'))>0`,Field) as FieldFiltered
From Table
I have tried each of the following formulas to get the highest number when I have a duplicate record. They both give me what appears to be the same output, but I know of at least one ID# where the response for both is "6" when I am expecting "7".
F2 = ID# to look for
PSStatus!$A = Hour; data ranges from 1 to 7; column is formatted as a number.
PSStatus!$F = ID#s
=INDEX(QUERY(PSStatus!$A$2:$A,,), MATCH(MAX($F2), (QUERY(PSStatus!$F$2:$F,,)),0))
=MAX(INDEX(QUERY(PSStatus!$A$2:$A,,), MATCH($F2, (QUERY(PSStatus!$F$2:$F,,)),0)))
I'll assume that your data table looks like the following one. Please, forgive me if I am mistaken.
If my assumption is correct, you can use the formula =MAX(FILTER({DATA TABLE RANGE}, {ID COLUMN FROM DATA TABLE}={ID})). That formula will first use FILTER to pick only the requested ID and then MAX will pick the highest one. In my example above, the formula should be =MAX(FILTER(Sheet1!$A$2:$B$26, Sheet1!$A$2:$A$26=A2)) for the first row. This is the end result:
Please, ask me anything if you need further help.
I have one table with rows and each row has a column that contains a field name (say raw1 - 'Number001', raw2-'ShortChar003', etc). In order for me to get that value of these fields I have to use a second table; this table has 1 raw with many columns (number001, Number002, ShortChar003, etc).
How can I extract the value?
Good Question..You can use lookup function
=Lookup(Fields!CityColumn.Value, Fields!CityColumn.Value, Fields!CountColumn.Value, "Dataset1")
Or you might have to use string functions..ex LEFT, Substring, Right same like SQL.If possible pls post some data of both tables, I will explain in detail