how to identify first record in a column pysaprk - dataframe

I have a dataframe with many columns
So I have to identigy first record of a column and assign it one value and for others assign another value
i.e
if df[price].first_record = df[amt]
else
df[price] = df[amt]+df[delivery_charges]
how do I identify the first record in a column/dataframe


You can do this in following way:
window = Window.orderBy('Id')
df.withColumn('row',f.row_number().over(window)).withColumn('price',f.when(f.col('row')==1,f.col('amt')).otherwise(f.col('amt')+f.col('delivery_charges'))).show()

Related

Python : How to create a new boolean column in my dataframe if the value of another column is in a list

I have a dataframe and I want to create a new column which take the value 1 if the value of an other column is in a list and 0 else. I try this but it did not work. Thank you

is there a way to give column names to pd.read_clipboard() as it is treating first row of data as column names

I am using pd.read_clipboard() function to get an excel table that doesnt have column names as first row . The dataframe returned has first row as column labels. How to fix that.
I would like results to be
and not this

Though not showing up on help for read_clipboard() function , passing read_clipboard(names=['c1','c2']) where c1 and c2 are the column names fixes the read_clipboard() function to not treat first row as column names i.e provide column names to avoid having the function treat first row as column names

Reference another cell in a different row having calculated the minimum value in a datatable column

Reference another cell in the same row having calculated the minimum value in a datatable column
The above answered my question as to how to reference another cell in the same row having calculated the minimum value in a datatable column. But how do I reference a cell in another row still using that original cell?
In other words, if the min value in a column is in row 5, I now know how to get the values of cells in all other columns within row 5, but how do I get the values from columns within other rows (eg row 6) using the original cell as a starting point?
Dim Data() As DataRow = datatable.Select("sourcecolumn = " & test.ToString())
Dim column2 = Data(0)(1)
was the answer to my first question, how would I amend this to get to another row? I tried:
dim column2 = Data(1)(1)
but it returns an array error. I think the numbers in brackets are absolute, but I need to use relative ones.
Thanks in advance again.

VLOOKUP to return multiple matches

I want to ask if there's a way/formula/vba to return multiple values when using vlookup? For example, I vlookup a data and when that data has multiple values to return, it will return the other values. Thanks.

For something as generic as this, just use Google.
Step #1) www.google.com
Step #2) get your answer in less time than it takes you to post here.
Return MULTIPLE corresponding values for ONE Lookup Value
The Excel VLOOKUP Function searches for a value (ie. Lookup_value) in the first column of a table array and returns a value in the same row from another column in the table array. In case of multiple occurrences of the Lookup value, the function searches the first occurrence of the Lookup value, and returns the corresponding value in the same row from another column.
In case you want to return multiple corresponding values, for the one Lookup value which has multiple occurrences, we show how it can be done using INDEX, SMALL, IF & ROW excel functions, as follows.
Consider the table array ("A2:B8"), in which you want to lookup the value "Apples" in column A which has multiple occurrences, and return all corresponding values in column B.
Enter the lookup value "Apples" in cell A11. In cell B11, enter below formula, as an array formula (CTRL+SHIFT+ENTER), and copy it downward in the same column B, in 7 rows (ie. number of times as the number of records in the table array "A2:B8". Multiple corresponding values (of the lookup value "Apples") will get copied vertically, starting from cell B11 till B17. Refer Table 1.
=INDEX($B$2:$B$8, SMALL(IF($A$11=$A$2:$A$8, ROW($A$2:$A$8)-ROW($A$2)+1), ROW(1:1)))
http://www.globaliconnect.com/excel/index.php?option=com_content&view=article&id=119:vlookup-multiple-values-return-multiple-corresponding-values-for-one-lookup-value&catid=77&Itemid=473

vb.net check the value of each column for only the 1st row datatable

I want to check the first row only of a data table. I want to verify that the right name is in the right column. The first row has header name and they have to be in the right order. for example I want to check the first row and the first column to check that it has "First Name"
If I do the following it give me the right column data but it gives me the 2nd row not the first. I don't get it since I am asking for row 0 and col 0.
Dim ExcelData As DataTable
field = ExcelData.Rows(0)(0).ToString()
(actually I will want to do a IF to check this column. first wanted to see what value is there before I code it all )