If I have data in a table with integers like the example below, is it possible to calculate for each row the sum of several columns and output that sum as well as several other columns through an sqlite query command?
My table looks like this below
|Timestamp |Email |Name |Year|Make |Model |Car_ID|Judge_ID|Judge_Name|Racer_Turbo|Racer_Supercharged|Racer_Performance|Racer_Horsepower|Car_Overall|Engine_Modifications|Engine_Performance|Engine_Chrome|Engine_Detailing|Engine_Cleanliness|Body_Frame_Undercarriage|Body_Frame_Suspension|Body_Frame_Chrome|Body_Frame_Detailing|Body_Frame_Cleanliness|Mods_Paint|Mods_Body|Mods_Wrap|Mods_Rims|Mods_Interior|Mods_Other|Mods_ICE|Mods_Aftermarket|Mods_WIP|Mods_Overall|
|--------------|---------------------------|----------|----|--------|---------|------|--------|----------|-----------|------------------|-----------------|----------------|-----------|--------------------|------------------|-------------|----------------|------------------|------------------------|---------------------|-----------------|--------------------|----------------------|----------|---------|---------|---------|-------------|----------|--------|----------------|--------|------------|
|8/5/2018 14:10|honoland13#japanpost.jp |Hernando |2015|Acura |TLX |48 |J04 |Bob |0 |0 |2 |2 |4 |4 |0 |2 |4 |4 |2 |4 |2 |2 |2 |2 |2 |0 |4 |4 |4 |6 |2 |0 |4 |
|8/5/2018 15:11|nlighterness2q#umn.edu |Noel |2015|Jeep |Wrangler |124 |J02 |Carl |0 |6 |4 |2 |4 |6 |6 |4 |4 |4 |6 |6 |6 |6 |6 |4 |6 |6 |6 |6 |6 |4 |6 |4 |6 |
|8/5/2018 17:10|eguest47#microsoft.com |Edan |2015|Lexus |Is250 |222 |J05 |Adrian |0 |0 |0 |0 |0 |0 |0 |0 |6 |6 |6 |0 |0 |6 |6 |6 |0 |0 |0 |0 |0 |0 |0 |0 |4 |
|8/5/2018 17:34|hchilley40#fema.gov |Hieronymus|1993|Honda |Civic eG |207 |J06 |Aaron |0 |0 |2 |2 |2 |2 |2 |2 |0 |4 |2 |2 |2 |2 |2 |2 |4 |2 |2 |0 |0 |0 |2 |2 |0 |
|8/5/2018 14:30|nnowick3d#tuttocitta.it |Nickolas |2016|Ford |Mystang |167 |J02 |Carl |0 |0 |2 |2 |0 |2 |2 |0 |0 |0 |0 |2 |0 |2 |2 |2 |0 |0 |2 |0 |0 |0 |0 |0 |2 |
|8/5/2018 16:12|mdearl39#amazon.co.uk |Martin |2013|Hyundai |Gen coupe|159 |J04 |Bob |0 |0 |2 |0 |0 |0 |2 |0 |0 |0 |0 |2 |0 |2 |2 |0 |2 |0 |2 |0 |0 |0 |0 |0 |0 |
How can I find the sum from column 10 to 34 for each row, then output each row up to column 7 followed by a column with the total for each row? So far I've only figured out how to get the sum for each column individually but not to across several columns for each row and to output each the desired columns.
SELECT Car_ID, Year, Make, Model, SUM(Mods_ICE) FROM Carstable
But this only outputs data for one row at the bottom of the table with the sum. Expected outcome would be something like below
|Car_ID|Year |Make |Model |Total |
|------|------|------|---------|-------|
|48 |2015 |Acura |TLX |89 |
|22 |2015 |Chevy |Camaro |101 |
|19 |2006 |Ford |Mustang |55 |
|101 |2011 |Subaru|WRX |91 |
For sum of columns in a single row you need no extra function like SUM. Use + oerator:
SELECT column10 + ... + column34 FROM Carstable
I have this dataframe -
data = [(0,1,1,201505,3),
(1,1,1,201506,5),
(2,1,1,201507,7),
(3,1,1,201508,2),
(4,2,2,201750,3),
(5,2,2,201751,0),
(6,2,2,201752,1),
(7,2,2,201753,1)
]
cols = ['id','item','store','week','sales']
data_df = spark.createDataFrame(data=data,schema=cols)
display(data_df)
What I want it this -
data_new = [(0,1,1,201505,3,0),
(1,1,1,201506,5,0),
(2,1,1,201507,7,0),
(3,1,1,201508,2,0),
(4,1,1,201509,0,0),
(5,1,1,201510,0,0),
(6,1,1,201511,0,0),
(7,1,1,201512,0,0),
(8,2,2,201750,3,0),
(9,2,2,201751,0,0),
(10,2,2,201752,1,0),
(11,2,2,201753,1,0),
(12,2,2,201801,0,0),
(13,2,2,201802,0,0),
(14,2,2,201803,0,0),
(15,2,2,201804,0,0)]
cols_new = ['id','item','store','week','sales','flag',]
data_df_new = spark.createDataFrame(data=data_new,schema=cols_new)
display(data_df_new)
So basically, I want 8 (this can also be 6 or 10) weeks of data for each item-store groupby combination. Wherever the 52/53 weeks for the year ends, I need the weeks for the next year, as I have mentioned in the sample. I need this in PySpark, thanks in advance!
See my attempt below. Could have made it shorter but felt should be as explicit as I can so I dint chain the soultions. code below
from pyspark.sql import functions as F
spark.sql("set spark.sql.legacy.timeParserPolicy=LEGACY")
# Convert week of the year to date
s=data_df.withColumn("week", expr("cast (week as string)")).withColumn('date', F.to_date(F.concat("week",F.lit("6")), "yyyywwu"))
s = (s.groupby('item', 'store').agg(F.collect_list('sales').alias('sales'),F.collect_list('date').alias('date'))#Put sales and dates in an array
.withColumn("id", sequence(lit(0), lit(6)))#Create sequence ids with the required expansion range per group
)
#Explode datframe back with each item/store combination in a row
s =s.selectExpr('item','store','inline(arrays_zip(date,id,sales))')
#Create partition window broadcasting from start to end for each item/store combination
w = Window.partitionBy('item','store').orderBy('id').rowsBetween(-sys.maxsize, sys.maxsize)
#Create partition window broadcasting from start to end for each item/store/date combination. the purpose here is to aggregate over null dates as group
w1 = Window.partitionBy('item','store','date').orderBy('id').rowsBetween(Window.unboundedPreceding, Window.currentRow)
s=(s.withColumn('increment', F.when(col('date').isNull(),(row_number().over(w1))*7).otherwise(0))#Create increment values per item/store combination
.withColumn('date1', F.when(col('date').isNull(),max('date').over(w)).otherwise(col('date')))#get last date in each item/store combination
)
# #Compute the week of year and drop columns not wanted
s = s.withColumn("weekofyear", expr("weekofyear(date_add(date1, cast(increment as int)))")).drop('date','increment','date1').na.fill(0)
s.show(truncate=False)
Outcome
+----+-----+---+-----+----------+
|item|store|id |sales|weekofyear|
+----+-----+---+-----+----------+
|1 |1 |0 |3 |5 |
|1 |1 |1 |5 |6 |
|1 |1 |2 |7 |7 |
|1 |1 |3 |2 |8 |
|1 |1 |4 |0 |9 |
|1 |1 |5 |0 |10 |
|1 |1 |6 |0 |11 |
|2 |2 |0 |3 |50 |
|2 |2 |1 |0 |51 |
|2 |2 |2 |1 |52 |
|2 |2 |3 |1 |1 |
|2 |2 |4 |0 |2 |
|2 |2 |5 |0 |3 |
|2 |2 |6 |0 |4 |
+----+-----+---+-----+----------+
Current Data:
|LotNumber |ItemNumber |ItemName
|76 |1 |WidgetRight
|76 |1 |WidgetLeft
|76 |2 |WidgetRight
|76 |2 |WidgetLeft
|76 |7 |WidgetRight
|76 |7 |WidgetLeft
|76 |9 |WidgetRight
|76 |10 |WidgetRight
|76 |10 |WidgetLeft
|80 |3 |WidgetRight
|80 |3 |WidgetLeft
|80 |7 |WidgetRight
|80 |7 |WidgetLeft
|80 |11 |WidgetRight
|80 |31 |WidgetRight
Need to add a column that alternates between 1 and 2 every time the ItemNumber changes
|LotNumber |ItemNumber |ItemName |NewField
|76 |1 |WidgetRight |1
|76 |1 |WidgetLeft |1
|76 |2 |WidgetRight |2
|76 |2 |WidgetLeft |2
|76 |7 |WidgetRight |1
|76 |7 |WidgetLeft |1
|76 |9 |WidgetRight |2
|76 |10 |WidgetRight |1
|76 |10 |WidgetLeft |1
|80 |3 |WidgetRight |2
|80 |3 |WidgetLeft |2
|80 |7 |WidgetRight |1
|80 |7 |WidgetLeft |1
|80 |11 |WidgetRight |2
|80 |31 |WidgetRight |1
Current Code not working ...
DECLARE #LastLine Varchar(1)
SELECT ItemNumber, ItemName,
CASE WHEN ItemNumber <> LAG (ItemNumber, 1, 0) OVER (PARTITION BY LotNumber ORDER BY LotNumber, ItemNumber )
THEN
CASE WHEN #LastLine = '1'
THEN '2'
ELSE '1'
END
END AS LastLine
Thank you for any help.
You can just use dense_rank() and arithmetics:
select t.*,
1 + (dense_rank() over(order by lotnumber, itemnumber) + 1) % 2 as newfield
from mytable t
Demo on DB Fiddle
How can I find the SUM of all scores for the minimum date of each lesson_id please:
-----------------------------------------------------------
|id |uid |group_id |lesson_id |game_id |score |date |
-----------------------------------------------------------
|1 |145 |1 |1 |0 |40 |1391627323 |
|2 |145 |1 |1 |0 |80 |1391627567 |
|3 |145 |1 |2 |0 |40 |1391627323 |
|4 |145 |1 |3 |0 |30 |1391627323 |
|5 |145 |1 |3 |0 |90 |1391627567 |
|6 |145 |1 |4 |0 |20 |1391628000 |
|7 |145 |1 |5 |0 |35 |1391628000 |
-----------------------------------------------------------
I need output:
-------------------
|sum_first_scores |
-------------------
|165 |
-------------------
I have this so far, which lists the score for each minimum date, per lesson, but I need to sum those results as above:
SELECT lesson_id, MIN(date), score AS first_score FROM cdu_user_progress
WHERE cdu_user_progress.uid = 145
GROUP BY lesson_id
You can identify the first score as the one where no earlier record exists. Then just take the sum:
select sum(score)
from edu_user_progress eup
where cdu_user_progress.uid = 145 and
not exists (select 1
from edu_user_progress eup2
where eup2.uid = eup.uid and
eup2.lesson_id = eup.lesson_id and
eup2.date < eup.date
);
This assumes that the minimum date for the lesson id has only one score.
I have the following data in sql.
Date |Customer|DespatchOrder|Value|ApprovedCredit
08/10|Axe |AGB-01 |50 |200
08/12|Axe |AGB-02 |75 |200
08/13|Axe |AGB-03 |80 |200
08/15|Axe |AGB-04 |10 |200
08/09|Vex |VSE-01 |50 |150
08/10|Vex |VSE-02 |60 |150
08/13|Vex |VSE-03 |40 |150
With some manipulation in sql I have managed to display my data in a crosstab in crystal reports like in the examples below:
AXE
Date |ApprovedCredit|Total|AGB-01|AGB-02|AGB-03|AGB-04
08/09|200 |0 |0 |0 |0 |0
08/10|200 |50 |50 |0 |0 |0
08/11|200 |50 |50 |0 |0 |0
08/12|200 |125 |50 |75 |0 |0
08/13|200 |205 |50 |75 |80 |0
08/14|200 |205 |50 |75 |80 |0
08/15|200 |215 |50 |75 |80 |10
VEX
Date |ApprovedCredit|Total|VSE-01|VSE-02|VSE-03
08/09|120 |50 |50 |0 |0
08/10|120 |110 |50 |60 |0
08/11|120 |110 |50 |60 |0
08/12|120 |110 |50 |60 |0
08/13|120 |150 |50 |60 |40
08/14|120 |150 |50 |60 |40
08/15|120 |150 |50 |60 |40
The problem that I have, I cannot find a way to calculate my overlimit column inside of a crosstab, the over limit is calculated by approvedcredit - Total(Column inside of crosstab)
AXE
Date |ApprovedCredit|Total|OverLimit|AGB-01|AGB-02|AGB-03|AGB-04
08/09|200 |0 |0 |0 |0 |0 |0
08/10|200 |50 |0 |50 |0 |0 |0
08/11|200 |50 |0 |50 |0 |0 |0
08/12|200 |125 |0 |50 |75 |0 |0
08/13|200 |205 |5 |50 |75 |80 |0
08/14|200 |205 |5 |50 |75 |80 |0
08/15|200 |215 |15 |50 |75 |80 |10
VEX
Date |ApprovedCredit|Total|OverLimit|VSE-01|VSE-02|VSE-03
08/09|120 |50 |0 |50 |0 |0
08/10|120 |110 |0 |50 |60 |0
08/11|120 |110 |0 |50 |60 |0
08/12|120 |110 |0 |50 |60 |0
08/13|120 |150 |30 |50 |60 |40
08/14|120 |150 |30 |50 |60 |40
08/15|120 |150 |30 |50 |60 |40
Please help if you have any idea how to do this, I have tried to create my report without a crosstab, but was not successful with the dynamic column headers. I would like to keep on using the crosstab due to the dynamic nature of this report and sometime there can be more than 10 DespatchOrders but if there is no way to do this using a cross tab please help with alternative.
I found away to create the over limit column inside of Crystal Report using a Calculated Member with a value formula
ApprovedCredit - GridValueAt(CurrentRowIndex, 0 ,0)
The GridValueAt function is now supported by crystal reports xi neither is a calculated member. Just another brick wall
If you can calculate your overlimit value upstream, the crosstab will be easy. If you can't do that, you'll need to base your crosstab columns and summarized fields value on a formula instead of actual fields from your datasource. In your formula, you return the field value you want to appear in most of the columns, and calculate the value to display in the overlimit column.