I have df1 which has three columns (loadgroup, cartons, blocks) like this
loadgroup
cartons
blocks
cartonsPercent
blocksPercent
1
2269
14
26%
21%
2
1168
13
13%
19%
3
937
8
11%
12%
4
2753
24
31%
35%
5
1686
9
19%
13%
total(sum of column)
8813
68
100%
100%
The interpretation is like this: out of df1 26% cartons which is also 21% of blocks are assigned to loadgroup 1, etc. we can assume blocks are 1 to 68, cartons are 1 to 8813.
I also have df2 which also has cartons and blocks columns. but does not have loadgroup.
My goal is to assign loadgroup (1-5 as well) to df2 (100 blocks 29608 cartons in total), but keep the proportions, for example, for df2, 26% cartons 21% blocks assign loadgroup 1, 13% cartons 19% blocks assign loadgroup 2, etc.
df2 is like this:
block
cartons
0
533
1
257
2
96
3
104
4
130
5
71
6
68
7
87
8
99
9
51
10
291
11
119
12
274
13
316
14
87
15
149
16
120
17
222
18
100
19
148
20
192
21
188
22
293
23
120
24
224
25
449
26
385
27
395
28
418
29
423
30
244
31
327
32
337
33
249
34
528
35
528
36
494
37
540
38
368
39
533
40
614
41
462
42
350
43
618
44
463
45
552
46
397
47
401
48
397
49
365
50
475
51
379
52
541
53
488
54
383
55
354
56
760
57
327
58
211
59
356
60
552
61
401
62
320
63
368
64
311
65
421
66
458
67
278
68
504
69
385
70
242
71
413
72
246
73
465
74
386
75
231
76
154
77
294
78
275
79
169
80
398
81
227
82
273
83
319
84
177
85
272
86
204
87
139
88
187
89
263
90
90
91
134
92
67
93
115
94
45
95
65
96
40
97
108
98
60
99
102
total 100 blocks
29608 cartons
I want to add loadgroup column to df2, try to keep those proportions as close as possible. How to do it please? Thank you very much for the help.
I don't know how to find loadgroup column based on both cartons percent and blocks percent. But generate random loadgroup based on either cartons percent or blocks percent is easy.
Here is what I did. I generate 100,000 seeds first, then for each seed, I add column loadgroup1 based on cartons percent, loadgroup2 based on blocks percent, then calculate both percentages, then compare with df1 percentages, get absolute difference, record it. For these 100,000 seeds, I take the minimum difference one as my solution, which is sufficient for my job.
But this is not the optimal solution, and I am looking for quick and easy way to do this. Hope somebody can help.
Here is my code.
df = pd.DataFrame()
np.random.seed(10000)
seeds = np.random.randint(1, 1000000, size = 100000)
for i in range(46530, 46537):
print(seeds[i])
np.random.seed(seeds[i])
df2['loadGroup1'] = np.random.choice(df1.loadgroup, len(df2), p = df1.CartonsPercent)
df2['loadGroup2'] = np.random.choice(df1.loadgroup, len(df2), p = df1.blocksPercent)
df2.reset_index(inplace = True)
three = pd.DataFrame(df2.groupby('loadGroup1').agg(Cartons = ('cartons', 'sum'), blocks = ('block', 'count')))
three['CartonsPercent'] = three.Cartons/three.Cartons.sum()
three['blocksPercent'] = three.blocks/three.blocks.sum()
four = df1[['CartonsPercent','blocksPercent']] - three[['CartonsPercent','blocksPercent']]
four = four.abs()
subdf = pd.DataFrame({'i':[i],'Seed':[seeds[i]], 'Percent':['CartonsPercent'], 'AbsDiff':[four.sum().sum()]})
df = pd.concat([df,subdf])
three = pd.DataFrame(df2.groupby('loadGroup2').agg(Cartons = ('cartons', 'sum'), blocks = ('block', 'count')))
three['CartonsPercent'] = three.Cartons/three.Cartons.sum()
three['blocksPercent'] = three.blocks/three.blocks.sum()
four = df1[['CartonsPercent','blocksPercent']] - three[['CartonsPercent','blocksPercent']]
four = four.abs()
subdf = pd.DataFrame({'i':[i],'Seed':[seeds[i]], 'Percent':['blocksPercent'], 'AbsDiff':[four.sum().sum()]})
df = pd.concat([df,subdf])
df.sort_values(by = 'AbsDiff', ascending = True, inplace = True)
df = df.head(10)
Actually the first row of df will tell me the seed I am looking for, I kept 10 rows just for curiosity.
Here is my solution.
block
cartons
loadgroup
0
533
4
1
257
1
2
96
4
3
104
4
4
130
4
5
71
2
6
68
1
7
87
4
8
99
4
9
51
4
10
291
4
11
119
2
12
274
2
13
316
4
14
87
4
15
149
5
16
120
3
17
222
2
18
100
2
19
148
2
20
192
3
21
188
4
22
293
1
23
120
2
24
224
4
25
449
1
26
385
5
27
395
3
28
418
1
29
423
4
30
244
5
31
327
1
32
337
5
33
249
4
34
528
1
35
528
1
36
494
5
37
540
3
38
368
2
39
533
4
40
614
5
41
462
4
42
350
5
43
618
4
44
463
2
45
552
1
46
397
3
47
401
3
48
397
1
49
365
1
50
475
4
51
379
1
52
541
1
53
488
2
54
383
2
55
354
1
56
760
5
57
327
4
58
211
2
59
356
5
60
552
4
61
401
1
62
320
1
63
368
3
64
311
3
65
421
2
66
458
5
67
278
4
68
504
5
69
385
4
70
242
4
71
413
1
72
246
2
73
465
5
74
386
4
75
231
1
76
154
4
77
294
4
78
275
1
79
169
4
80
398
4
81
227
4
82
273
1
83
319
3
84
177
4
85
272
5
86
204
3
87
139
1
88
187
4
89
263
4
90
90
4
91
134
4
92
67
3
93
115
3
94
45
2
95
65
2
96
40
4
97
108
2
98
60
2
99
102
1
Here are the summaries.
loadgroup
cartons
blocks
cartonsPercent
blocksPercent
1
7610
22
26%
22%
2
3912
18
13%
18%
3
3429
12
12%
12%
4
9269
35
31%
35%
5
5388
13
18%
13%
It's very close to my target though.
I have a TABLE_A, which contains a column with comma separated values as data. Now I have to put these comma separated values into TABLE B of 250 columns, this has to be done dynamically?
Here's a quick and dirty script to get you going
SQL> create table T ( c clob );
Table created.
SQL>
SQL> create table t1 ( c1 varchar2(50) );
Table created.
SQL> begin
2 for i in 2 .. 250 loop
3 execute immediate 'alter table t1 add c'||i||' varchar2(50)';
4 end loop;
5 end;
6 /
PL/SQL procedure successfully completed.
SQL>
SQL> declare
2 v clob := 'somedata1';
3 begin
4 for i in 2 .. 250 loop
5 v := v || ',somedata'||i;
6 end loop;
7 insert into t
8 select v from dual connect by level <= 2000;
9 end;
10 /
PL/SQL procedure successfully completed.
SQL>
SQL> set timing on
SQL> declare
2 type t_cols is table of varchar2(100) index by pls_integer;
3 l_cols t_cols;
4
5 l_comma pls_integer;
6 l_text varchar2(32767);
7
8 l_insert_sql varchar2(4000) := 'insert into t1 values (:1';
9 begin
10 for i in 2 .. 250 loop
11 l_insert_sql := l_insert_sql || ',:'||i;
12 end loop;
13 l_insert_sql := l_insert_sql || ')';
14
15 for i in ( select rownum r, c from t ) loop
16 l_text := i.c||',';
17
18 for x in 1 .. 250 loop
19 l_comma := instr(l_text,',');
20 l_cols(x) := substr(l_text,1,l_comma-1);
21 l_text := substr(l_text,l_comma+1);
22 end loop;
23
24 execute immediate l_insert_sql
25 using
26 l_cols(1)
27 ,l_cols(2)
28 ,l_cols(3)
29 ,l_cols(4)
30 ,l_cols(5)
31 ,l_cols(6)
32 ,l_cols(7)
33 ,l_cols(8)
34 ,l_cols(9)
35 ,l_cols(10)
36 ,l_cols(11)
37 ,l_cols(12)
38 ,l_cols(13)
39 ,l_cols(14)
40 ,l_cols(15)
41 ,l_cols(16)
42 ,l_cols(17)
43 ,l_cols(18)
44 ,l_cols(19)
45 ,l_cols(20)
46 ,l_cols(21)
47 ,l_cols(22)
48 ,l_cols(23)
49 ,l_cols(24)
50 ,l_cols(25)
51 ,l_cols(26)
52 ,l_cols(27)
53 ,l_cols(28)
54 ,l_cols(29)
55 ,l_cols(30)
56 ,l_cols(31)
57 ,l_cols(32)
58 ,l_cols(33)
59 ,l_cols(34)
60 ,l_cols(35)
61 ,l_cols(36)
62 ,l_cols(37)
63 ,l_cols(38)
64 ,l_cols(39)
65 ,l_cols(40)
66 ,l_cols(41)
67 ,l_cols(42)
68 ,l_cols(43)
69 ,l_cols(44)
70 ,l_cols(45)
71 ,l_cols(46)
72 ,l_cols(47)
73 ,l_cols(48)
74 ,l_cols(49)
75 ,l_cols(50)
76 ,l_cols(51)
77 ,l_cols(52)
78 ,l_cols(53)
79 ,l_cols(54)
80 ,l_cols(55)
81 ,l_cols(56)
82 ,l_cols(57)
83 ,l_cols(58)
84 ,l_cols(59)
85 ,l_cols(60)
86 ,l_cols(61)
87 ,l_cols(62)
88 ,l_cols(63)
89 ,l_cols(64)
90 ,l_cols(65)
91 ,l_cols(66)
92 ,l_cols(67)
93 ,l_cols(68)
94 ,l_cols(69)
95 ,l_cols(70)
96 ,l_cols(71)
97 ,l_cols(72)
98 ,l_cols(73)
99 ,l_cols(74)
100 ,l_cols(75)
101 ,l_cols(76)
102 ,l_cols(77)
103 ,l_cols(78)
104 ,l_cols(79)
105 ,l_cols(80)
106 ,l_cols(81)
107 ,l_cols(82)
108 ,l_cols(83)
109 ,l_cols(84)
110 ,l_cols(85)
111 ,l_cols(86)
112 ,l_cols(87)
113 ,l_cols(88)
114 ,l_cols(89)
115 ,l_cols(90)
116 ,l_cols(91)
117 ,l_cols(92)
118 ,l_cols(93)
119 ,l_cols(94)
120 ,l_cols(95)
121 ,l_cols(96)
122 ,l_cols(97)
123 ,l_cols(98)
124 ,l_cols(99)
125 ,l_cols(100)
126 ,l_cols(101)
127 ,l_cols(102)
128 ,l_cols(103)
129 ,l_cols(104)
130 ,l_cols(105)
131 ,l_cols(106)
132 ,l_cols(107)
133 ,l_cols(108)
134 ,l_cols(109)
135 ,l_cols(110)
136 ,l_cols(111)
137 ,l_cols(112)
138 ,l_cols(113)
139 ,l_cols(114)
140 ,l_cols(115)
141 ,l_cols(116)
142 ,l_cols(117)
143 ,l_cols(118)
144 ,l_cols(119)
145 ,l_cols(120)
146 ,l_cols(121)
147 ,l_cols(122)
148 ,l_cols(123)
149 ,l_cols(124)
150 ,l_cols(125)
151 ,l_cols(126)
152 ,l_cols(127)
153 ,l_cols(128)
154 ,l_cols(129)
155 ,l_cols(130)
156 ,l_cols(131)
157 ,l_cols(132)
158 ,l_cols(133)
159 ,l_cols(134)
160 ,l_cols(135)
161 ,l_cols(136)
162 ,l_cols(137)
163 ,l_cols(138)
164 ,l_cols(139)
165 ,l_cols(140)
166 ,l_cols(141)
167 ,l_cols(142)
168 ,l_cols(143)
169 ,l_cols(144)
170 ,l_cols(145)
171 ,l_cols(146)
172 ,l_cols(147)
173 ,l_cols(148)
174 ,l_cols(149)
175 ,l_cols(150)
176 ,l_cols(151)
177 ,l_cols(152)
178 ,l_cols(153)
179 ,l_cols(154)
180 ,l_cols(155)
181 ,l_cols(156)
182 ,l_cols(157)
183 ,l_cols(158)
184 ,l_cols(159)
185 ,l_cols(160)
186 ,l_cols(161)
187 ,l_cols(162)
188 ,l_cols(163)
189 ,l_cols(164)
190 ,l_cols(165)
191 ,l_cols(166)
192 ,l_cols(167)
193 ,l_cols(168)
194 ,l_cols(169)
195 ,l_cols(170)
196 ,l_cols(171)
197 ,l_cols(172)
198 ,l_cols(173)
199 ,l_cols(174)
200 ,l_cols(175)
201 ,l_cols(176)
202 ,l_cols(177)
203 ,l_cols(178)
204 ,l_cols(179)
205 ,l_cols(180)
206 ,l_cols(181)
207 ,l_cols(182)
208 ,l_cols(183)
209 ,l_cols(184)
210 ,l_cols(185)
211 ,l_cols(186)
212 ,l_cols(187)
213 ,l_cols(188)
214 ,l_cols(189)
215 ,l_cols(190)
216 ,l_cols(191)
217 ,l_cols(192)
218 ,l_cols(193)
219 ,l_cols(194)
220 ,l_cols(195)
221 ,l_cols(196)
222 ,l_cols(197)
223 ,l_cols(198)
224 ,l_cols(199)
225 ,l_cols(200)
226 ,l_cols(201)
227 ,l_cols(202)
228 ,l_cols(203)
229 ,l_cols(204)
230 ,l_cols(205)
231 ,l_cols(206)
232 ,l_cols(207)
233 ,l_cols(208)
234 ,l_cols(209)
235 ,l_cols(210)
236 ,l_cols(211)
237 ,l_cols(212)
238 ,l_cols(213)
239 ,l_cols(214)
240 ,l_cols(215)
241 ,l_cols(216)
242 ,l_cols(217)
243 ,l_cols(218)
244 ,l_cols(219)
245 ,l_cols(220)
246 ,l_cols(221)
247 ,l_cols(222)
248 ,l_cols(223)
249 ,l_cols(224)
250 ,l_cols(225)
251 ,l_cols(226)
252 ,l_cols(227)
253 ,l_cols(228)
254 ,l_cols(229)
255 ,l_cols(230)
256 ,l_cols(231)
257 ,l_cols(232)
258 ,l_cols(233)
259 ,l_cols(234)
260 ,l_cols(235)
261 ,l_cols(236)
262 ,l_cols(237)
263 ,l_cols(238)
264 ,l_cols(239)
265 ,l_cols(240)
266 ,l_cols(241)
267 ,l_cols(242)
268 ,l_cols(243)
269 ,l_cols(244)
270 ,l_cols(245)
271 ,l_cols(246)
272 ,l_cols(247)
273 ,l_cols(248)
274 ,l_cols(249)
275 ,l_cols(250);
276
277 end loop;
278
279 end;
280 /
PL/SQL procedure successfully completed.
Elapsed: 00:00:01.11
SQL>
I have 3 tables: Deliveries, IssuedWarehouse, ReturnedStock.
Deliveries: ID, OrderNumber, Material, Width, Gauge, DelKG
IssuedWarehouse: OrderNumber, IssuedKG
ReturnedStock: OrderNumber, IssuedKG
What I'd like to do is group all the orders by Material, Width and Gauge and then sum the amount delivered, issued to the warehouse and issued back to stock.
This is the SQL that is really quite close:
SELECT
DELIVERIES.Material,
DELIVERIES.Width,
DELIVERIES.Gauge,
Count(DELIVERIES.OrderNo) AS [Orders Placed],
Sum(DELIVERIES.DeldQtyKilos) AS [KG Delivered],
Sum(IssuedWarehouse.[Qty Issued]) AS [Film Issued],
Sum([Film Retns].[Qty Issued]) AS [Film Returned],
[KG Delivered]-[Film Issued]+[Film Returned] AS [Qty Remaining]
FROM (DELIVERIES
INNER JOIN IssuedWarehouse
ON DELIVERIES.OrderNo = IssuedWarehouse.[Order No From])
INNER JOIN [Film Retns]
ON DELIVERIES.OrderNo = [Film Retns].[Order No From]
GROUP BY Material, Width, Gauge, ActDelDate
HAVING ActDelDate Between [start date] And [end date]
ORDER BY DELIVERIES.Material;
This groups the products almost perfectly. However if you take a look at the results:
Material Width Gauge Orders Placed Delivered Qnty Kilos Film Issued Film Returned Qty Remaining
COEX-GLOSS 590 75 1 534 500 124 158
COEX-MATT 1080 80 1 4226 4226 52 52
CPP 660 38 8 6720 2768 1384 5336
CPP 666 47 1 5677 5716 536 497
CPP 690 65 2 1232 717 202 717
CPP 760 38 3 3444 1318 510 2636
CPP 770 38 4 4316 3318 2592 3590
CPP 786 38 2 672 442 212 442
CPP 800 47 1 1122 1122 116 116
CPP 810 47 1 1127 1134 69 62
CPP 810 47 2 2250 1285 320 1285
CPP 1460 38 12 6540 4704 2442 4278
LD 975 75 1 502 502 182 182
LDPE 450 50 1 252 252 50 50
LDPE 520 70 1 250 250 95 95
LDPE 570 65 2 504 295 86 295
LDPE 570 65 2 508 278 48 278
LDPE 620 50 1 252 252 67 67
LDPE 660 50 1 256 256 62 62
LDPE 670 75 1 248 248 80 80
LDPE 690 47 1 476 476 390 390
LDPE 790 38 2 2104 1122 140 1122
LDPE 790 50 1 286 286 134 134
LDPE 790 50 1 250 250 125 125
LDPE 810 30 1 4062 4062 100 100
LDPE 843 33 1 408 408 835 835
LDPE 850 80 1 412 412 34 34
LDPE 855 30 1 740 740 83 83
LDPE 880 60 1 304 304 130 130
LDPE 900 70 2 1000 650 500 850
LDPE 1017 60 1 1056 1056 174 174
OPP 25 1100 1 381 381 95 95
OPP 1000 30 2 1358 1112 300 546
OPP 1000 30 1 1492 1491 100 101
OPP 1200 20 1 418 417 461 462
PET 760 12 3 1227 1876 132 -517
You'll see that there are some materials that have the same width and gauge yet they are not grouped. I think this is because the delivered qty is different on the orders. For example:
Material Width Gauge Orders Placed Delivered Qnty Kilos Film Issued Film Returned Qty Remaining
LDPE 620 50 1 252 252 67 67
LDPE 660 50 1 256 256 62 62
I would like these two rows to be grouped. They have the same material, width and gauge but the delivered qty is different therefore it hasn't grouped it.
Can anyone help me group these strange rows?
Your "problem" is that the deliveries occurred on different dates, and you're grouping by ActDelDate so the data splits, but because you haven't selected the ActDelDate column, this isn't obvious.
The fix is: Remove ActDelDate from the group by list
You should also remove the unnecessary brackets around the first join, and change
HAVING ActDelDate Between [start date] And [end date]
to
WHERE ActDelDate Between [start date] And [end date]
and have it before the GROUP BY
You are grouping by the delivery date, which is causing the rows to be split. Either omit the delivery date from the results and group by, or take the min/max of the delivery date.