I am importing data from a flat file to a SQL table called TCVS_tmp_PO_Detail using SSIS and SQL. Now and then I get some characters in the Purchase Order column that I would like to find and eliminate if they occur.
The characters are , so could I trim these 3 characters on the left of the number out? It does not occur all the time so I can run it on the front end of my SSIS job as a query to correct it before exporting it.
Here is an example of what it looks like when it occurs
Purchase Order
7
7
8
8
8
8
8
8
9
10
10
10
10
11
12
13
11
12
13
14
14
15
15
16
16
17
19
18
19
20
22
I'm not sure where the data lies for the processing, but you can easily do this with a case expression:
(case when purchase_order like '%' then stuff(purchase_order, 1, 3, '')
else purchase_order
end) as purchase_order
I suppose you use "Data Flow Task"
Add "Derived Column" component
Add new column
Add this code to the Expression
REPLACE(Purchase,"","")
The results:
Related
I have one table that I need to get some metrics from.
For example I have the following table:
meas_count
skippings
links
extra
10
8
4.2
some
10
9
5.8
some
10
9
5.8
some_2
11
8
4.2
some
11
8
5.8
some
11
9
5.9
some
I need to get a view of an existing table in the following form for further work:
meas_count
skippings
links_min
links_max
10
8
0
4
10
8
4
5
10
8
5
6
10
9
0
4
10
9
4
5
10
9
5
6
11
8
0
4
11
8
4
5
11
8
5
6
11
9
0
4
11
9
4
5
11
9
5
6
At the moment I have 2 queries, the results of which I need to combine to get the result I need.
First request:
SELECT meas_count,skippings FROM current_stats GROUP BY meas_count,skippings
Creates the following:
meas_count
skippings
10
8
10
9
11
8
11
9
Second request:
SELECT
LAG(rounded) OVER (ORDER BY rounded) as links_min,
rounded as links_max FROM
(SELECT * FROM
(SELECT ROUND(links, 1) as rounded FROM current_stats)
GROUP BY rounded ORDER BY rounded)
Creates the following:
links_min
links_max
NULL
4
4
5
5
6
I need something like result of sets multiplication...
What query should be executed to get the table of the view I need as a result?
I also have an additional question: is the execution of the second query slowed down due to several SELECTs inside?
You can do that by doing an INNER JOIN on the two tables without specifying a join condition. That will give you every combination of the two sets of rows.
SELECT * FROM
(
SELECT meas_count,skippings
FROM current_stats
GROUP BY meas_count,skippings)
AS one
INNER JOIN
(
SELECT LAG(rounded) OVER (ORDER BY rounded) as links_min,
rounded as links_max FROM
(SELECT * FROM
(SELECT ROUND(links, 1) as rounded FROM current_stats)
GROUP BY rounded
ORDER BY rounded
)
) AS two;
As for performance, that's really only an issue if there is a better way to do it. Of course nested SELECTs take time, but the query optimizers in today's SQL engine are pretty good at determining what you MEANT from what you SAID.
I have a table with two columns:
sort_column = A column I use for sorting
value_column = My metric of interest (a positive integer)
Using SQL, I need to create contiguous groups of rows, ordered by sort_column, such that the sum of value_column within each group is the largest possible but staying below 100 (100 not included).
Find below an example of my desired result.
Thanks
sort_column
value_column
desired_result
1
53
1
2
25
1
3
33
2
4
25
2
5
10
2
6
46
3
7
9
3
8
49
4
9
48
4
10
53
5
11
33
5
12
52
6
13
29
6
14
16
6
15
66
7
16
1
7
17
62
8
18
57
9
19
47
10
20
12
10
Ok, so after a few lengthy attempts, I came to the conclusion the task is impossible with pure SQL, because a given value of the desired column depends on previous values of that same column, in a way that cannot be obtained from the first two columns alone, so the problem is impossible to tackle without using a recursive CTE, which BigQuery does not support.
I solved the issue by writing a javascript UDF for the task. It seems to be working fine and produces the expected results.
Many thanks everyone!
I want to sort a string column which can include both numbers and alphabets.
SQL Script:
select distinct a.UoA, b.rating , b.tot from omt_source a left join
wlm_progress_Scored b
on a.UoA = b.UoA
where a.UoA in (select UoA from UserAccess_dev
where trim(App_User) = lower(:APP_USER))
order by
regexp_substr(UoA, '^\D*') ,
to_number(regexp_substr(UoA, '\d+'))--);
Output I'm currently getting:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
23
26B
26A
27
28
30
31
32
33
34B
34A
But, I want 26 and 34 to be in this order
26A
26B
34A
34B
Any suggestion will be much helpful
Thanks
If your first order by clause ensures that the primary sort order is based on the numerical component of the UoA field, then your second order clause could actually be just the UoA field itself. I.e.
order by
regexp_substr(UoA, '^\D*'), UoA;
I need to create a stored procedure in SQL Server that accepts the following two parameters:
A select statement returning 1 column.
A number of columns.
The stored procedure would then run the select statement and return the result of the select statement with the values of the single column split into the given amount of columns per row.
Here are some examples:
exec stored_proc ‘select id from table where id between 1 and 20’, 5
The result of the select would be:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
The result of the stored procedure call would be:
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
16 17 18 19 20
Or the call could be:
exec stored_proc ‘select id from table where id between 1 and 20’, 10
Giving the result of:
1 2 3 4 5 6 7 8 9 10
11 12 13 14 15 16 17 18 19 20
Though I'm not sure you should be doing this in SQL, it can be done.
I think the way to do it would be do create a cursor and use it's iterations to build a dynamic SQL statement.
During each iteration, add each piece of data as a new column (field) and when you reach the number of columns add something like Union Select
I have built an access application for a manufacturing plant and have provided them with a report that lists different data points along a process. I have a way to generate a report that looks like the following.
Batch Zone Value1 Value 2 etc.
25 1 5 15
25 2 12 31
26 1 6 14
26 2 10 32
However, there is demand to view the data in a different format. They would like one line per batch, with all data horizontal. Like this...
Zone 1 Zone 2
Batch Value1 Value2 Value1 Value2
25 5 15 12 31
26 6 14 10 32
In all there will be 157 columns, if displayed as in the second example. There are 7 unique field names, but the rest are 14 different data types that are repeated. I can't get a query to display the data in the format the they want, do to the fact that the field names are the same, but it is not hard to do it the first way. I can use VBA to insert the data into a table, but I can't use duplicate field names, so when I go to export this to Excel the field names won't mean anything, and there can't be sections (like zone1, zone2, etc.) I can link a report to this, but the report width can only be 22", so I would have to export and then do some vba handling of the excel sheet on the other end to display in a legible way.
I can get the data into format #1, is there some way I can get the data to display in one long row based on batch number? Does anyone else have a great idea of how this is doable?
Open to any suggestions. Thanks!
In your question you say that
I have a way to generate a report that looks like the following
and then list the data as
Batch Zone Value1 Value2
----- ---- ------ ------
25 1 5 15
25 2 12 31
26 1 6 14
26 2 10 32
Now perhaps the data may already be in "un-pivoted" form somewhere (with different Values in separate rows), but if not then you would use something like the following query to achieve that
SELECT
[Batch],
"Zone" & [Zone] & "_" & "Value1" AS [ValueID],
[Value1] AS [ValueValue]
FROM BatchDataByZone
UNION ALL
SELECT
[Batch],
"Zone" & [Zone] & "_" & "Value2" AS [ValueID],
[Value2] AS [ValueValue]
FROM BatchDataByZone
...returning:
Batch ValueID ValueValue
----- ------------ ----------
25 Zone1_Value1 5
25 Zone2_Value1 12
26 Zone1_Value1 6
26 Zone2_Value1 10
25 Zone1_Value2 15
25 Zone2_Value2 31
26 Zone1_Value2 14
26 Zone2_Value2 32
However you get to that point, if you save that query as [BatchDataUnpivoted] then you could use a simple Crosstab Query to "string out" the values for each batch...
TRANSFORM Sum(BatchDataUnpivoted.[ValueValue]) AS SumOfValueValue
SELECT BatchDataUnpivoted.[Batch]
FROM BatchDataUnpivoted
GROUP BY BatchDataUnpivoted.[Batch]
PIVOT BatchDataUnpivoted.[ValueID];
...returning...
Batch Zone1_Value1 Zone1_Value2 Zone2_Value1 Zone2_Value2
----- ------------ ------------ ------------ ------------
25 5 15 12 31
26 6 14 10 32