how to preserve column names on dynamic pivot - sql

Sales data contains dynamic product names which can contian any characters.
Dynamic pivot table is created based on sample from
Crosstab with a large or undefined number of categories
translate() is used to remove bad characters.
In result pivot table column names are corrupted: missing characters and spaces are removed.
How to return data with same column names as in source data ?
I tried to use
quote_ident(productname) as tootjakood,
instead of
'C'||upper(Translate(productname,'Ø. &/+-,%','O')) as tootjakood,
but it returns error
ERROR: column "Ø, 12.3/3mm" does not exist
testcase:
create temp table sales ( saledate date, productname char(20), quantity int );
insert into sales values ( '2016-1-1', 'Ø 12.3/3mm', 2);
insert into sales values ( '2016-1-1', '+-3,4%/3mm', 52);
insert into sales values ( '2016-1-3', '/3,2m-', 246);
do $do$
declare
voter_list text;
begin
create temp table myyk on commit drop as
select saledate as kuupaev,
'C'||upper(Translate(productname,'Ø. &/+-,%','O')) as tootjakood,
sum(quantity)::int as kogus
from sales
group by 1,2
;
drop table if exists pivot;
voter_list := (
select string_agg(distinct tootjakood, ' ' order by tootjakood) from myyk
);
execute(format('
create table pivot (
kuupaev date,
%1$s
)', (replace(voter_list, ' ', ' integer, ') || ' integer')
));
execute (format($f$
insert into pivot
select
kuupaev,
%2$s
from crosstab($ct$
select
kuupaev,tootjakood,kogus
from myyk
order by 1
$ct$,$ct$
select distinct tootjakood
from myyk
order by 1
$ct$
) as (
kuupaev date,
%4$s
);$f$,
replace(voter_list, ' ', ' + '),
replace(voter_list, ' ', ', '),
'',
replace(voter_list, ' ', ' integer, ') || ' integer' -- 4.
));
end; $do$;
select * from pivot;
Postgres 9.1 is used.

You should use double quotes. Because you are using spaces to identify column separators, you should remove spaces from column names (or change the way of separators identification).
With
...
select saledate as kuupaev,
format ('"%s"', replace (upper(productname), ' ', '')) as tootjakood,
sum(quantity)::int as kogus
from sales
...
you'll get:
kuupaev | /3,2M- | +-3,4%/3MM | O12.3/3MM
------------+--------+------------+-----------
2016-01-01 | | 52 | 2
2016-01-03 | 246 | |
(2 rows)

Related

Storing and Query Blank Values in Hive Columns

I have a requirement for storing blank strings of length 1, 2, and 3 in some columns of my Hive table.
Storing:
If my column type is char, then I see that the data is always trimmed before storing. i.e. length(column) is always 0
If my column type is varchar then the data is not trimmed. so length(column) is 1, 2 and 3 respectively.
So that solves my storing problem.
Querying:
I am unable to query the column by value.
say. select * from hive table where column = ' ';
it only works if I do something like
select * from hive table where length(column) > 0 and trim(column) = '';
Is there a way to handle this separately ?
say I want to query those records where column value is of a blank string of length 3? How do I do this?
This is what i Tried (Note that the issues seems to be when the file is stored as parquet)
CREATE EXTERNAL TABLE IF NOT EXISTS DUMMY5 (
col1 varchar(3))
STORED AS PARQUET
LOCATION "/DUMMY5";
insert into DUMMY5 values (' '); // 2 character strings
insert into DUMMY5 values (' '); //3 character strings
select col1, length(col1) from DUMMY5;
+-------+------+--+
| col1 | _c1 |
+-------+------+--+
| | 3 |
| | 2 |
+-------+------+--+
select col1, length(col1) from DUMMY5 where col1 = ' '; // 0 record
select col1, length(col1) from DUMMY5 where col1 = ' '; // 0 record
Running Hive 2.1.1
drop table dummy_tbl;
CREATE TABLE dummy_tbl (
col1 char(1),
col2 varchar(1),
col3 char(3),
col4 varchar(3)) ;
insert into dummy_tbl values (' ', ' ', ' ', ' ');
select length(col1), length(col2), length(col3), length(col4) from dummy_tbl;
Result:
c0 c1 c2 c3
0 1 0 2
Varchar column works absolutely correct. col2 was trimmed on insert, it is documented.
col4 varchar(2) works correctly, this query returns 1:
select count(*) from dummy_tbl where col4=' '; --returns 1
And length of all char columns shows 0 and comparison ignoring spaces like it is documented:
select count(*) from dummy_tbl where col1=' '; --single space --returns 1
select count(*) from dummy_tbl where col1=' '; --two spaces --also returns 1 because it is ignoring spaces
You can use varchar with proper length. Or STRING type if you not sure about length.

SQL - How to use Comma separated column values in a where clause

I have a table called Configuration. It contains the values like below,
Id SourceColumns TargetColumns SourceTable TargetTable
1 Name, Age CName, CAge STable TTable
2 EId EmplId EmpTable TTable
In a stored procedure, I have to get the column names from the above table and I have to compare the source table and target table.
I am able to do that easily for the 2nd record as it has only one column name, so in the where clause I can write sourcecolumn = targetcolumn like,
SELECT
EId
, EmplId
FROM
EmpTable E
JOIN TTable T ON E.Eid = T.EmplId
The first record in the table has 2 columns separated by comma (,).
I have to compare like this,
SELECT
Name
, Age
FROM
STable S
JOIN TTable T ON S.Name = T.CName AND S.Age = T.CAge
In some cases the source columns and target columns may have more column names separated by comma(,)
Please help me on this.
As I don't know whether you have completely understood the data model I suggested in the request comments and in order to properly answer the question:
Your table is not normalized, as the data in the columns SourceColumns and TargetColumns is not atomic. And one even has to interpret the data (the separator is the comma and the nth element in one column relates to the nth element in the other column).
This is how your tables should look like instead (the create statements are pseudo code):
create table configuration_tables
(
id_configuration_tables int,
source_table text,
target_table text,
primary key (id_configuration_tables),
unique key (source_table),
unique key (target_table) -- or not? in your sample two souce table map to the same target table
);
create table configuration_columns
(
id_configuration_columns int,
id_configuration_tables int,
source_column text,
target_column text,
primary key (id_configuration_columns),
foreign key (id_configuration_tables) references configuration_tables (id_configuration_tables)
);
Your sample data would then become
configuration_tables
id_configuration_tables | source_table | target_table
------------------------+--------------+-------------
1 | STable | TTable
2 | EmpTable | TTable
configuration_columns
id_configuration_columns | id_configuration_tables | source_column | target_column
-------------------------+-------------------------+---------------+--------------
1 | 1 | Name | CName
2 | 1 | Age | CAge
3 | 2 | EId | EmplId
As of SQL Server 2017 you can use STRING_AGG to create your queries is. In earlier versions this was also possible with some STRING_AGG emulation you will easily find wit Google or SO.
select
'select s.' + string_agg (c.source_column + ', t.' + c.target_column, ', ') +
' from ' + t.source_table + ' s' +
' join ' + t.target_table + ' t' +
' on ' + string_agg('t.' + c.target_column + ' = s.' + c.source_column, ' and ') +
';' as query
from configuration_tables t
join configuration_columns c on c.id_configuration_tables = t.id_configuration_tables
group by t.source_table, t.target_table
order by t.source_table, t.target_table;
Demo: https://dbfiddle.uk/?rdbms=sqlserver_2019&fiddle=8866b2485ba9bba92c2391c67bb8cae0

Sql Procedure to count words of a given string [duplicate]

I'm trying to count how many words there are in a string in SQL.
Select ("Hello To Oracle") from dual;
I want to show the number of words. In the given example it would be 3 words though there could be more than one space between words.
You can use something similar to this. This gets the length of the string, then substracts the length of the string with the spaces removed. By then adding the number one to that should give you the number of words:
Select length(yourCol) - length(replace(yourcol, ' ', '')) + 1 NumbofWords
from yourtable
See SQL Fiddle with Demo
If you use the following data:
CREATE TABLE yourtable
(yourCol varchar2(15))
;
INSERT ALL
INTO yourtable (yourCol)
VALUES ('Hello To Oracle')
INTO yourtable (yourCol)
VALUES ('oneword')
INTO yourtable (yourCol)
VALUES ('two words')
SELECT * FROM dual
;
And the query:
Select yourcol,
length(yourCol) - length(replace(yourcol, ' ', '')) + 1 NumbofWords
from yourtable
The result is:
| YOURCOL | NUMBOFWORDS |
---------------------------------
| Hello To Oracle | 3 |
| oneword | 1 |
| two words | 2 |
Since you're using Oracle 11g it's even simpler-
select regexp_count(your_column, '[^ ]+') from your_table
Here is a sqlfiddle demo
If your requirement is to remove multiple spaces too, try this:
Select length('500 text Oracle Parkway Redwood Shores CA') - length(REGEXP_REPLACE('500 text Oracle Parkway Redwood Shores CA',
'( ){1,}', '')) NumbofWords
from dual;
Since I have used the dual table you can test this directly in your own development environment.
DECLARE #List NVARCHAR(MAX) = ' ab a
x'; /*Your column/Param*/
DECLARE #Delimiter NVARCHAR(255) = ' ';/*space*/
DECLARE #WordsTable TABLE (Data VARCHAR(1000));
/*convert by XML the string to table*/
INSERT INTO #WordsTable(Data)
SELECT Data = y.i.value('(./text())[1]', 'VARCHAR(1000)')
FROM
(
SELECT x = CONVERT(XML, '<i>'
+ REPLACE(#List, #Delimiter, '</i><i>')
+ '</i>').query('.')
) AS a CROSS APPLY x.nodes('i') AS y(i)
/*Your total words*/
select count(*) NumberOfWords
from #WordsTable
where Data is not null;
/*words list*/
select *
from #WordsTable
where Data is not null
/from this Logic you can continue alon/

Combining duplicate records in SQL Server

I have a table in SQL Server 2012 that holds a list of parts, location of the parts and the quantity on hand. The problem I have is someone put a space in front of the location when they added it to the database. This allowed there to be two records.
I need to create a job that will find the parts with spaces before the location and add those parts to the identical parts without spaces in front of the location. I'm not quite sure where to even start with this.
This is the before:
Partno | PartRev | Location | OnHand | Identity_Column
--------------------------------------------------------------------
0D6591D 000 MV3 55.000 103939
0D6591D 000 MV3 -55.000 104618
This is what I would like to have after the job ran:
Partno | PartRev | Location | OnHand | Identity_Column
--------------------------------------------------------------------
0D6591D 000 MV3 0 104618
Two steps: 1. update the records with the correct locations, 2. delete the records with the wrong locations.
update mytable
set onhand = onhand +
(
select coalesce(sum(wrong.onhand), 0)
from mytable wrong
where wrong.location like ' %'
and trim(wrong.location) = mytable.location
)
where location not like ' %';
delete from mytable where location like ' %';
You can do some grouping with a HAVING clause on to identify the records. I've used REPLACE to replace spaces with empty strings in the location column, you could also use LTRIM and RTRIM:
CREATE TABLE #Sample
(
[Partno] VARCHAR(7) ,
[PartRev] INT ,
[Location] VARCHAR(5) ,
[OnHand] INT ,
[Identity_Column] INT
);
INSERT INTO #Sample
([Partno], [PartRev], [Location], [OnHand], [Identity_Column])
VALUES
('0D6591D', 000, ' MV3', 55.000, 103939),
('0D6591D', 000, 'MV3', -55.000, 104618)
;
SELECT Partno ,
PartRev ,
REPLACE( Location, ' ', '') Location,
SUM(OnHand) [OnHand]
FROM #Sample
GROUP BY REPLACE(Location, ' ', '') ,
Partno ,
PartRev
HAVING COUNT(Identity_Column) > 1;
DROP TABLE #Sample;
Produces:
Partno PartRev Location OnHand
0D6591D 0 MV3 0

How can I count the number of words in a string in Oracle?

I'm trying to count how many words there are in a string in SQL.
Select ("Hello To Oracle") from dual;
I want to show the number of words. In the given example it would be 3 words though there could be more than one space between words.
You can use something similar to this. This gets the length of the string, then substracts the length of the string with the spaces removed. By then adding the number one to that should give you the number of words:
Select length(yourCol) - length(replace(yourcol, ' ', '')) + 1 NumbofWords
from yourtable
See SQL Fiddle with Demo
If you use the following data:
CREATE TABLE yourtable
(yourCol varchar2(15))
;
INSERT ALL
INTO yourtable (yourCol)
VALUES ('Hello To Oracle')
INTO yourtable (yourCol)
VALUES ('oneword')
INTO yourtable (yourCol)
VALUES ('two words')
SELECT * FROM dual
;
And the query:
Select yourcol,
length(yourCol) - length(replace(yourcol, ' ', '')) + 1 NumbofWords
from yourtable
The result is:
| YOURCOL | NUMBOFWORDS |
---------------------------------
| Hello To Oracle | 3 |
| oneword | 1 |
| two words | 2 |
Since you're using Oracle 11g it's even simpler-
select regexp_count(your_column, '[^ ]+') from your_table
Here is a sqlfiddle demo
If your requirement is to remove multiple spaces too, try this:
Select length('500 text Oracle Parkway Redwood Shores CA') - length(REGEXP_REPLACE('500 text Oracle Parkway Redwood Shores CA',
'( ){1,}', '')) NumbofWords
from dual;
Since I have used the dual table you can test this directly in your own development environment.
DECLARE #List NVARCHAR(MAX) = ' ab a
x'; /*Your column/Param*/
DECLARE #Delimiter NVARCHAR(255) = ' ';/*space*/
DECLARE #WordsTable TABLE (Data VARCHAR(1000));
/*convert by XML the string to table*/
INSERT INTO #WordsTable(Data)
SELECT Data = y.i.value('(./text())[1]', 'VARCHAR(1000)')
FROM
(
SELECT x = CONVERT(XML, '<i>'
+ REPLACE(#List, #Delimiter, '</i><i>')
+ '</i>').query('.')
) AS a CROSS APPLY x.nodes('i') AS y(i)
/*Your total words*/
select count(*) NumberOfWords
from #WordsTable
where Data is not null;
/*words list*/
select *
from #WordsTable
where Data is not null
/from this Logic you can continue alon/