SQL Replace Commas in the a row of a table - sql

I have table test( ID Numeric(11,0), report varchar(255) )
and data looks below
1 ,Age,,,,,,family_status,,,,,,
2 ,,,,,,,,retaliation,hostile environment,,,,
3 ,,,,,,,,,,,,,
4 ,,,,,,,,retaliation,,,,,
5 ,,,,,,,,,hostile environment,,,,
6 ,Age,,,,,,,,,,,,
7 ,,,,national_origin,,,,,,,,,
8 Sex,,,,,,,,,,,,,
9 ,,,,national_origin,,disability,,retaliation,,,,,
10 Sex,,,,,,,,retaliation,,,,,
11 ,,,,,,,,
and i would like to update this table by replacing or using any other data scribing to remove extra commas so that data looks
1 Age,family_status
2 retaliation,hostile environment
3
4 retaliation
5 hostile environment
6 Age
7 national_origin,
8 Sex
9 national_origin,disability,retaliation
10 Sex,retaliation
11
i try to use the below statement but not sure how to loop through so that it will check and remove all the commas
UPDATE table test SET report = replace(report , ',,', ',')

If you are just doing this as a one off task (rather than a scripted process you expect to use repeatedly) you could always just run this query repeatedly until you get 0 rows updated
UPDATE table test SET report = replace(report , ',,', ',')
WHERE report like '%,,%'
If you need to do this over and over, or put it in a program I recommend using your procedural (non SQL code) to do the replace where you have better text manipulation commands.
If you aren't thrilled with that, check out this blog article I wrote on a similar problem of replacing repeating spaces from a string.

Related

Dynamically Generate file connection for several packages in SSIS

In a project we have several SSIS packages (around 200), all the package names are stored in a control table. We need to create a master package which can run all the 200 packages.
Since the max concurrent executable setting was set to 8. So planning to create 8 execute package tasks in a container and was thinking of generating the connection string(Execute package task- File connection String) dynamically using the package names stored in the table.
The control table is in the below format
Id PackageName
---------------
1 Package1
2 Package2
Ideas on how should be implemented helps.
I covered this pattern on https://stackoverflow.com/a/34868545/181965 but you're looking for a package that looks something like this
A sequence container that contains everything that one of those 8 discrete buckets of work would require. In your case, a Variable for
CurrentPackage String
rsObject Object
ContainerId Int32
The containerId will be the values 0 through 7 (since you have 8 buckets of work). As outlined in the other answer, we must scope the variables to the Sequence Container. The default in 2012+ is to create them at the Control Flow level, whereas 2005/2008 would create them at the level of the selected object.
Set up
I created a table and loaded it with 200 rows
CREATE TABLE dbo.so_35415549
(
id int IDENTITY(1,1) NOT NULL
, PackageName sysname
);
INSERT INTO
dbo.so_35415549
(
PackageName
)
SELECT TOP 200
'Package' + CAST(ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) AS varchar(3))
FROM
sys.all_columns AS AC;
Get My Bucket's data
The modulus, modulo, mod whatever you call it operator is our friend here. The mod operator will return the remainder after division. e.g. 10 mod 3 is 1 because 3*3 + 1 = 10
In your case, you'll be modding via 8 so you know the remainder will be bounded between 0 and 7.
SQL Server implements the mod operator as % and you can test the correctness via the following query
SELECT
S.id
, S.PackageName
, S.id % 8 AS ModValue
FROM
dbo.so_35415549 AS S
ORDER BY
1;
Sample output
id PackageName ModValue
1 Package1 1
2 Package2 2
3 Package3 3
4 Package4 4
5 Package5 5
6 Package6 6
7 Package7 7
8 Package8 0
9 Package9 1
10 Package10 2
...
199 Package199 7
200 Package200 0
SQL Get Work List
Using the above query as a template, we will use the following query. Notice the ? in there. That is the placeholder for an Execute SQL Tasks parameterization for an OLE DB Connection Manager.
SELECT
S.PackageName
FROM
dbo.so_35415549 AS S
WHERE
S.id % 8 = ?
ORDER BY
1;
The Parameter we pass in will be #[User::ContainerId]
The Result Set option will be updated from None to Full ResultSet and we push the value into rsObject
FELC Shred Work List
This is a standard shredding of a recordset. We got our variable populated in the previous step so let's enumerate through the results. There will be one column in our result set and you will map that to User::CurrentPackageName
EPT Run Package
This is your Execute Package Task. Use the value of CurrentPackageName and you're set.

Applying patterns to introduced text after Insert

select distinct SUBSTR(dni,4,2) as counter_dni
from persona
where SUBSTR(dni,4,2)<=10
order by counter_dni;
It returns:
1 - 1
2 - 10
3 - 2
4 - 3
The first number is just the row number, using Oracle SQL developer.
The problem is that I have multiple inserts like:
DNI1
DNI2
DNI3
until DNI 15 or so.
What i want to do is replace the format after DNI with a specific pattern like:
DNI0001
DNI0010
filling the gaps with 0 in a lenght of 4 digits for example. How can i do that after the rows were inserted?
Try
(select REGEXP_REPLACE(dni,'([0-9])+$', LPAD('\1',5,'0')) from counter_dni;
to check the output and
update counter_dni set dni = (REGEXP_REPLACE(dni,'([0-9])+$', LPAD('\1',5,'0')));
to update your values

Explode range of integers out for joining in SQL

I have one table that stores a range of integers in a field, sort of like a print range, (e.g. "1-2,4-7,9-11"). This field could also contain a single number.
My goal is to join this table to a second one that has discrete values instead of ranges.
So if table one contains
1-2,5
9-15
7
And table two contains
1
2
3
4
5
6
7
8
9
10
The result of the join would be
1-2,5 1
1-2,5 2
1-2,5 5
7 7
9-15 9
9-15 10
Working in SQL Server 2008 R2.
Use a string split function of your choice to split on comma. Figure out the min/max values and join using between.
SQL Fiddle
MS SQL Server 2012 Schema Setup:
create table T1(Col1 varchar(10))
create table T2(Col2 int)
insert into T1 values
('1-2,5'),
('9-15'),
('7')
insert into T2 values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10)
Query 1:
select T1.Col1,
T2.Col2
from T2
inner join (
select T1.Col1,
cast(left(S.Item, charindex('-', S.Item+'-')-1) as int) MinValue,
cast(stuff(S.Item, 1, charindex('-', S.Item), '') as int) MaxValue
from T1
cross apply dbo.Split(T1.Col1, ',') as S
) as T1
on T2.Col2 between T1.MinValue and T1.MaxValue
Results:
| COL1 | COL2 |
----------------
| 1-2,5 | 1 |
| 1-2,5 | 2 |
| 1-2,5 | 5 |
| 9-15 | 9 |
| 9-15 | 10 |
| 7 | 7 |
Like everybody has said, this is a pain to do natively in SQL Server. If you must then I think this is the proper approach.
First determine your rules for parsing the string, then break down the process into well-defined and understood problems.
Based on your example, I think this is the process:
Separate comma separated values in the string into rows
If the data does not contain a dash, then it's finished (it's a standalone value)
If it does contain a dash, parse the left and right sides of the dash
Given the left and right sides (the range) determine all the values between them into rows
I would create a temp table to populate the parsing results into which needs two columns:
SourceRowID INT, ContainedValue INT
and another to use for intermediate processing:
SourceRowID INT, ContainedValues VARCHAR
Parse your comma-separated values into their own rows using a CTE like this Step 1 is now a well-defined and understood problem to solve:
Turning a Comma Separated string into individual rows
So your result from the source
'1-2,5'
will be:
'1-2'
'5'
From there, SELECT from that processing table where the field does not contain a dash. Step 2 is now a well-defined and understood problem to solve These are standalone numbers and can go straight into the results temp table. The results table should also get the ID reference to the original row.
Next would be to parse the values to the left and right of the dash using CHARINDEX to locate it, then the appropriate LEFT and RIGHT functions as needed. This will give you the starting and ending value.
Here is a relevant question for accomplishing this step 3 is now a well-defined and understood problem to solve:
T-SQL substring - separating first and last name
Now you have separated the starting and ending values. Use another function which can explode this range. Step 4 is now a well-defined and understood problem to solve:
SQL: create sequential list of numbers from various starting points
SELECT all N between #min and #max
What is the best way to create and populate a numbers table?
and, also, insert it into the temp table.
Now what you should have is a temp table with every value in the exploded range.
Simply JOIN that to the other table on the values now, then to your source table on the ID reference and you're there.
My suggestion is to add one more field and many more records to your ranges table. Specifically, the primary key would be the integer and the other field would be the range. Records would look like this:
number range
1 1-2,5
2 1-2,5
3 na
4 na
5 1-2,5
etc
Having said that, this is still rather limiting because a number can only have one range. If you want to be thorough, set up a many to many relationship between numbers and ranges.
As far as I can tell you best option is something like below:
Create a table value function that accepts your ranges an converts them to a collection of ints. So 1-3,5 would return:
1
2
3
5
Then use these results to join to other tables. I don't have an exact function to do this at hand, but this one seems like an excellent start.

SQL Select where id is in `column`

I have a column that has multiple numbers separated by a comma. Example for a row:
`numbers`:
1,2,6,66,4,9
I want to make a query that will select the row only if the number 6 (for example) is in the column numbers.
I cant use LIKE because if there is 66 it'll work too.
You can use like. Concatenate the field separators at the beginning and end of the list and then use like. Here is the SQL Server sytnax:
where ','+numbers+',' like '%,'+'6'+',%'
SQL Server uses + for string concatenation. Other databases use || or the concat() function.
You should change your database to rather have a new table that joins numbers with the row of your current table. So if your row looks like this:
id numbers
1 1,2,6,66,4,9
You would have a new table that joins those values like so
row_id number
1 1
1 2
1 6
1 66
1 4
1 9
Then you can search for the number 6 in the number column and get the row_id

SQL Recursive Tables

I have the following tables, the groups table which contains hierarchically ordered groups and group_member which stores which groups a user belongs to.
groups
---------
id
parent_id
name
group_member
---------
id
group_id
user_id
ID PARENT_ID NAME
---------------------------
1 NULL Cerebra
2 1 CATS
3 2 CATS 2.0
4 1 Cerepedia
5 4 Cerepedia 2.0
6 1 CMS
ID GROUP_ID USER_ID
---------------------------
1 1 3
2 1 4
3 1 5
4 2 7
5 2 6
6 4 6
7 5 12
8 4 9
9 1 10
I want to retrieve the visible groups for a given user. That it is to say groups a user belongs to and children of these groups. For example, with the above data:
USER VISIBLE_GROUPS
9 4, 5
3 1,2,4,5,6
12 5
I am getting these values using recursion and several database queries. But I would like to know if it is possible to do this with a single SQL query to improve my app performance. I am using MySQL.
Two things come to mind:
1 - You can repeatedly outer-join the table to itself to recursively walk up your tree, as in:
SELECT *
FROM
MY_GROUPS MG1
,MY_GROUPS MG2
,MY_GROUPS MG3
,MY_GROUPS MG4
,MY_GROUPS MG5
,MY_GROUP_MEMBERS MGM
WHERE MG1.PARENT_ID = MG2.UNIQID (+)
AND MG1.UNIQID = MGM.GROUP_ID (+)
AND MG2.PARENT_ID = MG3.UNIQID (+)
AND MG3.PARENT_ID = MG4.UNIQID (+)
AND MG4.PARENT_ID = MG5.UNIQID (+)
AND MGM.USER_ID = 9
That's gonna give you results like this:
UNIQID PARENT_ID NAME UNIQID_1 PARENT_ID_1 NAME_1 UNIQID_2 PARENT_ID_2 NAME_2 UNIQID_3 PARENT_ID_3 NAME_3 UNIQID_4 PARENT_ID_4 NAME_4 UNIQID_5 GROUP_ID USER_ID
4 2 Cerepedia 2 1 CATS 1 null Cerebra null null null null null null 8 4 9
The limit here is that you must add a new join for each "level" you want to walk up the tree. If your tree has less than, say, 20 levels, then you could probably get away with it by creating a view that showed 20 levels from every user.
2 - The only other approach that I know of is to create a recursive database function, and call that from code. You'll still have some lookup overhead that way (i.e., your # of queries will still be equal to the # of levels you are walking on the tree), but overall it should be faster since it's all taking place within the database.
I'm not sure about MySql, but in Oracle, such a function would be similar to this one (you'll have to change the table and field names; I'm just copying something I did in the past):
CREATE OR REPLACE FUNCTION GoUpLevel(WO_ID INTEGER, UPLEVEL INTEGER) RETURN INTEGER
IS
BEGIN
DECLARE
iResult INTEGER;
iParent INTEGER;
BEGIN
IF UPLEVEL <= 0 THEN
iResult := WO_ID;
ELSE
SELECT PARENT_ID
INTO iParent
FROM WOTREE
WHERE ID = WO_ID;
iResult := GoUpLevel(iParent,UPLEVEL-1); --recursive
END;
RETURN iResult;
EXCEPTION WHEN NO_DATA_FOUND THEN
RETURN NULL;
END;
END GoUpLevel;
/
Joe Cleko's books "SQL for Smarties" and "Trees and Hierarchies in SQL for Smarties" describe methods that avoid recursion entirely, by using nested sets. That complicates the updating, but makes other queries (that would normally need recursion) comparatively straightforward. There are some examples in this article written by Joe back in 1996.
I don't think that this can be accomplished without using recursion. You can accomplish it with with a single stored procedure using mySQL, but recursion is not allowed in stored procedures by default. This article has information about how to enable recursion. I'm not certain about how much impact this would have on performance verses the multiple query approach. mySQL may do some optimization of stored procedures, but otherwise I would expect the performance to be similar.
Didn't know if you had a Users table, so I get the list via the User_ID's stored in the Group_Member table...
SELECT GroupUsers.User_ID,
(
SELECT
STUFF((SELECT ',' +
Cast(Group_ID As Varchar(10))
FROM Group_Member Member (nolock)
WHERE Member.User_ID=GroupUsers.User_ID
FOR XML PATH('')),1,1,'')
) As Groups
FROM (SELECT User_ID FROM Group_Member GROUP BY User_ID) GroupUsers
That returns:
User_ID Groups
3 1
4 1
5 1
6 2,4
7 2
9 4
10 1
12 5
Which seems right according to the data in your table. But doesn't match up with your expected value list (e.g. User 9 is only in one group in your table data but you show it in the results as belonging to two)
EDIT: Dang. Just noticed that you're using MySQL. My solution was for SQL Server. Sorry.
-- Kevin Fairchild
There was already similar question raised.
Here is my answer (a bit edited):
I am not sure I understand correctly your question, but this could work My take on trees in SQL.
Linked post described method of storing tree in database -- PostgreSQL in that case -- but the method is clear enough, so it can be adopted easily for any database.
With this method you can easy update all the nodes depend on modified node K with about N simple SELECTs queries where N is distance of K from root node.
Good Luck!
I don't remember which SO question I found the link under, but this article on sitepoint.com (second page) shows another way of storing hierarchical trees in a table that makes it easy to find all child nodes, or the path to the top, things like that. Good explanation with example code.
PS. Newish to StackOverflow, is the above ok as an answer, or should it really have been a comment on the question since it's just a pointer to a different solution (not exactly answering the question itself)?
There's no way to do this in the SQL standard, but you can usually find vendor-specific extensions, e.g., CONNECT BY in Oracle.
UPDATE: As the comments point out, this was added in SQL 99.