How to deduplicate in Presto SQL without with only varchar data - sql
My question is related to this one. However, I have only varchar data so I can't use the solutions there. My data looks like this:
id | activity | type
------------------------
al12 | a1a4 | MOVE
la23 | 2a5e | WAIT
la23 | 2a5e | WAIT
ie42 | 35a8 | STAY
The third row is a duplicate. How can I remove it?
Use DISTINCT?
SELECT DISTINCT id, activity, type
FROM your_table
https://prestodb.github.io/docs/current/sql/select.html
Related
PostgreSQL query to select records which a specific value doesn't include in text array
I have a table like this
| id | data |
|---------------|---------------------|
| org:abc:basic | {org,org:abc:basic} |
| org:xyz:basic | {org,basic} |
| org:efg:basic | {org} |
I need to write a query to select all the rows which doesn't have the id inside the data column.
Or at least I need to query all the records which doesn't have a text starting from org: and ending with :basic within data.
Currently for this I try to run
SELECT * FROM t_permission WHERE 'org:%:basic' NOT LIKE ANY (data)
query which returns everything even the first row.
you can use the <> operator with ALL against the array: select * from the_table where id <> all(data);
HIVE SQL: Select rows whose values contain string in a column
I want to select rows whose values contain a string in a column. For example, I want to select all rows whose values contain a string '123' in the column 'app'. table: app id 123helper xdas 323helper fafd 2123helper dsaa 3123helper fafd md5321 asdx md5123 dsad result: app id 123helper xdas 2123helper dsaa 3123helper fafd md5123 dsad I am not familiar with SQL query. Could anyone help me? . Thanks in advances.
In a number of ways:
like:
select * from table
where app like '%123%'
rlike:
...
where app rlike '123'
instr:
...
where instr(app, '123')>0
locate:
...
where locate('123', app)>0
Invent your own way.
Read manual: String Functions and Operators.
Try the following using like select * from yourTable where app like '%123%' Output: | app | id | | ---------- | ---- | | 123helper | xdas | | 2123helper | dsaa | | 3123helper | fafd | | md5123 | dsad |
Please use below query, select app, id from table where app like '%123%'; Below are few additional information, like '123%' --> Starts with 123 like '%123' --> Ends with 123 like '%123%'--> Contains 123 anywhere in the string
SQL query to get latest user to update record
I have a postgres database that contains an audit log table which holds a historical log of updates to documents. It contains which document was updated, which field was updated, which user made the change, and when the change was made. Some sample data looks like this: doc_id | user_id | created_date | field | old_value | new_value --------+---------+------------------------+-------------+---------------+------------ A | 1 | 2018-07-30 15:43:44-05 | Title | | War and Piece A | 2 | 2018-07-30 15:45:13-05 | Title | War and Piece | War and Peas A | 1 | 2018-07-30 16:05:59-05 | Title | War and Peas | War and Peace B | 1 | 2018-07-30 15:43:44-05 | Description | test 1 | test 2 B | 2 | 2018-07-30 17:45:44-05 | Description | test 2 | test 3 You can see that the Title of document A was changed three times, first by user 1 then by user 2, then again by user 1. Basically I need to know which user was the last one to update a field on a particular document. So for example, I need to know that User 1 was the last user to update the Title field on document A. I don't really care what time it happened, just the document, field, and user. So sample output would be something like this: doc_id | field | user_id --------+-------------+--------- A | Title | 1 B | Description | 2 Seems like it should be fairly straightforward query to write but I'm having some trouble with it. I would think that group by would be in order but the problem is that if I group by doc_id I lose the user data: select doc_id, max(created_date) from document_history group by doc_id; doc_id | max --------+------------------------ B | 2018-07-30 15:00:00-05 A | 2018-07-30 16:00:00-05 I could join these results table back to the document_history table but I would need to do so based on the doc_id and timestamp which doesn't seem quite right. If two people editing a document at the exact same time I would get multiple rows back for that document and field. Maybe that's so unlikely I shouldn't worry about it, but still... Any thoughts on a way to do this in a single query?
You want to filter the records, so think where, not group by: select dh.* from document_history where dh.created_date = (select max(dh2.created_date) from document_history dh2 where dh2.doc_id = dh.doc_id); In most databases, this will have better performance than a group by, if you have an index on document_history(doc_id, created_date).
If your DBMS supports window functions (e.g. PostgreSQL, SQL Server; aka analytic function in Oracle) you could do something like this (SQLFiddle with Postgres, other systems might differ slightly in the syntax): http://sqlfiddle.com/#!17/981af/4 SELECT DISTINCT doc_id, field, first_value(user_id) OVER (PARTITION BY doc_id, field ORDER BY created_date DESC) as last_user FROM get_last_updated first_value() OVER (... ORDER BY x DESC) orders the window frames/partitions descending and then takes the first value which is your latest time stamp. I added the DISTINCT to get your expected result. The window function just adds a new column to your SELECT result but within the same partition with the same value. If you do not need it, remove it and then you are able to work with the origin data plus the new won information.
Recursive self join over file data
I know there are many questions about recursive self joins, but they're mostly in a hierarchical data structure as follows: ID | Value | Parent id ----------------------------- But I was wondering if there was a way to do this in a specific case that I have where I don't necessarily have a parent id. My data will look like this when I initially load the file. ID | Line | ------------------------- 1 | 3,Formula,1,2,3,4,... 2 | *,record,abc,efg,hij,... 3 | ,,1,x,y,z,... 4 | ,,2,q,r,s,... 5 | 3,Formula,5,6,7,8,... 6 | *,record,lmn,opq,rst,... 7 | ,,1,t,u,v,... 8 | ,,2,l,m,n,... Essentially, its a CSV file where each row in the table is a line in the file. Lines 1 and 5 identify an object header and lines 3, 4, 7, and 8 identify the rows belonging to the object. The object header lines can have only 40 attributes which is why the object is broken up across multiple sections in the CSV file. What I'd like to do is take the table, separate out the record # column, and join it with itself multiple times so it achieves something like this: ID | Line | ------------------------- 1 | 3,Formula,1,2,3,4,5,6,7,8,... 2 | *,record,abc,efg,hij,lmn,opq,rst 3 | ,,1,x,y,z,t,u,v,... 4 | ,,2,q,r,s,l,m,n,... I know its probably possible, I'm just not sure where to start. My initial idea was to create a view that separates out the first and second columns in a view, and use the view as a way of joining in a repeated fashion on those two columns. However, I have some problems: I don't know how many sections will occur in the file for the same object The file can contain other objects as well so joining on the first two columns would be problematic if you have something like ID | Line | ------------------------- 1 | 3,Formula,1,2,3,4,... 2 | *,record,abc,efg,hij,... 3 | ,,1,x,y,z,... 4 | ,,2,q,r,s,... 5 | 3,Formula,5,6,7,8,... 6 | *,record,lmn,opq,rst,... 7 | ,,1,t,u,v,... 8 | ,,2,l,m,n,... 9 | ,4,Data,1,2,3,4,... 10 | *,record,lmn,opq,rst,... 11 | ,,1,t,u,v,... In the above case, my plan could join rows from the Data object in row 9 with the first rows of the Formula object by matching the record value of 1. UPDATE I know this is somewhat confusing. I tried doing this with C# a while back, but I had to basically write a recursive decent parser to parse the specific file format and it simply took to long because I had to get it in the database afterwards and it was too much for entity framework. It was taking hours just to convert one file since these files are excessively large. Either way, #Nolan Shang has the closest result to what I want. The only difference is this (sorry for the bad formatting): +----+------------+------------------------------------------+-----------------------+ | ID | header | x | value | +----+------------+------------------------------------------+-----------------------+ | 1 | 3,Formula, | ,1,2,3,4,5,6,7,8 |3,Formula,1,2,3,4,5,6,7,8 | | 2 | ,, | ,1,x,y,z,t,u,v | ,1,x,y,z,t,u,v | | 3 | ,, | ,2,q,r,s,l,m,n | ,2,q,r,s,l,m,n | | 4 | *,record, | ,abc,efg,hij,lmn,opq,rst |*,record,abc,efg,hij,lmn,opq,rst | | 5 | ,4, | ,Data,1,2,3,4 |,4,Data,1,2,3,4 | | 6 | *,record, | ,lmn,opq,rst | ,lmn,opq,rst | | 7 | ,, | ,1,t,u,v | ,1,t,u,v | +----+------------+------------------------------------------+-----------------------------------------------+
I agree that it would be better to export this to a scripting language and do it there. This will be a lot of work in TSQL. You've intimated that there are other possible scenarios you haven't shown, so I obviously can't give a comprehensive solution. I'm guessing this isn't something you need to do quickly on a repeated basis. More of a one-time transformation, so performance isn't an issue. One approach would be to do a LEFT JOIN to a hard-coded table of the possible identifying sub-strings like: 3,Formula, *,record, ,,1, ,,2, ,4,Data, Looks like it pretty much has to be human-selected and hard-coded because I can't find a reliable pattern that can be used to SELECT only these sub-strings. Then you SELECT from this artificially-created table (or derived table, or CTE) and LEFT JOIN to your actual table with a LIKE to get all the rows that use each of these values as their starting substring, strip out the starting characters to get the rest of the string, and use the STUFF..FOR XML trick to build the desired Line. How you get the ID column depends on what you want, for instance in your second example, I don't know what ID you want for the ,4,Data,... line. Do you want 5 because that's the next number in the results, or do you want 9 because that's the ID of the first occurrance of that sub-string? Code accordingly. If you want 5 it's a ROW_NUMBER(). If you want 9, you can add an ID column to the artificial table you created at the start of this approach. BTW, there's really nothing recursive about what you need done, so if you're still thinking in those terms, now would be a good time to stop. This is more of a "Group Concatenation" problem.
Here is a sample, but has some different with you need.
It is because I use the value the second comma as group header, so the ,,1 and ,,2 will be treated as same group, if you can use a parent id to indicated a group will be better
DECLARE #testdata TABLE(ID int,Line varchar(8000))
INSERT INTO #testdata
SELECT 1,'3,Formula,1,2,3,4,...' UNION ALL
SELECT 2,'*,record,abc,efg,hij,...' UNION ALL
SELECT 3,',,1,x,y,z,...' UNION ALL
SELECT 4,',,2,q,r,s,...' UNION ALL
SELECT 5,'3,Formula,5,6,7,8,...' UNION ALL
SELECT 6,'*,record,lmn,opq,rst,...' UNION ALL
SELECT 7,',,1,t,u,v,...' UNION ALL
SELECT 8,',,2,l,m,n,...' UNION ALL
SELECT 9,',4,Data,1,2,3,4,...' UNION ALL
SELECT 10,'*,record,lmn,opq,rst,...' UNION ALL
SELECT 11,',,1,t,u,v,...'
;WITH t AS(
SELECT *,REPLACE(SUBSTRING(t.Line,LEN(c.header)+1,LEN(t.Line)),',...','') AS data
FROM #testdata AS t
CROSS APPLY(VALUES(LEFT(t.Line,CHARINDEX(',',t.Line, CHARINDEX(',',t.Line)+1 )))) c(header)
)
SELECT MIN(ID) AS ID,t.header,c.x,t.header+STUFF(c.x,1,1,'') AS value
FROM t
OUTER APPLY(SELECT ','+tb.data FROM t AS tb WHERE tb.header=t.header FOR XML PATH('') ) c(x)
GROUP BY t.header,c.x
+----+------------+------------------------------------------+-----------------------------------------------+
| ID | header | x | value |
+----+------------+------------------------------------------+-----------------------------------------------+
| 1 | 3,Formula, | ,1,2,3,4,5,6,7,8 | 3,Formula,1,2,3,4,5,6,7,8 |
| 3 | ,, | ,1,x,y,z,2,q,r,s,1,t,u,v,2,l,m,n,1,t,u,v | ,,1,x,y,z,2,q,r,s,1,t,u,v,2,l,m,n,1,t,u,v |
| 2 | *,record, | ,abc,efg,hij,lmn,opq,rst,lmn,opq,rst | *,record,abc,efg,hij,lmn,opq,rst,lmn,opq,rst |
| 9 | ,4, | ,Data,1,2,3,4 | ,4,Data,1,2,3,4 |
+----+------------+------------------------------------------+-----------------------------------------------+
How to get numbers arranged right to left in sql server SELECT statements
When performing SELECT statements including number columns (prices, for example), the result always is left to right ordered, which reduces the readability. Therefore I'm searching a method to format the output of number columns right to left.
I already tried to use something like
SELECT ... SPACE(15-LEN(A.Nummer))+A.Nummer ...
FROM Artikel AS A ...
which gives close results, but depending on font not really. An alternative would be to replace 'SPACE()' with 'REPLICATE('_',...)', but I don't really like the underscores in output.
Beside that this formula will crash on numbers with more digits than 15, therefore I searched for a way finding the maximum length of entries to make it more save like
SELECT ... SPACE(MAX(A.Nummer)-LEN(A.Nummer))+A.Nummer ...
FROM Artikel AS A ...
but this does not work due to the aggregate character of the MAX-function.
So, what's the best way to achieve the right-justified order for the number-columns?
Thanks,
Rainer
To get you problem with the list box solved have a look at this link: http://www.lebans.com/List_Combo.htm I strongly believe that this type of adjustment should be made in the UI layer and not mixed in with data retrieval. But to answer your original question i have created a SQL Fiddle: MS SQL Server 2008 Schema Setup: CREATE TABLE dbo.some_numbers(n INT); Create some example data: INSERT INTO dbo.some_numbers SELECT CHECKSUM(NEWID()) FROM (VALUES (1),(1),(1),(1),(1),(1),(1),(1),(1),(1))X(x); The following query is using the OVER() clause to specify that the MAX() is to be applied over all rows. The > and < that the result is wrapped in is just for illustration purposes and not required for the solution. Query 1: SELECT '>'+ SPACE(MAX(LEN(CAST(n AS VARCHAR(MAX))))OVER()-LEN(CAST(n AS VARCHAR(MAX))))+ CAST(n AS VARCHAR(MAX))+ '<' FROM dbo.some_numbers SN; Results: | COLUMN_0 | |---------------| | >-1486993739< | | > 1620287540< | | >-1451542215< | | >-1257364471< | | > -819471559< | | >-1364318127< | | >-1190313739< | | > 1682890896< | | >-1050938840< | | > 484064148< | This query does a straight case to show the difference: Query 2: SELECT '>'+CAST(n AS VARCHAR(MAX))+'<' FROM dbo.some_numbers SN; Results: | COLUMN_0 | |---------------| | >-1486993739< | | >1620287540< | | >-1451542215< | | >-1257364471< | | >-819471559< | | >-1364318127< | | >-1190313739< | | >1682890896< | | >-1050938840< | | >484064148< | With this query you still need to change the display font to a monospaced font like COURIER NEW. Otherwise, as you have noticed, the result is still misaligned.