Drop null rows from a column that has a dynamic name - apache-spark-sql

How do I drop null rows from a column name that's dynamic?
It's dynamic because the only column in this table is from a UDF and it's a temporary view.
CREATE TEMPORARY VIEW temp_table
AS
SELECT udf1(input)
FROM input_table;
So the temporary view will look something like
|UDF:udf1(some input)|
|--------------------|
| 1 |
| 2 |
| null |
--------------------
I want to drop all null rows, so the temporary view looks like
|UDF:udf1(some input)|
|--------------------|
| 1 |
| 2 |
--------------------
I've been trying something like
DELETE FROM temp_table
WHERE ??? IS NULL;
but it seems like the column name isn't static...and I've been looking at INFORMATION_SCHEMA.COLUMNS but my sql isn't recognizing such a view. I also looked at trying to do something like drop from table where 'column that starts with UDF' is null but apparently that's not allowed either

Related

How to expand each row in a tableto each distinct value in another table

So I have a a very basic table with one column with several values in it. I have another table (we can call it tableabc the structure is irrelevant). For each row in tableabc I want it exploded to add a column with each unique Value in TableValue. So for example:
TableValue
| Value |
| ------ |
| aaa |
| bbb |
| ccc |
tableabc
| c_1 | c_2 |c_3|
|-|-|-|
|1|random|doesnt
|2|data|matter
OutputTable
| c_1 | c_2 |c_3|Value|
|-|-|-|-|
|1|random|doesnt|aaa|
|1|random|doesnt|bbb|
|1|random|doesnt|ccc|
|2|data|matter|aaa|
|2|data|matter|bbb|
|2|data|matter|ccc|
What type of action would this be? Obviously not a join, I tried look at "explode" but that seems to be on a column level. Not sure what this type of relation would be and what that would look like in SQL.
As simple as below
select *
from tableabc
cross join TableValue
If applied to sample data in your question - output is

PostgreSQL add new not null column and fill with ids from insert statement

I´ve got 2 tables.
CREATE TABLE content (
id bigserial NOT NULL,
name text
);
CREATE TABLE data (
id bigserial NOT NULL,
...
);
The tables are already filled with a lot of data.
Now I want to add a new column content_id (NOT NULL) to the data table.
It should be a foreign key to the content table.
Is it possible to automatically create an entry in the content table to set a content_id in the data table.
For example
**content**
| id | name |
| 1 | abc |
| 2 | cde |
data
| id |... |
| 1 |... |
| 2 |... |
| 3 |... |
Now I need an update statement that creates 3 (in this example) content entries and add the ids to the data table to get this result:
content
| id | name |
| 1 | abc |
| 2 | cde |
| 3 | ... |
| 4 | ... |
| 5 | ... |
data
| id |... | content_id |
| 1 |... | 3 |
| 2 |... | 4 |
| 3 |... | 5 |
demo:db<>fiddle
According to the answers presented here: How can I add a column that doesn't allow nulls in a Postgresql database?, there are several ways of adding a new NOT NULL column and fill this directly.
Basicly there are 3 steps. Choose the best fitting (with or without transaction, setting a default value first and remove after, leave the NOT NULL contraint first and add afterwards, ...)
Step 1: Adding new column (without NOT NULL constraint, because the values of the new column values are not available at this point)
ALTER TABLE data ADD COLUMN content_id integer;
Step 2: Inserting the data into both tables in a row:
WITH inserted AS ( -- 1
INSERT INTO content
SELECT
generate_series(
(SELECT MAX(id) + 1 FROM content),
(SELECT MAX(id) FROM content) + (SELECT COUNT(*) FROM data)
),
'dummy text'
RETURNING id
), matched AS ( -- 2
SELECT
d.id AS data_id,
i.id AS content_id
FROM (
SELECT
id,
row_number() OVER ()
FROM data
) d
JOIN (
SELECT
id,
row_number() OVER ()
FROM inserted
) i ON i.row_number = d.row_number
) -- 3
UPDATE data d
SET content_id = s.content_id
FROM (
SELECT * FROM matched
) s
WHERE d.id = s.data_id;
Executing several statements one after another by using the results of the previous one can be achieved using WITH clauses (CTEs):
Insert data into content table: This generates an integer series starting at the MAX() + 1 value of the current content's id values and has as many records as the data table. Afterwards the new ids are returned
Now we need to match the current records of the data table with the new ids. So for both sides, we use row_number() window function to generate a consecutive row count for each records. Because both, the insert result and the actual data table have the same number of records, this can be used as join criterion. So we can match the id column of the data table with the new content's id values
This matched data can used in the final update of the new content_id column
Step 3: Add the NOT NULL constraint
ALTER TABLE data ALTER COLUMN content_id SET NOT NULL;

SQL query to format table data for DataSource in GridView

I am looking for a SQL Server query that could transfer source SQL table data:
TextID | Text | LanguageID
-------|-------|-------------------------------------
app.aa | Hi | 6a13ea09-46ea-4c93-9b6a-e26bdc6ff4d8
app.cc | Hund | 0c894bb7-4937-4903-906a-d1b1dd64935c
app.aa | Hallo | 0c894bb7-4937-4903-906a-d1b1dd64935c
app.cc | Dog | 6a13ea09-46ea-4c93-9b6a-e26bdc6ff4d8
app.bb | Star | 6a13ea09-46ea-4c93-9b6a-e26bdc6ff4d8
...
into table like this one:
TextID | Original | Translated
-------|----------|-----------
app.aa | Hi | Hallo
app.bb | Star | -
app.cc | Dog | Hund
...
so that I can use it as a DataSource for GridView in ASP .NET. Thank you in advance for your help.
Whenever you need to combine data from two different rows into one, you need to join. For example:
select src.TextID "TextID", src.Text "Original", tr.Text "Translated"
from source_table src
left join source_table tr
on src.TextID = tr.TextID
and src.LangID = 'xxx' -- xxx is the source language id
and tr.LangID = 'yyy' -- yyy is the target language id
The left join ensures that untranslated words are included with a null translated value. To make a table for your DataSource, you'll need to wrap create table (or maybe create view) around the select:
create table translations as
select ...

SQL Statement - Select from two tables, create column if secondary table has related record

I'm posting here because I have not been able to find what I'm looking for, or even the correct keywords to search on. If there are better answers that I was unable to find, please feel free to point me in that direction.
However I have two tables which Table 1 is the primary table, and I need to SELECT all records out of it and add an additional column in the SELECT that returns if any related records in Table 2.
I have boiled the problem down to the following and any help would be much appreciated.
Table 1 has a many relationship to Table 2
SELECT must return all rows from Table 1
SELECT must have an additional column (preferably a BOOLEAN/INTEGER) column that represents if there are any related records in Table 2.
SELECT must work in both Access and SQL Server
TABLE 1
--------
GUID1 | DATA FIELD | DATA FIELD
GUID2 | DATA FIELD | DATA FIELD
GUID3 | DATA FIELD | DATA FIELD
TABLE 2
--------
GUID1 | TABLE 1 GUID | DATA FIELD | DATA FIELD
GUID2 | TABLE 1 GUID | DATA FIELD | DATA FIELD
GUID3 | TABLE 2 GUID | DATA FIELD | DATA FIELD
GUID4 | TABLE 2 GUID | DATA FIELD | DATA FIELD
SELECTED TABLE ( 1 JOINED ON TABLE 2 )
--------
GUID1 | DATA FIELD | DATA FIELD | 1 (EXISTS IN TABLE 2)
GUID2 | DATA FIELD | DATA FIELD | 1 (EXISTS IN TABLE 2)
GUID3 | DATA FIELD | DATA FIELD | 0 (DOES NOT EXISTS IN TABLE 2)
You can use a LEFT OUTER JOIN with a case statement to check if the data in the second table is null. Here is an example:
SELECT First.*,
CASE
WHEN Second.DATA3 IS NULL
THEN 0
ELSE 1
END
FROM First
LEFT OUTER JOIN Second ON First.GUID1 = Second.GUID1
SQL Fiddle: http://sqlfiddle.com/#!6/ab17a/1

SQL JOIN on Dynamic Column based on Variable

I have an image summary table [summary] that will serve as a reporting table in the near future. There is a reference table [views] and a third table that the image team populates [TeamImage]. The summary table has 1 row per part number (table has distinct part numbers) and many columns of image views (TOP, BOT, FRO, BAC, etc.). The [views] table lists each of these views with an id field, which is an IDENTITY field. The [TeamImage] table contains part numbers and views (part number field is not unique as the part numbers will be listed multiple times as they have image views).
Example:
TABLE [summary]
Part_Number | TOP | BOT | FRO | BAC |
12345 | | | | |
67890 | | | | |
TABLE [views]
id | View |
1 | TOP |
2 | BOT |
3 | FRO |
4 | BAC |
TABLE [TeamImage]
PartNum | View |
12345 | TOP |
12345 | BOT |
12345 | FRO |
12345 | BAC |
67890 | FRO |
67890 | BAC |
Here's what I need in the end:
TABLE [summary]
Part_Number | TOP | BOT | FRO | BAC |
12345 | 1 | 1 | 1 | 1 |
67890 | | | 1 | 1 |
I could run several update queries but I have 27 views and about 2 million part numbers. I was hoping I could run something like below, even though I know I cannot use a variable as the column name:
DECLARE #id int = (SELECT max(id) FROM [views]), #ViewType nvarchar(3);
WHILE #id IS NOT NULL
BEGIN
SELECT #ViewType = (SELECT [View] FROM [views] WHERE id = #id);
UPDATE a
SET a.[#ViewType] = '1'
FROM [summary] a
INNER JOIN [TeamImage] b
AND a.[Part_Number] = b.[PartNum]
WHERE b.[View] = #ViewType;
SELECT #id = max(id) FROM [views] WHERE id < #id;
END;
Basically, I was hoping to use a variable to grab the different views from the [views] table (id = 27 down to id=1...could have counted up but doesn't matter) and populate the corresponding field in the [summary] table.
I know the SET a.[#ViewType] = '1' won't work, and a colleague of mine mentioned using VB but didn't know if that really was the most efficient option. I understand that I could use a PIVOT on the [TeamImage] table, but I'm not sure that will allow me to update my [summary] table (which has many more fields in it than just the image views). It still seems I need something that will effectively loop through update queries. I could write 4 update queries, one for each view (although my real table has 27 views), but I need something more dynamic in case we add views in the future.
To create your final summary, you can do via a simple pivot, yet this is fixed to the few codes you've done... but I know SQL does have a PIVOT command, but not directly familiar enough with it.
select
TA.PartNum,
max( case when TA.TeamImage = 'TOP' then '1' else ' ' end ) as TOPview,
max( case when TA.TeamImage = 'BOT' then '1' else ' ' end ) as BOTview,
max( case when TA.TeamImage = 'FRO' then '1' else ' ' end ) as FROview,
max( case when TA.TeamImage = 'BAC' then '1' else ' ' end ) as BACview
from
TeamImage TA
group by
TA.PartNum
Obviously simple to expand, but you can also look into the "PIVOT" syntax
I asked the question a little better here: SQL output as variable in VB.net and was able to receive an answer that worked for what I was looking for. I appreciate DRapp providing a solution through PIVOT, but I think the VB way will be easier for me moving forward. In short, using VB with ExecuteScalar and ExecuteNonQuery, I was able to re-write my query using the variables I had above.