Lets say i have the following sql table named urls:
url
redirect
revenue
realRevenue
clicksGermany
clicksUSA
clicksIndia
gaxktgq
google.com
0.321
69.51
15
28
33
oqjkgf1
example.cn
0.252
1424.3
1202
10
69
gaxktgq
corn.shop
1.242
42525.2
325525
1230
420
Now i want to fetch every column after realRevenue.
In this example you could just fetch by using the names of the columns that come after realRevenue but in my case there are way more fields that come after.
What query do i need?
If the fields you have correspond to the ones presented in your sample input, you should directly select those who are needed by you. If they're more than what we see here, check the next part.
As long as SQLite does not support dynamic queries, you can't create a query in an automatical way.
Although you can retrieve your table interesting fields by accessing the two tables "sqlite_master" and "pragma_table_info", that contains information regarding your table name and table fields respectively. By filtering on the table name and on the field id, you can have a list of all your fields.
SELECT p.name AS column_name
FROM sqlite_master AS m
JOIN pragma_table_info(m.name) AS p ON m.name = 'tab' AND p.cid >= 4
Output:
column_name
clicksGermany
clicksUSA
clicksIndia
But you can also have them prepared to be hardcoded into a SELECT statement, applying a GROUP_CONCAT on the concatenation of the table name and each table field.
SELECT GROUP_CONCAT(m.name || '.' || p.name, ', ') AS fields
FROM sqlite_master AS m
JOIN pragma_table_info(m.name) AS p ON m.name = 'tab' AND p.cid >= 4
fields
tab.clicksGermany, tab.clicksUSA, tab.clicksIndia
Check the demo here.
Note: This solution gets useful if your amount of fields is very big, such that writing all of them by hand becomes a time-consuming task.
I've seen a few of the questions and answers for this, but they all seem different than my problem. I am trying to insert into a temp table with a where clause from a real table to the id on a different temp table. Let me explain
Here is my first insert. It creates a temp table based on the parameters
Insert Into #programs (programs_id, state_program_ID, org_no, bldg_no)
Select programs_ID, state_program_ID, org_no, bldg_no
From programs as p
Where p.org_no = #org_no
And p.bldg_no = #bldg_no
And p.school_yr = #school_year
This returns a table that has a flat list of programs. Programs are offered at the school level and are slightly modified from the related state_program.
Then I need a list of all students that have taken the program from the program_student table.
Insert Into #programStudent (programs_id , ss_id, status_id)
Select ps.programs_id, ps.ss_id, ps.status_id
From program_student as ps
Where ps.programs_id = #programs.program_id
--'#programs.program_id' throws error
This would meet my need having all students that have taken any of the programs offered by the school at that school year.
The full error is
The multi-part identifier '#programs.program_id' could not be bound.
You are not addressing the #programs table in your second query. that last line will have to change to something like this:
WHERE EXISTS (SELECT TOP 1 1 FROM #Programs WHERE #Programs.programs_id = ps.program_id)
This is how you must address temp tables - they do not become variables in your current script - they are actual tables which get cleaned up after you disconnect. As such, they need to be introduced as tables - in a from clause, for each query that needs to reference them.
You had a typo when defining the #programs table - you called the column programs_id and not program_id. Just fix it, and you should be fine:
Insert Into #programs (program_id, state_program_ID, org_no, bldg_no)
-- "s" removed Here ---------^
Select programs_ID, state_program_ID, org_no, bldg_no
From programs as p
Where p.org_no = #org_no
And p.bldg_no = #bldg_no
And p.school_yr = #school_year
I have a DB Table [List1], 2 columns, Name, Number
SQLFiddle
I have an excel spreadsheet with 2 columns,names and numbers.
I want to match the Names in Excel to the Names column in SQL and If a match is found insert the relevant number in the second column.
Something tells me I will need to build an array / or csv and run some Tsql to achieve this.
I originally used the Task> Import data to build the DB Table.
Will importing the data again just overwrite the existing data?
What is the most efficient way to import the info, but update existing numbers? [EDIT, I have made some progress, read on]
I have managed to Create an conditional insert:
SET #PersonName = 'Andy
insert into People (Name, Number)
select
#PersonName
where not exists (
select * from People where Name = #PersonName
);
How do I pump the name list into the #PersonName variable and loop through the command in SQL?
Update:
I want to update the Datasets based on a dual column First/Last name.
Will this Work?
Update : Yes it worked, final code below.
update p
set p.number = s.numbers
from People p
join dbo.[spreadsheet] s on p.Firstname = s.Firstname AND p.lastname = s.lastname
If I understood you correctly and you want to match relatively small amount of data (up to 2k-5k rows) between excel and database table you may perform the next sequence of actions:
In SSMS execute: create table dbo.[spreadsheet] (firstname nvarchar(100), lastname nvarchar(100), numbers int);
In Excel spreadsheet copy to buffer data from firstname, lastname and numbers columns (without headers)
In SSMS Object Explorer: Tables->Right click->Refresh
Select dbo.spreadsheet table->Right click->Edit top 200 rows
In the designer select last row->Right click the on row header->Paste
And finally execute following update statement (see below)
update p
set p.number = s.numbers
from People p
join dbo.[spreadsheet] s on s.firstname = p.firstname and s.lastname = p.lastname
I am trying to find Relative information from a table and return those results (along with other unrelated results) in one row as part of a larger query.
I already tried using this example, modified for my data.
How to return multiple values in one column (T-SQL)?
But I cannot get it to work. It will not pull any data (I am sure it is is user[me] error).
If I query the table directly using a TempTable, I can get the results correctly.
DECLARE #res NVARCHAR(100)
SET #res = ''
CREATE TABLE #tempResult ( item nvarchar(100) )
INSERT INTO #tempResult
SELECT Relation AS item
FROM tblNextOfKin
WHERE ID ='xxx' AND Address ='yyy'
ORDER BY Relation
SELECT #res = #res + item + ', ' from #tempResult
SELECT substring(#res,1,len(#res)-1) as Result
DROP TABLE #tempResult
Note the WHERE line above, xxx and yyy would vary based on the input criteria for the function. but since you cannot use TempTables in a function... I am stuck.
The relevant fields in the table I am trying to query are as follows.
tblNextOfKin
ID - varchar(12)
Name - varchar(60)
Relation - varchar(30)
Address - varchar(100)
I hope this makes enough sense... I saw on another post an expression that fits.
My SQL-fu is not so good.
Once I get a working function, I will place it into the main query for the SSIS package I am working on which is pulling data from many other tables.
I can provide more details if needed, but the site said to keep it simple, and I tried to do so.
Thanks !!!
Follow-up (because when I added a comment to the reponse below, I could not edit formatting)
I need to be able to get results from different columns.
ID Name Relation Address
1, Mike, SON, 100 Main St.
1, Sara, DAU, 100 Main St.
2, Tim , SON, 123 South St.
Both the first two people live at the same address, so if I query for ID='1' and Address='100 Main St.' I need the results to look something like...
"DAU, SON"
Mysql has GROUP_CONCAT
SELECT GROUP_CONCAT(Relation ORDER BY Relation SEPARATOR ', ') AS item
FROM tblNextOfKin
WHERE ID ='xxx' AND Address ='yyy'
You can do it for the whole table with
SELECT ID, Address, GROUP_CONCAT(Relation ORDER BY Relation SEPARATOR ', ') AS item
FROM tblNextOfKin
GROUP BY ID, Address
(assuming ID is not unique)
note: this is usually bad practice as an intermediate step, this is acceptable only as final formatting for presentation (otherwise you will end up ungrouping it which will be pain)
I think you need something like this (SQL Server):
SELECT stuff((select ',' +Relation
FROM tblNextOfKin a
WHERE ID ='xxx' AND Address ='yyy'
ORDER BY Relation
FOR XML path('')),1,1,'') AS res;
I know that that is not a question... erm anyway HERE is the question.
I have inherited a database that has 1(one) table in that looks much like this. Its aim is to record what species are found in the various (200 odd) countries.
ID
Species
Afghanistan
Albania
Algeria
American Samoa
Andorra
Angola
....
Western Sahara
Yemen
Zambia
Zimbabwe
A sample of the data would be something like this
id Species Afghanistan Albania American Samoa
1 SP1 null null null
2 SP2 1 1 null
3 SP3 null null 1
It seems to me this is a typical many to many situation and I want 3 tables.
Species, Country, and SpeciesFoundInCountry
The link table (SpeciesFoundInCountry) would have foreign keys in both the species and Country tables.
(It is hard to draw the diagram!)
Species
SpeciesID SpeciesName
Country
CountryID CountryName
SpeciesFoundInCountry
CountryID SpeciesID
Is there a magic way I can generate an insert statement that will get the CountryID from the new Country table based on the column name and the SpeciesID where there is a 1 in the original mega table?
I can do it for one Country (this is a select to show what I want out)
SELECT Species.ID, Country.CountryID
FROM Country, Species
WHERE (((Species.Afghanistan)=1)) AND (((Country.Country)="Afghanistan"));
(the mega table is called species)
But using this strategy I would need to do the query for each column in the original table.
Is there a way of doing this in sql?
I guess I can OR a load of my where clauses together and write a script to make the sql, seems inelegant though!
Any thoughts (or clarification required)?
I would use a script to generate all the individual queries, since this is a one-off import process.
Some programs such as Excel are good at mixing different dimensions of data (comparing column names to data inside rows) but relational databases rarely are.
However, you might find that some systems (such as Microsoft Access, surprisingly) have convenient tools which you can use to normalise the data. Personally I'd find it quicker to write the script but your relative skills with Access and scripting might be different to mine.
Why do you want to do it in SQL? Just write a little script that does the conversion.
When I run into these I write a script to do the conversion rather than trying to do it in SQL. It is typically much faster and easier for me. Pick any language you are comfortable with.
If this was SQL Server, you'd use the Unpivot commands, but looking at the tag you assigned it's for access - am I right?
Although there is a pivoting command in access, there is no reverse statement.
Looks like it can be done with a complex join. Check this interesting article for a lowdown on how to unpivot in a select command.
You're probably going to want to create replacement tables in place. The script sort of depends on the scripting language you have available to you, but you should be able to create the country ID table simply by listing the columns of the table you have now. Once you've done that, you can do some string substitutions to go through all of the unique country names and insert into the speciesFoundInCountry table where the given country column is not null.
You could probably get clever and query the system tables for the column names, and then build a dynamic query string to execute, but honestly that will probably be uglier than a quick script to generate the SQL statements for you.
Hopefully you don't have too much dynamic SQL code that accesses the old tables buried in your codebase. That could be the really hard part.
In SQL Server this will generate your custom select you demonstrate. You can extrapolate to an insert
select
'SELECT Species.ID, Country.CountryID FROM Country, Species WHERE (((Species.' +
c.name +
')=1)) AND (((Country.Country)="' +
c.name +
'"))'
from syscolumns c
inner join sysobjects o
on o.id = c.id
where o.name = 'old_table_name'
As with the others I would most likely just do it as a one time quick fix in whatever manner works for you.
With these types of conversions, they are one off items, quick fixes, and the code doesn't have to be elegant, it just has to work. For these types of things I have done it many ways.
If this is SQL Server, you can use the sys.columns table to find all of the columns of the original table. Then you can use dynamic SQL and the pivot command to do what you want. Look those up online for syntax.
I would definitely agree with your suggestion of writing a small script to produce your SQL with a query for every column.
In fact your script could have already been finished in the time you've spent thinking about this magical query (that you would use only one time and then throw away, so what's the use in making it all magicy and perfect)
Sorry, but the bloody posting parser removed the whitespace and formatting on my post. It makes it a log harder to read.
#stomp:
Above the box where you type the answer, there are several buttons. The one that is 101010 is a code sample. You select all your text that is code, and then click that button. Then it doesn't get messed with much.
cout>>"I don't know C"
cout>>"Hello World"
I would use a Union query, very roughly:
Dim db As Database
Dim tdf As TableDef
Set db = CurrentDb
Set tdf = db.TableDefs("SO")
strSQL = "SELECT ID, Species, """ & tdf.Fields(2).Name _
& """ AS Country, [" & tdf.Fields(2).Name & "] AS CountryValue FROM SO "
For i = 3 To tdf.Fields.Count - 1
strSQL = strSQL & vbCrLf & "UNION SELECT ID, Species, """ & tdf.Fields(i).Name _
& """ AS Country, [" & tdf.Fields(i).Name & "] AS CountryValue FROM SO "
Next
db.CreateQueryDef "UnionSO", strSQL
You would then have a view that could be appended to your new design.
When I read the title 'bad BAD database design', I was curious to find out how bad it is. You didn't disappoint me :)
As others mentioned, a script would be the easiest way. This can be accomplished by writing about 15 lines of code in PHP.
SELECT * FROM ugly_table;
while(row)
foreach(row as field => value)
if(value == 1)
SELECT country_id from country_table WHERE country_name = field;
if(field == 'Species')
SELECT species_id from species_table WHERE species_name = value;
INSERT INTO better_table (...)
Obviously this is pseudo code and will not work as it is. You can also populate the countries and species table on the fly by adding insert statements here.
Sorry, I've done very little Access programming but I can offer some guidance which should help.
First lets walk through the problem.
It is assumed that you will typically need to generate multiple rows in SpeciesFoundInCountry for every row in the original table. In other words species tend to be in more then one country. This is actually easy to do with a Cartesian product, a join with no join criteria.
To do a Cartesian product you will need to create the Country table. The table should have the country_id from 1 to N (N being the number of unique countries, 200 or so) and country name. To make life easy just use the numbers 1 to N in column order. That would make Afghanistan 1 and Albania 2 ... Zimbabwe N. You should be able to use the system tables to do this.
Next create a table or view from the original table which contains the species and a sting with a 0 or 1 for each country. You will need to convert the null, not null to a text 0 or 1 and concatenate all of the values into a single string. A description of the table and a text editor with regular expressions should make this easy. Experiment first with a single column and once that's working edit the create view/insert with all of the columns.
Next join the two tables together with no join criteria. This will give you a record for every species in every country, you're almost there.
Now all you have to do is filter out the records which are not valid, they will have a zero in the corresponding location in the string. Since the country table's country_code column has the substring location all you need to do is filter out the records where it's 0.
where substring(new_column,country_code) = '1'
You will still need to create the species table and join to that
where a.species_name = b.species_name
a and b are table aliases.
Hope this help
OBTW,
If you have queries that already run against the old table you will need to create a view which replicates the old tables using the new tables. You will need to do a group by to denormalize the tables.
Tell your users that the old table/view will not be supported in the future and all new queries or updates to older queries will have to use the new tables.
If I ever have to create a truckload of similar SQL statements and execute all of them, I often find Excel is very handy. Take your original query. If you have a country list in column A and your SQL statement in column B, formated as text (in quotes) with cell references inserted where the country appears in the sql
e.g. ="INSERT INTO new_table SELECT ... (species." & A1 & ")= ... ));"
then just copy the formula down to create 200 different SQL statements, copy/paste the column to your editor and hit F5. You can of course do this with as many variables as you want.
When I've been faced with similar problems, I've found it convenient to generate a script that generates SQL scripts. Here's the sample you gave, abstracted to use %PAR1% in place of Afghanistan.
SELECT Species.ID, Country.CountryID
FROM Country, Species
WHERE (((Species.%PAR1%)=1)) AND (((Country.Country)="%PAR1%"))
UNION
Also the key word union has been added as a way to combine all the selects.
Next, you need a list of countries, generated from your existing data:
Afghanistan
Albania
.
,
.
Next you need a script that can iterate through the country list, and for each iteration,
produce an output that substitutes Afghanistan for %PAR1% on the first iteration, Albania for the second iteration and so on. The algorithm is just like mail-merge in a word processor. It's a little work to write this script. But, once you have it, you can use it in dozens of one-off projects like this one.
Finally, you need to manually change the last "UNION" back to a semicolon.
If you can get Access to perform this giant union, you can get the data you want in the form you want, and insert it into your new table.
I would make it a three step process with a slight temporary modification to your SpeciesFoundInCountry table. I would add a column to that table to store the Country name. Then the steps would be as follows.
1) Create/Run a script that walks columns in the source table and creates a record in SpeciesFoundInCountry for each column that has a true value. This record would contain the country name.
2) Run a SQL statement that updates the SpeciesFoundInCountry.CountryID field by joining to the Country table on Country Name.
3) Cleanup the SpeciesFoundInCountry table by removing the CountryName column.
Here is a little MS Access VB/VBA pseudo code to give you the gist
Public Sub CreateRelationshipRecords()
Dim rstSource as DAO.Recordset
Dim rstDestination as DAO.Recordset
Dim fld as DAO.Field
dim strSQL as String
Dim lngSpeciesID as Long
strSQL = "SELECT * FROM [ORIGINALTABLE]"
Set rstSource = CurrentDB.OpenRecordset(strSQL)
set rstDestination = CurrentDB.OpenRecordset("SpeciesFoundInCountry")
rstSource.MoveFirst
' Step through each record in the original table
Do Until rstSource.EOF
lngSpeciesID = rstSource.ID
' Now step through the fields(columns). If the field
' value is one (1), then create a relationship record
' using the field name as the Country Name
For Each fld in rstSource.Fields
If fld.Value = 1 then
with rstDestination
.AddNew
.Fields("CountryID").Value = Null
.Fields("CountryName").Value = fld.Name
.Fields("SpeciesID").Value = lngSpeciesID
.Update
End With
End IF
Next fld
rstSource.MoveNext
Loop
' Clean up
rstSource.Close
Set rstSource = nothing
....
End Sub
After this you could run a simple SQL statement to update the CountryID values in the SpeciesFoundInCountry table.
UPDATE SpeciesFoundInCountry INNER JOIN Country ON SpeciesFoundInCountry.CountryName = Country.CountryName SET SpeciesFoundInCountry.CountryID = Country.CountryID;
Finally, all you have to do is cleanup the SpeciesFoundInCountry table by removing the CountryName column.
****SIDE NOTE: I have found it usefull to have country tables that also include the ISO abbreviations (country codes). Occassionally they are used as Foreign Keys in other tables so that a join to the Country table does not have to be included in queries.
For more info: http://en.wikipedia.org/wiki/Iso_country_codes
This is (hopefully) a one-off exercise, so an inelegant solution might not be as bad as it sounds.
The problem (as, I'm sure you're only too aware!) is that at some point in your query you've got to list all those columns. :( The question is, what is the most elegant way to do this? Below is my attempt. It looks unwieldy because there are so many columns, but it might be what you're after, or at least it might point you in the right direction.
Possible SQL Solution:
/* if you have N countries */
CREATE TABLE Country
(id int,
name varchar(50))
INSERT Country
SELECT 1, 'Afghanistan'
UNION SELECT 2, 'Albania',
UNION SELECT 3, 'Algeria' ,
UNION SELECT 4, 'American Samoa' ,
UNION SELECT 5, 'Andorra' ,
UNION SELECT 6, 'Angola' ,
...
UNION SELECT N-3, 'Western Sahara',
UNION SELECT N-2, 'Yemen',
UNION SELECT N-1, 'Zambia',
UNION SELECT N, 'Zimbabwe',
CREATE TABLE #tmp
(key varchar(N),
country_id int)
/* "key" field needs to be as long as N */
INSERT #tmp
SELECT '1________ ... _', 'Afghanistan'
/* '1' followed by underscores to make the length = N */
UNION SELECT '_1_______ ... ___', 'Albania'
UNION SELECT '__1______ ... ___', 'Algeria'
...
UNION SELECT '________ ... _1_', 'Zambia'
UNION SELECT '________ ... __1', 'Zimbabwe'
CREATE TABLE new_table
(country_id int,
species_id int)
INSERT new_table
SELECT species.id, country_id
FROM species s ,
#tmp t
WHERE isnull( s.Afghanistan, ' ' ) +
isnull( s.Albania, ' ' ) +
... +
isnull( s.Zambia, ' ' ) +
isnull( s.Zimbabwe, ' ' ) like t.key
My Suggestion
Personally, I would not do this. I would do a quick and dirty solution like the one to which you allude, except that I would hard-code the country ids (because you're only going to do this once, right? And you can do it right after you create the country table, so you know what all the IDs are):
INSERT new_table SELECT Species.ID, 1 FROM Species WHERE Species.Afghanistan = 1
INSERT new_table SELECT Species.ID, 2 FROM Species WHERE Species.Albania= 1
...
INSERT new_table SELECT Species.ID, 999 FROM Species WHERE Species.Zambia= 1
INSERT new_table SELECT Species.ID, 1000 FROM Species WHERE Species.Zimbabwe= 1