I am trying to create a new table using pandas sqldf as shown below:
pip install -U pandasql
from pandasql import sqldf
sqldf("create table mytable (id integer, name text)")
sqldf("select * from mytable")
This is failing showing
OperationalError: no such table: mytable
My goal is to insert data into the table as shown below:
sqldf("insert into mytable values (1, 'abc')");
sqldf("insert into mytable values (2, 'def')");
sqldf("insert into mytable values (3, 'ghk')");
Any help would be appreciated.
Related
I am working on a project that has a C# front end that will be used to select a file for importing into an MSSQL SQL Database. In the table there will be an additional column called 'recommendedAction' (tinyint - 0-5 only)
I would like to have sql fill in the 'recommendedAction' column based on criteria in a different table.
Is there a way that when SQL is importing (SSIS or pure TSQL) it could read the values of a table and fill in the 'action' based on the criteria? Or is this something that should be done in the C# frontend?
EDIT
SQL table structure for imported data (with additional column)
Create Table ImportedData (
Column1 INT Identity,
Column2 VARCHAR(10) NOT NULL,
Column3 CHAR(6) NOT NULL,
RecommendedAction TINYINT NOT NULL
)
Table structure of recommended action criteria
Create Table RecommendedActions(
ID INT Identity,
ActionID TINYINT NOT NULL, --value to put in the RecommendedAction column if criteria is a match
CriteriaColumn VARCHAR(255) NOT NULL --Criteria to match against the records
)
Example records for RecommendedActions
ID ActionID CriteriaColumn
1 2 'Column2 LIKE ''6%'''
2 3 'Column2 LIKE ''4%'''
Now when a new set of data is imported, if Column2 has a value of '6032' it would fill in a RecommendedAction of 2
Many ways exist. For example you can insert into the tb table a value selected from the ta table according to criteria.
Example setup
create table ta(
Id int,
val int);
insert into ta(ID, val) values
(1, 30)
,(2, 29)
,(3, 28)
,(4, 27)
,(5, 26);
create table tb
(Id int,
ref int);
Example insert
-- parameters
declare #p1 int = 1,
#p2 int = 27;
-- parameterized INSERT
insert tb(Id, ref)
values(#p1, (select ta.id from ta where ta.val=#p2));
Below added Stored procedure will do the job. It gets the Action column value based on the Column2 parameter and insert into the ImportedData table. You can execute this Stored procedure inside the C# code with required parameters. I added sample execute statements for to test the query.
Sample data inserted to the RecommendedActions Table:
INSERT INTO RecommendedActions
VALUES
(2, 'Column2 LIKE ''6%''')
,(3, 'Column2 LIKE ''4%''')
Stored Procedure Implementation :
CREATE PROCEDURE Insert_ImportedData(
#Column2 AS VARCHAR(10)
,#Column3 AS CHAR(3)
)
AS
BEGIN
DECLARE #RecommendedAction AS TINYINT
SELECT #RecommendedAction = ActionID
FROM RecommendedActions
WHERE SUBSTRING(CriteriaColumn, 15, 1) = LEFT(#Column2 , 1)
INSERT INTO ImportedData VALUES (#Column2,#Column3,#RecommendedAction)
END
GO
This is the execute statement for the Above Stored procedure
EXEC Insert_ImportedData '43258' , 'ATT'
EXEC Insert_ImportedData '63258' , 'AOT'
you can use sqlalchemy in python and load your data into a dataframe then append the dataframe to the sql table. You can set the dtype for each of the field datatype in the read_csv using a dictionary. Loading data with Python is super powerful because the bulk load is fast. Use your c# code to build the csv file using stream io and use linq to for your conditions for data fields. Then use python to load your csv.
import pandas as pd
from sqlalchemy import create_engine
engine = create_engine(connectionstring)
df = pd.read_csv("your_data.csv", header=None)
df.columns = ['field1', 'field2', 'field3']
df.to_sql(name="my_sql_table", con=connection, if_exists='append', index=False)
I created a table in DBBrowser:
CREATE VIRTUAL TABLE IF NOT EXISTS Students USING FTS5
(
GroupId UNINDEXED,
StudentName
);
and insert values to it. After that I add DB with this table to my project.
It is declaration of this table in sqldelight .sq file:
CREATE VIRTUAL TABLE IF NOT EXISTS Students USING FTS5
(
GroupId INTEGER AS Int,
StudentName TEXT,
rank REAL
);
I need to explicit declare rank because I want to apply HAVING MIN(rank) for it when SELECT from table (otherwise it is not compile), but when I trying to insert values in table like that:
insert:
INSERT INTO Students VALUES (?,?);
I receive an error:
Unexpected number of values being inserted. found: 2 expected: 3
If I do like that:
insert:
INSERT INTO Students VALUES (?,?,?);
I receive an exception:
SQLiteException - table Students has 2 columns but 3 values were supplied (code 1): , while compiling: INSERT INTO Students VALUES (?,?,?)
How I can perform insert? Or maybe I can apply HAVING MIN(rank) without explicit declare?
does
insert:
INSERT INTO Students(GroupId, StudentName) VALUES (?,?);
work?
I have some temp table:
CREATE TEMP TABLE IF NOT EXISTS temp_test (
col1 INTEGER NOT NULL,
col2 CHARACTER VARYING NOT NULL,
col3 BOOLEAN);
Then I do some inserts into temp_test (that works fine).
Later, without creating a new table test, I try doing the following:
INSERT INTO test(col1,col2,col3) SELECT col1,col2,col3 FROM temp_tes;
And I get the following error:
ERROR: relation "test" does not exist
I thought that if I'm using INSERT INTO, it should create the table for me. does it not?
If it matters, I'm using PostgreSQL 9.6.16.
You are wrong. INSERT inserts into an existing table; it does not create a table.
If you want to create a table, use CREATE TABLE AS:
CREATE TABLE test AS
SELECT col1, ol2, col3
FROM temp_tes;
I have create the following SQL table in databricks (using the magic %sql) as follows:
%sql
CREATE TABLE mytable (
id INT
,name STRING
,met_area_name STRING
,state STRING
,type STRING
) USING CSV
I am now trying insert data into the table using the following command:
%sql
INSERT INTO TABLE mytable VALUES (id,name,type)
SELECT DISTINCT criteria1, criteria2, 'b'
FROM tablex
WHERE somecriteria1 = 0
ORDER BY somecriteria2;
However, I'm getting the following error:
Error in SQL statement: ParseException:
mismatched input 'FROM' expecting <EOF>(line 2, pos 2)
== SQL ==
INSERT INTO TABLE mytable VALUES (id,name,type)
FROM tablex
--^^^
WHERE somecriteria1 = 0
ORDER BY somecriteria2
I'm sure there is something very obvious that I'm missing, but I can't see it.
Any assistance much appreciated.
Cheers
So I learnt from here how to insert values into an array column:
INSERT INTO table
SELECT ARRAY("line1", "line2", "line3") as myArray
FROM source1;
And from here how to insert values into an struct column:
INSERT INTO table
SELECT NAMED_STRUCT('houseno','123','streetname','GoldStreet', 'town','London', 'postcode','W1a9JF') AS address
FROM source2;
Now I was trying to insert in the same way values in an array of structs. Which has got the following schema:
additionalattribute:array<struct<attribute_value:string,key:string,value:string>
I tried to extrapolate like this:
INSERT INTO table
ARRAY(NAMED_STRUCT('attribute_value','null','key','null','value','null')) as additionalattribute
FROM source2;
But it is not working. Does anyone know how to approach this issue?
you are missing the select statement after the table name. Demo
create table temp4
(
additionalattribute array<struct<attribute_value:string,key:string,value:string>>
);
INSERT INTO temp4 select
ARRAY(NAMED_STRUCT('attribute_value','null','key','null','value','null')) as additionalattribute
FROM (select '1' ) t;