I have created a table and would like to import the data in my csv file over to pgAdmin. The following were the queries executed in the query editor :
create table offline_trx
(sale_key varchar(50),
order_id varchar(50),
country_code varchar(10),
qty_ordered integer,
qty_refunded integer,
extended_gmv_sgd DECIMAL(19,9),
extended_net_amount_sgd float DECIMAL(19,9),
product_sku varchar(50),
product_type varchar(20),
category varchar(20),
created_at_time_ist date
);
...and the below query to copy data from csv file ...
COPY public.offline_trx(
sale_key, order_id, country_code, qty_ordered, qty_refunded, extended_gmv_sgd, extended_net_amount_sgd, product_sku, product_type, category, created_at_time_ist)
FROM 'C:\Users\Sam\Downloads\offline_trx.csv'
DELIMITER ','
CSV HEADER;
After executing the above copy query, it showed that the query returned successfully. But when I select the table appears to be empty. So I'm not sure what am I missing. Is there a way that I can inspect the logs somewhere to better understand what exactly happened that resulted in it appearing to be "missing"?
Related
When I create a connection to SQL in Excel that has a temp table for a lookup and I hard code the parameters for the SQL query, everything runs fine. When I change the hard coded parameters to ? to be able to use dynamic parameters from a cell, I get an Invalid Object Name error on the Temp Table. ANy help is greatly appreciated.
SET NOCOUNT ON
CREATE TABLE #ACTUALPENDCODESLIST
(ID INT IDENTITY(1,1) PRIMARY KEY,
PENDCODE CHAR(2),
PENDDESCRIPTION VARCHAR(255),
AREAOFOWNERSHIP VARCHAR(255),
PendQueue VARCHAR(255),
PendPriority INT)
INSERT INTO #ACTUALPENDCODESLIST VALUES ('02','PEND - PHYSICIAN/GROUP NOT CONTRACTED FOR RENAL CARE SERVICES','N/A','Not Applicable',999)
INSERT INTO #ACTUALPENDCODESLIST VALUES ('03','PEND - PROC DETAIL NOT FOUND ON FILE','Claims','MCLAIM/HCLAIM',267)
SELECT DISTINCT(SUBSTRING(ph.FREE_FORM_DATA, 16,2)) AS 'Pend Code',
apc.PENDDESCRIPTION AS 'Description',
COUNT(DISTINCT(ph.CLAIM_NBR)) AS 'Count'
FROM process_hist ph
INNER JOIN #ACTUALPENDCODESLIST apc
ON SUBSTRING(ph.FREE_FORM_DATA, 16,2) = apc.PENDCODE
WHERE ph.YMDTRANS BETWEEN ? AND ?
AND ph.OP_NBR NOT LIKE 'SYS%'
AND ph.OP_NBR NOT LIKE 'SMK%'
AND SUBSTRING(ph.FREE_FORM_DATA, 13,2) = 'EX'
AND SUBSTRING(ph.FREE_FORM_DATA, 16,2) IN (SELECT PENDCODE FROM #ACTUALPENDCODESLIST)
GROUP BY SUBSTRING(ph.FREE_FORM_DATA, 16,2), apc.PENDDESCRIPTION
ORDER BY [Count] DESC, [Pend Code]
DROP TABLE #ACTUALPENDCODESLIST```
Hi everyone thanks so much for taking the time to read this.
I'd like to create a macro in Teradata that will create a table from another table based on specific parameters.
My original table consists of three columns patient_id, diagnosis_code and Date_of_birth
......
I'd like to build a macro that would allow me to specify a diagnosis code and it would then build the table consisting of data of all patients with that diagnosis code.
My current code looks like this
Create Macro All_pats (diag char) as (
create table pats as(
select *
from original_table
where diag = :diagnosis_code;)
with data primary index (patid);
I cant seem to get this to work - any tips?
Thanks once again
Your code has a semicolon in a wrong place and a missing closing bracket:
Create Macro All_pats (diag char) as (
create table pats as
(
select *
from original_table
where diag = :diagnosis_code
) with data primary index (patid);
);
Edit:
Passing multiple values as a delimited list is more complicated (unless you use Dynamic SQL in a Stored Procedure):
REPLACE MACRO All_lpats (diagnosis_codes VARCHAR( 1000)) AS
(
CREATE TABLE pats AS
(
SELECT *
FROM original_table AS t
JOIN TABLE (StrTok_Split_To_Table(1, :diagnosis_codes, ',')
RETURNS (outkey INTEGER,
tokennum INTEGER,
token VARCHAR(20) CHARACTER SET Unicode)
) AS dt
ON t.diag = dt.token
) WITH DATA PRIMARY INDEX (patid);
);
EXEC All_lpats('111,112,113');
As the name implies StrTok_Split_To_Table splits a delimited string into a table. You might need to adust the delimiter and the length of the resulting token.
I have a very basic question which is: How can I add a very simple table to Hive. My table is saved in a text file (.txt) which is saved in HDFS. I have tried to create an external table in Hive which points out this file but when I run an SQL query (select * from table_name) I don't get any output.
Here is an example code:
create external table Data (
dummy INT,
account_number INT,
balance INT,
firstname STRING,
lastname STRING,
age INT,
gender CHAR(1),
address STRING,
employer STRING,
email STRING,
city STRING,
state CHAR(2)
)
LOCATION 'hdfs:///KibTEst/Data.txt';
KibTEst/Data.txt is the path of the text file in HDFS.
The rows in the table are seperated by carriage return, and the columns are seperated by commas.
Thanks for your help!
You just need to create an external table pointing to your file
location in hdfs and with delimiter properties as below:
create external table Data (
dummy INT,
account_number INT,
balance INT,
firstname STRING,
lastname STRING,
age INT,
gender CHAR(1),
address STRING,
employer STRING,
email STRING,
city STRING,
state CHAR(2)
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
LOCATION 'hdfs:///KibTEst/Data.txt';
You need to run select query(because file is already in HDFS and external table directly fetches data from it when location is specified in create statement). So you test using below select statement:
SELECT * FROM Data;
create external table Data (
dummy INT,
account_number INT,
balance INT,
firstname STRING,
lastname STRING,
age INT,
gender CHAR(1),
address STRING,
employer STRING,
email STRING,
city STRING,
state CHAR(2)
)
row format delimited
FIELDS TERMINATED BY ‘,’
stored as textfile
LOCATION 'Your hdfs location for external table';
If data in HDFS then use :
LOAD DATA INPATH 'hdfs_file_or_directory_path' INTO TABLE tablename
The use select * from table_name
create external table Data (
dummy INT,
account_number INT,
balance INT,
firstname STRING,
lastname STRING,
age INT,
gender CHAR(1),
address STRING,
employer STRING,
email STRING,
city STRING,
state CHAR(2)
)
row format delimited
FIELDS TERMINATED BY ','
stored as textfile
LOCATION '/Data';
Then load file into table
LOAD DATA INPATH '/KibTEst/Data.txt' INTO TABLE Data;
Then
select * from Data;
I hope, below inputs will try to answer the question asked by #mshabeen.
There are different ways that you can use to load data in Hive table that is created as external table.
While creating the Hive external table you can either use the LOCATION option and specify the HDFS, S3 (in case of AWS) or File location, from where you want to load data OR you can use LOAD DATA INPATH option to load data from HDFS, S3 or File after creating the Hive table.
Alternatively you can also use ALTER TABLE command to load data in the Hive partitions.
Below are some details
Using LOCATION - Used while creating the Hive table. In this case data is already loaded and available in Hive table.
**LOAD DATA INPATH** option - This Hive command can be used to load data from specified location. Point to remember here is, the data will get MOVED from input path to Hive warehouse path.
Example -
LOAD DATA INPATH 'hdfs://cluster-ip/path/to/data/location/'
Using ALTER TABLE command - Mostly this is used to add data from other locations into the Hive partitions. In this case it is required that all partitions are already defined and the values for the partitions are already known. In case of dynamic partitions this command is not required.
Example -
ALTER TABLE table_name ADD PARTITION (date_col='2018-02-21') LOCATION 'hdfs/path/to/location/'
The above code will map the partition to the specified data location (in this case HDFS). However, the data will NOT MOVED to Hive internal warehouse location.
Additional details are available here
Attempting to bulk insert into a table and I am getting the error:
Bulk load data conversion error (type mismatch or invalid character for the specified codepage) for row 31, column 4 (Birthday).
Below is the code I am trying to use to insert the data:
Bulk Insert Dzt.dbo.Player
From 'A:\New Folder\Seed_Files\Player.csv'
With
(
FieldTerminator=',',
RowTerminator='\n',
FirstRow=2
)
Here is the code I used when making the table:
Use Dzt
Create Table Player
(
Player_ID int,
FirstName varchar(255),
LastName varchar(255),
Birthday date,
Email varchar(255),
L_Flag varchar(255)
);
This is my first attempt at making a table and inserting data so I am thinking it is likely a datatype error for the Birthday field but I have been unable to find anything online that I am able to grasp my head on at this time. I have also tried use the datatype datetime instead of date but I received the same error.
I am using SSMS 2012 to create and insert the data onto a 2012 SQL Server.
Let me know if there is anything else I can provide that might help.
As you suspect it could be a date format error, I would suggest importing the csv into a table with Birthday column set to varchar type. Then use this query to filter the erroneous records.
select birthday from temptable where isdate(birthday) = 0
You could then correct those records and then insert them into your old table.
I'm trying to load data via SQLoader, but it gives me error at the numeric field of Invalid Number
My Data File:
00163604~12002~S~N~N~Panasonic Juicer 1.5l Steel Color~ss~E~A~12/15/2014 3:33:57 PM~N~N~N~Y~294~SA
Control File:
LOAD DATA
INFILE "/home/dmf/ITEMLOC.txt"
APPEND
INTO TABLE DMF.MIG_ITEM_LC
FIELDS TERMINATED BY "~"
TRAILING NULLCOLS
(
ITEM "SUBSTRB(:ITEM,1,25)",
LOC "TO_NUMBER(:LOC)",
LOC_TYPE "SUBSTRB(:LOC_TYPE,1,1)",
CLEAR_IND "SUBSTRB(:CLEAR_IND,1,1)",
TAXABLE_IND "SUBSTRB(:TAXABLE_IND,1,1)",
LOCAL_ITEM_DESC "SUBSTRB(:LOCAL_ITEM_DESC,1,250)",
LOCAL_SHORT_DESC "SUBSTRB(:LOCAL_SHORT_DESC,1,120)",
STORE_ORD_MULT "SUBSTRB(:STORE_ORD_MULT,1,1)",
STATUS_UPDATE_DATE sysdate,
STATUS "SUBSTRB(:STATUS,1,1)",
STORE_PRICE_IND "SUBSTRB(:STORE_PRICE_IND,1,1)",
RPM_IND "SUBSTRB(:RPM_IND,1,1)",
EXT_UIN_IND "SUBSTRB(:EXT_UIN_IND,1,1)",
RANGED_IND "SUBSTRB(:RANGED_IND,1,1)",
PRIMARY_SUPP "TO_NUMBER(:PRIMARY_SUPP)", -- The Error is coming here
PRIMARY_CNTRY "SUBSTRB(:PRIMARY_CNTRY,1,3)"
)
Rejected - Error on table DMF.MIG_ITEM_LC, column PRIMARY_SUPP.
ORA-01722: invalid number
If i write give constant to it, it loads successfully.
What could be the issue?
Your data as posted loads fine for me.
SQL> select version from v$instance;
VERSION
-----------------
11.2.0.2.0
Here's the create table statement I used:
create table test1
(
item varchar2(25),
loc number,
loc_type char(1),
clear_ind char(1),
taxable_ind char(1),
local_item_desc varchar2(250),
local_short_desc varchar2(120),
store_ord_mult char(1),
status char(1),
store_price_ind char(1),
rpm_ind char(1),
ext_uin_ind char(1),
ranged_ind char(1),
primary_supp number,
primary_cntry varchar2(3)
);
If you get this error while trying to load only the one record that you posted, then I would suspect an unprintable character, like #Gary_W suggested. View the data with a hex viewer to check.
A character set difference between the file and your NLS_LANG setting could be at fault, but I doubt it in this case, since your data looks to be all ASCII values.