I am extracting data from a file daily and adding a timestamp to the ActiveDate column to capture when a particular record was imported.
Earlier we were using getdate() but we realized its not the most accurate solution for us. Fortunately the filename has a datestamp (something like XYZ 12232015).
Can I import the date as a date datatype into a field in Excel? I am not sure how to do that in SQL Server considering the file is saved on a shared drive. The file is moved to the archived folder after the SSIS package for the SQL code runs.
In SSIS you can get this using a Derived File Transformation.
Related
I have an SSIS package created using SSDT and running as a job on SQL Server 2014.
This SSIS package retrieves an Excel file (.xlsx) from a specific folder and exports its content into a specific table on my SQL Server database. The package runs fine.
My Data Flow is in the following sequence:
Import Excel file from folder
Apply a Conditional Split to split data with today's date
Export the data into the SQL Server table in the database
Here is my problem:
I will now have 4 additional Excel files into that folder and they will need to be exported into that same SQL Server table.
So what is the best way forward to achieve this (assuming all of them are possible solutions):
Rewrite 4 additional SSIS packages from scratch?
Use “Save As” existing package with a new name (4 times) and modify the file name to be retrieved?
Modify my existing SSIS package to accommodate for the additional 4 Excel files?
Any help would appreciated.
Assuming the 4 excel files are the same structure and going to the same table, you'll want to use the ForEach loop for each file in the folder.
SentryOne has a good example of looping through each file in a folder and archiving. I imagine it can be adjusted for your use case.
I'm fairly new to SSIS and my ingestion package is giving me a hard time :(
I have a .csv file with a column date with data as "2019/11/03". ("yyyy/mm/dd")within quotes
I have defined this as a DT_DBDATE in flat file connection manager
My SQL database table has a column with datatype as datetime.
While trying to ingest data into the table using SSIS, my table gets loaded with system date. i.e, the date when I'm running this package. I also tried DT_DATE and still received todays date in my table. It is so weird and I just cant seem to understand what the issue is.
I changed the datatype of the SQL table column from datetime to String and SSIS to DT_STR. Tried ingesting again and ended up getting 00:00 in the table. Not sure what is happening :(
Please help!!
I was able to use SQL Server Agent to create a job that extract Data from SQL Server 2008 to Excel Format Daily. However, is there anyway to create a job that keep all the extracts separately without overwrite the previous files? I would really much appreciated your help.
Thank you.
When using OLE DB/Jet data provider for Excel, there is a way to specify the target worksheet name. A worksheet in Excel is a rough equivalent of a database table. One option is to use a different worksheet name each time (say, based on a current date). Another option would be to append the data to the existing worksheet, if that's what you're after. If SQL Server Agent job does not allow you to do something like that, then you may want to create a small app instead.
While Copying from OLEDB to Excel, You can dynamically pass the file name through the expressions based on the Datetime.
Ex: Filepath+Date1_mmddyy_hhmmss
Every time the file generates, It will create a file with new file name and Excel file should be passed as an expression.
I'm trying to import a bunch of ach files and make a big sql table. An ach file is a text file with transaction information arranged in columns. The problem is that I need to add a date column. Currently the date is only contained in the file name and header. There are about 3000 files and each file is a different date.
I have basic knowledge of sql commands and how to query a database, but I just started learning about importing data for this project. The only tool I found is the program called "import and export data" as a part of sql server 2012. It allows me to import the text file and make it into a table.
The problem is that I have to import the text file and create the table. Then I add a column for the date, do
update table
set date='date'
then I can combine tables with an insert command. The do it again 3000 times.
Is there a better way?
Write a program that can open all files in directory.
Extend that program to parse ach files.
Extend that program to get date from file name.
Extend that program to write to database.
I'd say it's 3 hours of work.
We have a SQL SSIS package setup which is being run manually when needed. This will grab two CSV from the file system and process them to import data to sql table. This should be run once a month when the CSV file is updated.
Now I would like to automate this process. I want to add a step to the existing job that should check for the modified file dates (of two particular files) and compare it to the Today's date and if true, proceed to the next step otherwise not. I am not sure how to check for the modified date of the files (two files) on the file system and compare to today's date. Any thoughts on how to accomplish this? Thanks!!
You'll have to use a script task to pull the modified date in for comparison, using the following medthod:
File.GetLastWriteTime