I would like to create this in SSIS package, wherein, it will loop in the list of files in a directory or folder and check each files if it exist in a SQL queried table. If the filename does not exist in the table, it will be archive in a different folder.
Thanks is advance for any help! =)
You could use a For each Loop container that can be pointed to the said directory or folder and then store the file name (Name and Extension, Fully Qualified or Name only depending on the requirement) in a variable eg. LoopVariable. Use a Execute SQL task inside the container that has a prepared statement like -
If exists (select * from [Files] where [FileName] = ?)Select 'Yes'
Else Select 'No' -
In the Parameter pass the LoopVariable and Result map to another variable FileExists. In the precedence Constraint editor use - expression - [User::FileExists] =="No" and connect to a file system task that moves the file to the archive folder.
Note: You might need to form the file path in case you are not getting the fully qualified name from the For loop container.
Related
Start situation:
a folder with lots of files (images mostly png).
two tables in database (MariaDB) which contains supposed image filenames. I query the filenames like this:
select filename from table1
UNION
select filename from table2;
I want to know if I have files not registered in the database tables.
My first approach is to put the list of filenames inside a textfile (I've used Linux command line, list is filename per line), but I don't know how to continue.
I can't write in the database. UPDATED. I got more auth to perform my job tasks. Therefore I solved this with the suggestion.
You need to do couple of things:
Import the file with list of files into the database.
Use cursor to go through the list of file names and match it to your table list which contains the list-b of file names
To import file name you can use the import method or directly read the file as a table virtually.
Thanks
I want to read a tab delimited file using PLSQL and insert the file data into a table.
Everyday new file will be generated.
I am not sure if external table will help here because filename will be changed based on date.
Filename: SPRReadResponse_YYYYMMDD.txt
Below is the sample file data.
Option that works on your own PC is to use SQL*Loader. As file name changes every day, you'd use your operating system's batch script (on MS Windows, these are .BAT files) to pass a different name while calling sqlldr (and the control file).
External table requires you to have access to the database server and have (at least) read privilege on its directory which contains those .TXT files. Unless you're a DBA, you'll have to talk to them to provide environment. As of changing file name, you could use alter table ... location which is rather inconvenient.
If you want to have control over it, use UTL_FILE; yes, you still need to have access to that directory on the database server, but - writing a PL/SQL script, you can modify whatever you want, including file name.
Or, a simpler option, first rename input file to SPRReadResponse.txt, then load it and save yourself of all that trouble.
I am creating an SSIS package which needs to selectively loop through excel files in a folder that contain a specific keyword in their name (the keyword can appear anywhere in the name). This folder will include multiple files that do not include this keyword.
Bonus: I then need to use that same file name to create a date variable in SSIS (the file name will have the required components to create the date inside it). So far my efforts have failed, any help would be appreciated!
Create a Foreach File Loop Enumertor. In the FileSpec, use a wildcard, such as:
*SearchString*.csv or *SearchString*.xslx
You will probably want to select the Filename and Extension radio button. Map that to an SSIS package variable.
For the bonus, you can create another package variable with an expression based on the mapped variable and perform whatever string parsing you need in order to extract information from the file name.
I've few excel source files in one folder in SSIS. I want to pull data from these excel files and load in to SQL tables.
My problem is I want to save all the files names one by one and want to create SQL table with exactly same name as filename
and then want to load each excel file in corresponding table.
Please help me how to create a package for this.
Jayvee has presented the high level view which is good enough! Let me add in bit detail.
I am assuming that you have dynamic Excel file connection.
Declare a variable and named it as FileName. And assign it the first file name which is available in the folder.
Place Foreach Loop Container and double click on it. Specify the Folder: and Files: as shown in image below.
In the same Foreach Loop Editor, go to Variable Mappings. Select Variable from drop down list. This is the same variable which we defined in first step. Set its Index to 0. Click OK.
Remaining task is same as Jayvee explained.
See this link for further help. And this for Result Set Property Not Set Correctly. I think setting ResultSet property to SingleRow will do the job.
your control flow should look like this:
I wanted to insert data in a table from a text file where it is stored in csv format to a sql server table. For that, I am using bulk-insert statement. Now I need to specify the file name in "From" clause. I don't want to use networked locations or local locations over there. I want to upload my text file in the same directory as my executable file and give a relative path to it. Is it possible???
I don't think so. I just tried
SELECT BulkColumn
FROM OPENROWSET(BULK'files\doesnotexist.txt',SINGLE_BLOB)x;
And looked at it in Process Monitor
And it was looking for C:\WINDOWS\system32\files\doesnotexist.txt so I think you would need to put in the full path.