SSIS package failing - Working in BIDS but not working after deployment - Package contains excel connection with sql command [duplicate] - sql

I have a OLEDB task that connects to ms-access database on local file system and dumps data into sql database.
Access db connection string is: Data Source=\\my-share\accessdbfile.accdb;Provider=Microsoft.ACE.OLEDB.16.0;
The SSIS package runs OK in visual studio. Access 2016 32-bit runtime is installed on this computer.
After deployment to SSIS catalog, right click and execute works OK. Access 2016 64-bit runtime is installed on this computer.
When this package is scheduled to run via SQL Agent job, using proxy/credential [domain user - example: domainname\user1] (having read permission on the access db file), then the job fails. The SSIS execution report shows error as: Unexpected Termination. There is no further information.
Suppose if I add the proxy credential (domainname\user1) to the local administrators group (in computer management) then the SQL Job runs OK. What could be the solution to this so I can give only the minimum required permissions?

The error is very difficult to debug as it doesn't occur when running via visual studio. Even when deployed to the SQL server and run from the SSIS catalog, the execution works OK. The error occurs only when the package is run via SQL server agent job (via credential/proxy) - the SSIS execution report doesn't show any helpful messages; and surprisingly the error gets resolved when the credential is added as local administrator on the computer - which is a bad practice.
The solution is analyse what kind of ms access file you are using and appropriately use the correct access runtime version (2013 vs 2016) and the correct provider value in the connection string.
Access runtime:
In my case, I uninstalled Access 2016 64-bit runtime and installed Access 2013 64-bit runtime.
Provider version:
I changed the provider version from 16.0 to 15.0, as shown below-
Before: Data Source=\\my-share\accessdbfile.accdb;Provider=Microsoft.ACE.OLEDB.16.0;
After: Data Source=\\my-share\accessdbfile.accdb;Provider=Microsoft.ACE.OLEDB.15.0;
Note:
There is no need to add the user to the local administrator group.
There is no need to login to the computer using that account.
Only read permission (under windows folder security) is sufficient assuming the SSIS packge will only read the ms access database.

It's hard to debug this only with the information provided, but in general:
The user running the job requires at least write and delete access on the underlying folder in which the Access database resides. This is required to generate a lockfile for the database.
If this is undesirable because the job may be modified by a malicious user, I've seen deployments where a copy is made in a temporary folder with write access and a size quota, and that temporary folder is removed as soon as the job finishes, regardless of if the job succeeded.

Related

SSIS flat file folder permission error when NOT running from SQL Server Agent

Setup: A pretty standard data export SSIS package (SQL Server 2016 compatible), created in VS2019/Data Tools and deployed using the SSIS Project Deployment model to the Integration Services Catalog of a SQL Server 2016 instance. The package creates files in a network folder before sending the file out via FTP and putting a copy of the file in a Sent folder.
The project requirements include having the package running on a schedule using "default" parameter values, as well as allowing users to manually run the package using "non-default" parameter values from within a stand-alone application.
Current behavior: the package behaves correctly when run from a SQL Server Agent Job that is configured with a SQL proxy and credentials mapped to a domain login with the proper permissions for the network folder.
Problem: the Data Flow task fails to create the file with a "Cannot open the datafile" error when running the package directly using any of the following methods (even when the "current" session is using the same credentials as the SQL Server Credentials/Proxy used by the SQL Server Agent Job):
Using SSMS to right-click on the package and selecting Execute
Using the DTEXEC SQL utility
Using the SSISDB.catalog.start_execution SQL Server stored procedure
As far as I'm aware, these are the only methods capable of starting a SSIS package and changing the package's parameter values. I either need to get one of the latter 2 methods to work, find another option that allows for changing the parameter values while launching the package, or use one of 2 techniques I'm aware of (detailed below) that would add yet another failure point to the process as well as other potential issues.
Note: If the process is changed to initially create the file on the SQL Server's local harddrive, then the Data Flow task succeeds, but the later copy to Sent folder task fails with a very similar permissions error.
Alternative #1: this technique requires creating a new table, loading the parameter values to the table, changing the package to check the table and potentially set it's parameters/variables based on what it finds. The package can then be launched using a SQL Server Agent Job (for which there are multiple methods to manually launch them) and if the calling object has correctly populated the table, the package will behave as if it's parameters were changed at runtime otherwise it will run with the default values.
Alternative #2: Change all folders used by the package to point to folders local to the SQL Server instance and then create a separate scheduled task/application/whatever, with the valid credentials, that would synchronize or move the files to their proper network folders.
even when the "current" session is using the same credentials as the SQL Server Credentials/Proxy used by the SQL Server Agent Job
This is probably because the account is not logged on locally at the SQL Server, and so it's a Double-Hop Impersonation scenario, and would require Kerberos Constrained Delegation to be configured.
And you are correct in assessing the options. The general solution is to invoke catalog.start_execution from a session running on the SQL Server, and an Agent Job is the simplest built-in way to do this (the others being xp_cmdshell, Service Broker Activation, or SQL CLR).

SSIS: Unable to Read from Excel file from a UNC path when deployed to server

I appear to be unable to get my deployed SSIS package to read from a 1997-2003 Excel file.
I get stuck with the following 2 errors:
SSIS Error Code DTS_E_OLEDBERROR, Error code: 0x80004005 An OLE DB
record is available. Source Microsoft JET Database Engine, database
engine cannot open the file, it is already opened exclusively by
another user, or you need permission to view its data.
DTS_E_CANNOTAQUIRECONNECTIONFROMCONNECTIONMANAGER with Error code 0xC0202009
The package itself works with no issues when using SQL data tools on the server logged in as the service account only when deployed to server I see this issue.
To make this issue even more confusing I have setup and tried the following:
Runtime is set to 32 bit on the debugging menu on SQL data tools as well as the job configuration is set to use 32 bit runtime
I am able to UNC to the folder/file when logged in as the SQL service account
Any Ideas?
Thanks,
M
I figured it out in the end. Turns out I missed a step with the permissions.
Yes the Service account had permissions to the folder but I failed to add read permissions to the file itself.
Thanks for all comments/suggestions.
M

DTSX package runs in Visual Studio but not when called from a Database Job

I have an SSIS package that takes database backups and it runs fine in visual studio but when executed from a database job it fails with the following error. The part I can't understand is that 'GS\BOS-DBMONITOR$' isn't a user. 'BOS-DBMONITOR' is the server name.
Error:
Executed as user: NT Service\SQLSERVERAGENT. Microsoft (R) SQL Server Execute Package Utility Version 11.0.2100.60 for 64-bit Copyright (C) Microsoft Corporation. All rights reserved. Started: 10:58:03 AM Error: 2014-07-30 10:58:04.33 Code: 0xC0024104 Source: Back Up Database Task Description: The Execute method on the task returned error code 0x80131904 (Login failed for user 'GS\BOS-DBMONITOR$'.). The Execute method must succeed, and indicate the result using an "out" parameter. End Error DTExec: The package execution returned DTSER_FAILURE (1). Started: 10:58:03 AM Finished: 10:58:04 AM Elapsed: 0.593 seconds. The package execution failed. The step failed.
SQL Server Agent is the scheduler that comes with SQL Server. The first line of your error, Executed as user: NT Service\SQLSERVERAGENT specifies that your instance is running as the system user - not a "real" account. This machine is then trying to reach out to another computer and asks to log in to that SQL Server instance. SQL Server says "I'm not allowed to talk to strangers and GS\BOS-DBMONITOR$', you're strange."
How do you resolve it?
You need to let the foreign machines know about the account that's going to talk to them.
Use SQL Server Configuration Manager to change the service account from the the local account to a domain account MyDomain\NonExpiringAccount
Create a Credentialed user in the SQL Server instance on BOS-DBMONITOR and then assign them as a proxy for executing this, and any similar, SSIS packages. This allows the credentialed users' account to be presented to the foreign hosts. This has less potential impact on existing operations than changing the service account but requires more work
There's probably some voodoo you can work on all the servers you need to take a backup on to allow this user to log on but I wouldn't even go down that road. It's going to be work you have to perform on a per box basis and there's probably some security implications that go over my head

SQL Server Database is not getting attached again

I tried to detach my SQL Server 2008 R2 database for creating a duplicate of it, meanwhile connections to that database were active from Visual Studio (2010). So SQL Server denied to detach. After closing Visual Studio, I tried again and my database was successfully detached.
After detaching my database I tried to copy both files (.mdf and .ldf) to another folder as a duplicate copy, but I could not do so because I was getting error that says
You require permission from the computer administrator to make changes to this file
I am using Windows 7 and my account is Administrator. I also tried to attach my database again to SQL Server (running it with administrator permission) but I got the following error:
Failed to retrieve data for this request. (Microsoft.SqlServer.Management.Sdk.Sfc)
Additional Information:
An exception occurred while executing a Transact-Sql statement or batch.
(Microsoft.SqlServer.ConnectionInfo)
CREATE FILE encountered operating system error 5 (Access is denied,) while attempting to open or create the phyical file'C:/Mywor/MyDataBase.mdf'.
(Microsoft SQL Server, Error: 5123)
Please help me if someone know how to handle this problem.
Here are couple things you can try:
Stop SQL Server service and see if you can copy files (to stop SQL service go to SQL Server Configuration manager)
Try to copy some other database files in that directory and see if that works (make sure permissions for both files are exactly the same)
Try to copy .mdf and .ldf files in original SQL Server data directory (you can find it in SSMS -> Instance properties ->Database Settings
If you’re not using windows explorer for copying files make sure you run it as admin (right click -> Run as administrator)
This is a permissions issue on the database files directory, or the files themselves. Just because you're logged on as an administrator doesn't mean that Windows 7 allows you to do anything you want without whinging at you first. Also, the SQL Server service account may not have the appropriate permissions on the file/folder.

SSIS Package runs from Windows, but fails when run from SQL Agent jobs

I have a SSIS package (exports some database tables --> CSV file)
After creation of this package, I can run the package
from Visual Studio or just by clicking it.
But it fails if i add to execute this package as a SQL Agent Job scheduled at some arbitrary time of a day.
Pointers needed if i have to configure anything related to Permissions/Command line args ??
Thanks in advance
You don't give details of the error message received when the package is run by the Agent, but you need to check the rights of the Windows account under which the SQL Server Agent is running.
If all the package does is export database tables to text files you need to check that the service account has rights to:
connect to the database and run
select statements on the source
tables (if you are using Windows
Authentication to control the
database connection).
Create files
in the target location(s) in the
file system.
The windows account under which my SQL Server Agent runs belongs to sysadmin
Also the same user has admin privileges in my system to access the folders while exporting to the csv file.
The error message in the Job history :
Executed as user: MymachineName\JOBUSER. The package execution failed. The step failed.
In the step i have the SSIS Package included, with correct datasource connections,
Run as : SQL Agent Service Account (this is also run under MymachineName\JOBUSER user)