I am in trouble with running SSIS packages in my etl process.
There is a table that includes etl steps with different types. i.e. Stored Procedure steps, cmd (copy-move files) steps, sending mail, SSIS steps.
A main Stored procedures runs these steps according to type of steps and logs begin-end times, errors etc. This main stored procedure is
triggered by an sql server job.
When I scheduled this job, SSIS steps takes error which is below;
Error: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has
occurred. Error code: 0x80040E4D. An OLE DB record is available.
Source: "Microsoft SQL Server Native Client 11.0" Hresult: 0x80040E4D
Description: "Login failed for user 'server\user$'.".
Yet, when I run this main procedure manually (with my windows user login) it runs successfully.
I checked this running with proxy account, it didn't work for me. Because inside the sql server job, i run my main stored procedure not SSIS package.
proxy account reference link;
https://www.mssqltips.com/sqlservertip/2163/running-a-ssis-package-from-sql-server-agent-using-a-proxy-account/
It should work so long as your connection to the database is valid. The error message is telling you that server\user$ is not a valid login.
If the SSIS Package is connecting to the DB with a username and password (i.e., a SQL user), then it should be straight forward to set up / configure the login to have access to the server and database where your proc resides, and when you next run the job it should be able to successfully execute the proc.
If instead the SSIS Package is connecting with Windows Authentication, then you'll need to use a Proxy that specifies a Credential that references a Windows / Active Directory account with database access. In SSMS:
Server >> Security >> Credentials
create a new credential
give it a name
specify a windows account that has permissions to execute your proc
Server >> SQL Server Agent >> Proxies >> SSIS Package Execution
create a new proxy
select the credential you just created
Select "SQL Server Integration Services Package" subsystem
Server >> SQL Server Agent >> Jobs >> Your Job >> SSIS Job Step
in the "Run as" field, pick the proxy you just created
Now when you kick off the job, your SSIS package should make a connection to the database using the windows account specified in the credential set up in step 1, which should be able to successfully execute the stored proc.
does your package of SSIS any password? if not,set password for it .then edit your job and write password there too.
If you have a password in your SSIS package, make sure that the ProtectionLevel is set to "EncryptSensitiveWithPassword". If you use "EncryptSensitiveWithUserKey" then you'll be the only user who can run it.
Related
I have a SSIS package that talks to a remote server over HTTP. I execute the SSIS package using a stored procedure in my database (SQL Server 2012), which is called from a web server. The web server connects to the database using Windows Authentication. I now have a need to run the stored procedure (and therefore, the SSIS package) from a client which does not support Windows Authentication. The SSIS package is complicated enough that migrating to a different solution is not feasible.
The SSIS package has complex variables that are passed. The stored procedure that runs the package looks something like:
CREATE PROCEDURE [dbo].[SSISPackage]
#Parameter1 XML
AS
BEGIN
SET NOCOUNT ON;
DECLARE #execution_id BIGINT
EXEC [SSISDB].[catalog].[create_execution]
#package_name=N'Package.dtsx',
#execution_id=#execution_id OUTPUT,
#folder_name=N'API',
#project_name=N'APIProject',
#use32bitruntime=False,
#reference_id=Null
EXEC [SSISDB].[catalog].[set_execution_parameter_value]
#execution_id,
#object_type=30,
#parameter_name=N'Parameter1',
#parameter_value=#Parameter1
EXEC [SSISDB].[catalog].[start_execution] #execution_id
END
From what I've been reading, it is not possible to run SSIS packages with users authenticated using SQL Server Authentication.
My questions are:
Is it possible to somehow elevate the SQL user to a Windows-auth user, and then execute the stored procedure.
Are there typical approaches in dealing with this problem (e.g. CLR, a queue table, command line call to package)?
This is what I did, maybe it can help someone.
First step,
I created an SQL Authentication login as sysadmin and I allowed this login to impersonate other logins.
And then, I inpersonate a Windows Authentication login that is sysadmin too before calling SSIS packages. To execute scripts as another login, I used TSQL EXECUTE AS. (check the example below)
For instance: server\winuser with Windows Authentication as sysadmin, and a SQL Authentication login sqlauthuser as sysadmin too.
Enable the "Inpersonate Any Login" for sqlauthuser. Login (or use a connection string to connect) as sqlauthuser then impersonate server\winuser to be able to call SSIS packages or procedures:
EXECUTE AS LOGIN = 'server\winuser';
-- call ssis packages
-- call procedures that uses ssis packages
Reference: https://learn.microsoft.com/en-us/sql/t-sql/statements/execute-as-transact-sql?view=sql-server-ver15
I don't think you can execute this using an SQL Server authentication, you will receive the following exception:
The operation cannot be started by an account that uses SQL Server Authentication. Start the operation with an account that uses Windows Authentication
There are many workaround that you can do, check the following links:
The operation cannot be started by an account that uses SQL Server Authentication. SSIS Package
Issues with module signing and SSIS catalog internal procedures
I have created a ssis package to load data from server 1 table1 to server 2 table2. This package is running fine inside the business intelligence studio. But the issue is , when I schedule this package in SQL server agent as a job, the package execution is failing.
The error I am getting is
SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E4D. Description: "Login failed for user 'Domain\Server1$' .
Description: SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER.
The AcquireConnection method call to the connection manager "Server2.Database2" failed with error code 0xC0202009.
Please help me to fix the issue. I am using SQL Server 2014
Thank you very much for your time and help!
This is a security / permissions issue. Update the SQL Server Agent job to use an account which has sufficient permissions to complete all of the actions inside the SSIS package.
Explanation
The package completes succesfully when executed from Business Intelligence Studio, because that execution is running under you credentials. It is running as you. When you run the package from SQL Service Agent job, it is NOT using your credentials. Instead it is using account 'Domain\Server1$' and that account does NOT have sufficient rights against Server2.Database2 to complete all of the package actions.
To solve this, you could update the security on Server2.Database2 and allow 'Domain\Server1$' to perform the same actions as you, but that is unadvised. Instead, best practice here is to establish an SSIS proxy account. Here is a good article discussing how to go about that. After you have established this proxy account, grant the new proxy account sufficient permissions to complete the actions inside of your SSIS package and your error message will go away. Hope this helps!
SSIS Job was able to run and create CSV files to shared location until last week when it was executed from scheduled job. The ssis package reports success but does not write file to shared location. When I run the package manually from visual studio it works fine. Please help
Permissions to the file share seem to be your issue.
When running the package as a scheduled job from SQL Agent. It is executed under the context of the service account for the SQL Agent Service. So make sure the the sql agent account has the appropriate read/write access to the file share, and/or set the job to execute as a user with enough permissions.
You need to give Modify permissions to the file share for the SQL Server Agent account. Executing an SSIS package in Visual Studio is executing it under the context of your account (or the account that you are logged onto the server with if doing it from a server). But when you use SQL jobs to execute the ssis package, then it runs under the context of the service account that SQL Server Agent is running under.
I am trying to move all .zip in a specific folder to another folder. the source folder is located on another server, currently i am using
EXECUTE xp_cmdshell 'copy \\server1\e$\ETL\*.zip \\server2\e$\ETL\'
GO
Which is working if I am logged into both server, but the goal is to automate this process VIA sql server job agent. I have tried
EXECUTE sp_xp_cmdshell_proxy_account 'domain\useracc','pass'
GO
EXECUTE xp_cmdshell 'copy \\server1\e$\ETL\*.zip \\server2\e$\ETL\'
GO
but I am receiving the following error;
An error occurred during the execution of sp_xp_cmdshell_proxy_account. Possible reasons: the provided account was invalid or the '##xp_cmdshell_proxy_account##' credential could not be created. Error code: '0'.
And also not sure if this is my solution. Please help with how I can achieve this. The file names on server1 change name and quantity everyday.
I would strongly advise...Do not use xp_cmdshell. It opens up large security wholes in your surface area and makes you vulnerable to attack. xp_cmdshell should be disabled!
Instead, if you want to automate this with server agent you have 2 options. My preference would be to write a simple SSIS package with a file system task and schedule this package with server agent. SSIS is underutilized for this kind of task but is actually pretty good at it.
Alternatively re-write your script to use Server Agent CmdExec job steps. This does not require xp_cmdshell to be enabled and reduces the attack surface.
I Found that the following worked for me;
In the command prompt, type services.msc, this would open the list of all services on the server.
In the list of services, look for SQL Server Agent, Right Click -> Properties. Go to Logon Tab
Change the logon to a user with access on both servers. then re-write your script to use Server Agent CmdExec job steps(Thank you Pete Carter)
I have a SSIS package (exports some database tables --> CSV file)
After creation of this package, I can run the package
from Visual Studio or just by clicking it.
But it fails if i add to execute this package as a SQL Agent Job scheduled at some arbitrary time of a day.
Pointers needed if i have to configure anything related to Permissions/Command line args ??
Thanks in advance
You don't give details of the error message received when the package is run by the Agent, but you need to check the rights of the Windows account under which the SQL Server Agent is running.
If all the package does is export database tables to text files you need to check that the service account has rights to:
connect to the database and run
select statements on the source
tables (if you are using Windows
Authentication to control the
database connection).
Create files
in the target location(s) in the
file system.
The windows account under which my SQL Server Agent runs belongs to sysadmin
Also the same user has admin privileges in my system to access the folders while exporting to the csv file.
The error message in the Job history :
Executed as user: MymachineName\JOBUSER. The package execution failed. The step failed.
In the step i have the SSIS Package included, with correct datasource connections,
Run as : SQL Agent Service Account (this is also run under MymachineName\JOBUSER user)