"Uninitialized subscription" error in replication monitor - replication

I'm using SQL Server 2012 and trying to implement transactional replication. Im using the system stored procedures to create the publications and subscriptions. I was successful in creating these things, but when i check the replication monitor, it shows "Uninitialized subscription".
When I check the synchronization status on the subscription, i found this log
Date 6/20/2012 7:36:33 PM
Log Job History (HYDHTC0131320D-PublisherDB-PublicationOne-HYDHTC0131320D\MSS-ReplicationSubscri-7C1D7509-C8A6-4073-A901-0433A2B6D2D3)
Step ID 1
Server HYDHTC0131320D\MSSQLSERVER2
Job Name HYDHTC0131320D-PublisherDB-PublicationOne-HYDHTC0131320D\MSS-ReplicationSubscri-7C1D7509-C8A6-4073-A901-0433A2B6D2D3
Step Name Run agent.
Duration 00:07:41
Sql Severity 0
Sql Message ID 0
Operator Emailed
Operator Net sent
Operator Paged
Retries Attempted 0
Message
2012-06-20 14:14:13.986 Copyright (c) 2008 Microsoft Corporation
2012-06-20 14:14:13.986 Microsoft SQL Server Replication Agent: distrib
2012-06-20 14:14:13.986
2012-06-20 14:14:13.986 The timestamps prepended to the output lines are expressed in terms of UTC time.
2012-06-20 14:14:13.986 User-specified agent parameter values:
-Publisher HYDHTC0131320D
-PublisherDB PublisherDB
-Publication PublicationOne
-Distributor HYDHTC0131320D
-SubscriptionType 2
-Subscriber HYDHTC0131320D\MSSQLSERVER2
-SubscriberSecurityMode 1
-SubscriberDB ReplicationSubscriberDB
-Continuous
-XJOBID 0xDFE51AEC7F9E3F42A450CE8874B662CD
-XJOBNAME HYDHTC0131320D-PublisherDB-PublicationOne-HYDHTC0131320D\MSS-ReplicationSubscri-7C1D7509-C8A6-4073-A901-0433A2B6D2D3
-XSTEPID 1
-XSUBSYSTEM Distribution
-XSERVER HYDHTC0131320D\MSSQLSERVER2
-XCMDLINE 0
-XCancelEventHandle 000005F8
-XParentProcessHandle 00000560
2012-06-20 14:14:13.986 Startup Delay: 619 (msecs)
2012-06-20 14:14:14.606 Connecting to Subscriber 'HYDHTC0131320D\MSSQLSERVER2'
2012-06-20 14:14:14.656 Connecting to Distributor 'HYDHTC0131320D'
2012-06-20 14:14:14.671 Parameter values obtained from agent profile:
-bcpbatchsize 2147473647
-commitbatchsize 100
-commitbatchthreshold 1000
-historyverboselevel 1
-keepalivemessageinterval 300
-logintimeout 15
-maxbcpthreads 1
-maxdeliveredtransactions 0
-pollinginterval 5000
-querytimeout 1800
-skiperrors
-transactionsperhistory 100
2012-06-20 14:14:14.683 Agent message code 21040. Publication '' does not exist.
How do I solve this issue?

I received the same error. My fix was to explicitly define the Job_login and job_password which I had as null to start with.
EXEC sp_addpullsubscription_agent
#publisher = #publisher,
#publisher_db = #publicationDB,
#publication = #publication,
#distributor = #publisher,
#job_login = $(Login),
#job_password = $(Password);

It seems there is an error in your replication setup scripts.
I suspect the error is in the call to sp_addpushsubscription_agent (if it is push subscription) or sp_addpullsubscription_agent (if it is pull subscription). Specifically the #publication parameter is wrong as the Distribution Agent is stating that the specified Publication " does not exist.
Please review your script and try again.

I faced same problem and fixed by doing the following :
-Subscriber Job owner is same as Publication User
-Subscriber User has been added to the subscriber users list and added
to the sysadmin server role

Just a few notes as I managed to get a pull subscriber up and running with a backup initialisation:
Make sure you have the agent account of the publisher, distributor and subscriber as sysadmin logins across all instances, example subscriber must be on the distributor and publisher.
Create linked servers for all components on each instance, example distributor and publisher on the subscriber.
Then take a full backup of the source database and restore without recovery on the subscription instance.
Create the publication component on the publisher database (sp_addsubscription) and make sure you have #sync_type = N'replication support only'.
Then take a diff backup of the source database and restore with recovery on the subscription instance.
Then create the pull subscription on the subscription instance (sp_addpullsubscription), note that if the source is a mirrored instance, use the original instance value here (#publisher). If you the principal on the "mirror" instance, meta data will be created correct and the distributor contains the logic to connect the log reader to either of the mirror instances.
Now you will have a problem with primary keys etc failing as the distributor started capturing data before the differential backup ... no problem. Stop the subscriber job and then add a new profile to the distributor agent with setting the -SkipErrors parameter to "2601:2627". This will skip all primary key violation transactions and continue processing. Remember to select this agent profile then and hit "OK".
Restart the subscription job and monitor the subscription as it start to catch up on the transactions.
When caught up, stop the job, change the agent profile back to the default and restart the job.
Hope this help anyone still struggling with pull subscribers based on backup initialisation ... note that at no point did I use any backup configurations within the replication configuration. Also stated that the pull subscription must not initialise (#immediate_sync = 0).
Here is the scripts:
-----------------BEGIN: Script to be run at Publisher 'DB001\OLTP'-----------------
use [DB1]
go
exec sp_addsubscription #publication = N'DB1', #subscriber = N'DB002\OLTP', #destination_db = N'DB1', #sync_type = N'replication support only', #subscription_type = N'pull', #update_mode = N'read only'
GO
-----------------END: Script to be run at Publisher 'DB001\OLTP'-----------------
-----------------BEGIN: Script to be run at Subscriber 'DB002\OLTP'-----------------
use [DB1]
exec sp_addpullsubscription #publisher = N'DB001\OLTP', #publication = N'DB1', #publisher_db = N'DB1', #independent_agent = N'True', #subscription_type = N'pull', #description = N'', #update_mode = N'read only', #immediate_sync = 0
exec sp_addpullsubscription_agent #publisher = N'DB001\OLTP', #publisher_db = N'DB1', #publication = N'DB1', #distributor = N'DB003\DIST', #distributor_security_mode = 1, #distributor_login = N'', #distributor_password = null, #enabled_for_syncmgr = N'False', #frequency_type = 64, #frequency_interval = 0, #frequency_relative_interval = 0, #frequency_recurrence_factor = 0, #frequency_subday = 0, #frequency_subday_interval = 0, #active_start_time_of_day = 0, #active_end_time_of_day = 235959, #active_start_date = 20170327, #active_end_date = 99991231, #alt_snapshot_folder = N'\\DB001\Replication', #working_directory = N'', #use_ftp = N'False', #job_login = null, #job_password = null, #publication_type = 0
GO
-----------------END: Script to be run at Subscriber 'DB002\OLTP'-----------------

Related

PHP PDO MSSQL - Aborting DB transaction in query which raisses error with severity 0 (warnings)

I am creating web app in Laravel which connects to MSSQL DB (using PDO) and i'm stuck with a issue when running query (UPDATE in my case)
update dba.[vl_zahlavi] set [xcmd] = ' AFTERCOMMIT', [vl_zahlavi].[ts] = '2023-01-10 14:16:06.505' where [doklad] = '23VLTU0100000001'
which raises error with severity 0 within update trigger (used for sending progress bar information to client). Trigger is created within information system, so I'm not able to disable this functionality.
select #progress_count = convert(int/*BY*/,#akt_pocet), #progress_current = 0, #progress_dt_o_skl_o_skl_gen_poh_prub = DateAdd(ms, -1000, GetDate())
select #progress_arg = '#SQL_PROGRESS=O_SKL,O_SKL_GEN_POH_PRUB,' + convert(varchar(255)/*BY*/, ##nestlevel) + ',OPEN,' + convert(varchar(255)/*BY*/, #progress_count) + ';'
RAISERROR ('%s', 0, 1, #progress_arg) WITH NOWAIT
Problem is, that PHP client send abort and transaction is automatically rolled back. When I comment this section of code (or change severity to 16), everything works as expected. Bellow is printscreen from SQL server profiler.
printscreen
Is PDO able to catch user error messages?
Thank you in advance.
I have tried to change DB sets and PDO options, but without success.

Executing SQL Script causes error in procedure called by trigger

I'm also new to DevOps, but let me see if I can explain my situation, also if anyone has any advice on doing post deployment scripts and can share their experiences I will be most grateful.
I use SQL Source control from Redgate, Git and Azure DevOps. I have managed to get the build and deployment working perfectly. Additionally we would like to make some data changes so we have additional (Data Change) scripts we wish to run. Linking these tables as static data is not an option as we would end up having most tables linked and end up with extremely timeous build and deploy times.
The script inserts data to a table, thereby triggering the insert trigger which calls a procedure to write to an audit table. The actual error from the deployment log is:
Task : DbUp Migration
2021-01-25T12:52:17.2629353Z Description : Runs SQL Server change scripts, and only those which have not been run already.
2021-01-25T12:52:17.2629405Z Version : 2.1.4
2021-01-25T12:52:17.2629671Z Author : Johan Classon
2021-01-25T12:52:17.2629721Z Help : [More Information](https://github.com/johanclasson/vso-agent-tasks)
2021-01-25T12:52:17.2629775Z ==============================================================================
2021-01-25T12:52:19.6610608Z Beginning database upgrade
2021-01-25T12:52:19.6689401Z Checking whether journal table exists..
2021-01-25T12:52:19.6728756Z Journal table does not exist
2021-01-25T12:52:19.7775348Z Executing Database Server script '001 - EX27605 - rdl.sql'
2021-01-25T12:52:19.7908913Z Checking whether journal table exists..
2021-01-25T12:52:19.7929056Z Creating the [dbo].[_SchemaVersions] table
2021-01-25T12:52:19.8077238Z The [dbo].[_SchemaVersions] table has been created
2021-01-25T12:52:20.2264323Z ifExists - rdl_Rule_Definition_Lookup - rdl_Code = "9900226e"
2021-01-25T12:52:20.2277308Z insert - rdl_Rule_Definition_Lookup - rdl_Code = "9900226e"
2021-01-25T12:52:20.2805780Z SQL exception has occured in script: '001 - EX27605 - rdl.sql'
2021-01-25T12:52:20.2942912Z ##[error]Script block number: 0; Block line 74; Message: Trig_After_Ins_Upd_Del_rdl_Rule_Definition_Lookup
2021-01-25T12:52:20.4528752Z ##[error]System.Data.SqlClient.SqlException (0x80131904): p_dte_Audit_Backend: (Line: 108) [dte_admin]
So the 1st error refers to line 74 of the trigger, if I read this correctly which is a commit on a procedure call to write to the audit table, line 108 of my procedure sets the userID:
DECLARE #user VARCHAR(40)
SET #user =
CASE
WHEN suser_sname() = 'NT SERVICE\SQLAgent$'+##servicename THEN 'dte_admin'
WHEN suser_sname() = 'NT SERVICE\SQLSERVERAGENT' THEN 'dte_admin'
ELSE suser_sname()
END
--###############
select #udt_Audit = udt_Audit
from udt_User_Detail (nolock)
where udt_User_Id = #user--suser_sname()
--select #udt_Audit = udt_Audit from udt_User_Detail
--where udt_User_Id = suser_sname()
if ##rowcount = 0
begin
declare #udt_User_Id varchar(30)
set #udt_User_Id = suser_sname()
The last line being line 108.
Important to note that dte_admin is a login on the SQL Server, it is a sysadmin account and mapped to the database, the dte_admin user is also a user at database level and exists as a user in my user table udt_user_detail.
During the build and deployment I am using variables and they are set to dte_admin username and password.
My question is then, why does this fail with this user?
Deployment Pipeline result

SQL queries returning incorrect results during High Load

I have a table in which during the performance runs, there are inserts happening in the beginning when the job starts, during the insertion time there are also parallel operations(GET/UPDATE queries) happening on that table. The Get operation also updates a value in column marking that record as picked. However, the next get performed on the table would again return back the same record even when the record was marked in progress.
P.S. --> both the operations are done by the same single thread existing in the system. Logs below for reference, record marked in progress at Line 1 on 20:36:42,864, however, it is returned back in the result set of query executed after 20:36:42,891 by the same thread.
We also observed that during high load (usually during same scenario as mentioned above) some update operation (intermittent) were not happening on the table even when the update executed successfully (validated using the returned result and then doing a get just after that to check the updated value ) without throwing an exception.
13 Apr 2020 20:36:42,864 [SHT-4083-initial] FINEST - AbstractCacheHelper.markContactInProgress:2321 - Action state after mark in progresss contactId.ATTR=: 514409 for jobId : 4083 is actionState : 128
13 Apr 2020 20:36:42,891 [SHT-4083-initial] FINEST - CacheAdvListMgmtHelper.getNextContactToProcess:347 - Query : select priority, contact_id, action_state, pim_contact_store_id, action_id
, retry_session_id, attempt_type, zone_id, action_pos from pim_4083 where handler_id = ? and attempt_type != ? and next_attempt_after <= ? and action_state = ? and exclude_flag = ? order
by attempt_type desc, priority desc, next_attempt_after asc,contact_id asc limit 1
This happens usually during the performance runs when there are parallel JOB's started which are working on Ignite. Can anyone suggest what can be done to avoid such a situation..?
We have 2 ignite data nodes that are deployed as springBootService deployed in the cluster being accessed, by 3 client nodes.
Ignite version -> 2.7.6, Cache configuration is as follows,
IgniteConfiguration cfg = new IgniteConfiguration();
CacheConfiguration cachecfg = new CacheConfiguration(CACHE_NAME);
cachecfg.setRebalanceThrottle(100);
cachecfg.setBackups(1);
cachecfg.setCacheMode(CacheMode.REPLICATED);
cachecfg.setRebalanceMode(CacheRebalanceMode.ASYNC);
cachecfg.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL);
cachecfg.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);
// Defining and creating a new cache to be used by Ignite Spring Data repository.
CacheConfiguration ccfg = new CacheConfiguration(CACHE_TEMPLATE);
ccfg.setStatisticsEnabled(true);
ccfg.setCacheMode(CacheMode.REPLICATED);
ccfg.setBackups(1);
DataStorageConfiguration dsCfg = new DataStorageConfiguration();
dsCfg.getDefaultDataRegionConfiguration().setPersistenceEnabled(true);
dsCfg.setStoragePath(storagePath);
dsCfg.setWalMode(WALMode.FSYNC);
dsCfg.setWalPath(walStoragePath);
dsCfg.setWalArchivePath(archiveWalStoragePath);
dsCfg.setWriteThrottlingEnabled(true);
cfg.setAuthenticationEnabled(true);
dsCfg.getDefaultDataRegionConfiguration()
.setInitialSize(Long.parseLong(cacheInitialMemSize) * 1024 * 1024);
dsCfg.getDefaultDataRegionConfiguration().setMaxSize(Long.parseLong(cacheMaxMemSize) * 1024 * 1024);
cfg.setDataStorageConfiguration(dsCfg);
cfg.setClientConnectorConfiguration(clientCfg);
// Run the command to alter the default user credentials
// ALTER USER "ignite" WITH PASSWORD 'new_passwd'
cfg.setCacheConfiguration(cachecfg);
cfg.setFailureDetectionTimeout(Long.parseLong(cacheFailureTimeout));
ccfg.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL);
ccfg.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);
ccfg.setRebalanceMode(CacheRebalanceMode.ASYNC);
ccfg.setRebalanceThrottle(100);
int pool = cfg.getSystemThreadPoolSize();
cfg.setRebalanceThreadPoolSize(2);
cfg.setLifecycleBeans(new MyLifecycleBean());
logger.info(methodName, "Starting ignite service");
ignite = Ignition.start(cfg);
ignite.cluster().active(true);
// Get all server nodes that are already up and running.
Collection<ClusterNode> nodes = ignite.cluster().forServers().nodes();
// Set the baseline topology that is represented by these nodes.
ignite.cluster().setBaselineTopology(nodes);
ignite.addCacheConfiguration(ccfg);

AWS RDS PostgreSQL error "remaining connection slots are reserved for non-replication superuser connections"

In the dashboard I see there are currently 22 open connections to the DB instance, blocking new connections with the error:
remaining connection slots are reserved for non-replication superuser connections.
I'm accessing the DB from web service API running on EC2 instance and always keep the best practise of:
Connection connection = DriverManager.getConnection(URL, USER_NAME, PASSWORD);
Class.forName(DB_CLASS);
Statement statement = connection.createStatement();
ResultSet resultSet = statement.executeQuery(SQL_Query_String);
...
resultSet.close();
statement.close();
connection.close();
Can I do something else in the code?
Should I do something else in the DB management?
Is there a way to periodically close connections?
Amazon has to set the number of connections based on each model's right to demand a certain amount of memory and connections
MODEL max_connections innodb_buffer_pool_size
--------- --------------- -----------------------
t1.micro 34 326107136 ( 311M)
m1-small 125 1179648000 ( 1125M, 1.097G)
m1-large 623 5882511360 ( 5610M, 5.479G)
m1-xlarge 1263 11922309120 (11370M, 11.103G)
m2-xlarge 1441 13605273600 (12975M, 12.671G)
m2-2xlarge 2900 27367833600 (26100M, 25.488G)
m2-4xlarge 5816 54892953600 (52350M, 51.123G)
But if you want you can change the max connection size to custom value by
From RDS Console > Parameter Groups > Edit Parameters,
You can change the value of the max_connections parameter to a custom value.
For closing the connections periodically you can setup a cron job some thing like this.
select pg_terminate_backend(procpid)
from pg_stat_activity
where usename = 'yourusername'
and current_query = '<IDLE>'
and query_start < current_timestamp - interval '5 minutes';
I'm using Amazon RDS, SCALA, Postgresql & Slick. First of all - number of available connections in RDS depends on the amount of available RAM - i.e. size of the RDS instance. It's best not to change the default conn number.
You can check the max connection number by executing the following SQL statement on your RDS DB instance:
show max_connections;
Check your SPRING configuration to see how many threads you're spawning:
database {
dataSourceClass = org.postgresql.ds.PGSimpleDataSource
properties = {
url = "jdbc:postgresql://test.cb1111.us-east-2.rds.amazonaws.com:6666/dbtest"
user = "youruser"
password = "yourpass"
}
numThreads = 90
}
All of the connections ARE made upon SRING BOOT initialization so beware not to cross the RDS limit. That includes other services that connect to the DB. In this case the number of connections will be 90+.
The current limit for db.t2.small is 198 (4GB of RAM)
You can change in the parameter group idle_in_transaction_session_timeout to remove idle connections.
idle_in_transaction_session_timeout (integer)
Terminate any session with an open transaction that has been idle for
longer than the specified duration in milliseconds. This allows any
locks held by that session to be released and the connection slot to
be reused; it also allows tuples visible only to this transaction to
be vacuumed. See Section 24.1 for more details about this.
The default value of 0 disables this feature.
The current value in AWS RDS is 86400000 which when converted to hours (86400000/1000/60/60) is 24 hours.
you can change the max connections in the Parameters Group for your RDS instance. Try to increase it.
Or you can try to upgrade your instance, as the max connexions is set to {DBInstanceClassMemory/31457280}
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_WorkingWithParamGroups.html

Azure database copy, wait for ready state

I am making new Azure databases now and then and without any human interaction, by using a template database and azure copy database t-sql script.
After reading a few caveats I wonder what is the best way to wait for the copied database to be ready.
I have posted my own answer but it could end up in an infinite loop if the database copy fails.
What is the best way to be sure that the copying is done. This is very important as you are only allowed one copy database command at a time. I'm using an Azure worker with a c# queue that creates the databases as nessesary.
This is how I'm doing it now. This should work but unfortunatly errors when copying are rare.
declare
#sName varchar(max),
#sState varchar(max),
#iState int,
#sDbName varchar(50)
set #sDbName = 'myDbName'
set #iState = 7 --copying
while #iState = 7
begin
WAITFOR DELAY '00:00:05'
SELECT #sName = name, #iState = state, #sState = state_desc FROM sys.databases WHERE name = #sDbName
end
SELECT name, state,state_desc FROM sys.databases WHERE name = #sDbName
This code is actually run from a Azure worker in c#
Edit:
If you want more granular approch use sys.dm_database_copies
But otherwise remember that this sql returns the state as integer, and its only successfull if the state ==0 Othervise the copy has failed.
the states are:
0 = ONLINE
1 = RESTORING
2 = RECOVERING
3 = RECOVERY_PENDING
4 = SUSPECT
5 = EMERGENCY
6 = OFFLINE
7 = COPYING (Applies to Windows Azure SQL Database)
Windows Azure SQL Database returns the states 0, 1, 4, and 7.