How to get the list of queries executed in Azure Synapse Dedicated SQL Pool by user?
You can get the query details by either using Azure Query Store service which automatically captures a history of queries, plans, and runtime statistics, and retains these for your review or you can create your custom table in your database to capture all the required details.
Only Synapse Dedicated pool supports Query Store but it isn't enabled by default for new Azure Synapse Analytics databases.
Use the ALTER DATABASE statement to enable the query store for a given database. For example:
ALTER DATABASE <database_name>
SET QUERY_STORE = ON (OPERATION_MODE = READ_WRITE);
The following query returns information about queries and plans in the Query Store.
SELECT Txt.query_text_id, Txt.query_sql_text, Pl.plan_id, Qry.*
FROM sys.query_store_plan AS Pl
INNER JOIN sys.query_store_query AS Qry
ON Pl.query_id = Qry.query_id
INNER JOIN sys.query_store_query_text AS Txt
ON Qry.query_text_id = Txt.query_text_id;
On the other side, if you want to create a table to track the history of all the queries, you can refer the answer given by SqlWorldWide on this similar thread.
Related
Use case-
After learning that AD Passthrough is not working as expected on Synapse Serverless pool with ADLS Gen-2 ; I am trying to use traditional method of creating external tables on Serverless Pool and granting READ ONLY access to users to a set of tales and enable WRITE BACK option to another ADLS Gen-2 container using CETAS option .
Looks like I am stuck there as well - to move forward.
I have tried to explain my scenario in below image.
Now - I have 5 external tables on a database where I have a READ ONLY access to the schema's where those table exists.
I wanted to create few more tables - which ideally does a JOIN between those 5 tables and aggregates the data and writes back to ADLS Gen-2 for reporting/data science purpose.
What access should I grant for WRITE back purpose ?
I tried creating new schema and granting ALTER, CONTROL, SELECT access to that schema along with CREATE TABLE access at database level . I dont want to grant more access to database level - as it has data scoped credential having managed identity referenced- which will grant full access on ROC container objects.
Grant select on SCHEMA ::sandbox to sls_svc ;
Grant ALTER on SCHEMA ::sandbox to sls_svc ;
GRANT CONTROL ON SCHEMA::[sandbox ] TO [sls_svc];
Grant CREATE TABLE to sls_svc;
CREATE EXTERNAL TABLE sanbox.revenue-by-month
WITH (
LOCATION = '/ROW/revenue-by-month/',
DATA_SOURCE = ADLS-ROW,
FILE_FORMAT = EF_PARQUET
)
AS
SELECT * from table1;
all users in sls_svc role has STORAGE DATA CONTRIBUTOR access on READ-WRITE-CONTAINER (ROW)
Below are the error messages I am getting
I also tried creating a new database. hoping that i can grant full access on that database - so that cross DB query can work - but I am out of luck there as well.
Any thoughts ?
It seems that you have correctly set permissions https://learn.microsoft.com/en-us/azure/synapse-analytics/sql/develop-storage-files-overview?tabs=impersonation#permissions
Are you sure that you can successfully execute just select statement and that the issue is not in SELECT part?
GRANT CONNECT to the database that was created
+
GRANT DDL_ADMIN access
resolved the issue
I have 2 DB on same SQL Azure server and i have same table(TB1) on both DB, now i want to read data from TB1 of DB2 and insert data into TB1 of DB1.
I am using below query but getting error.
insert into TB1 select 1,* from [DB2].dbo.TB1
Error Message
Msg 40515, Level 15, State 1, Line 16
Reference to database and/or server name in 'DB2.dbo.TB1' is not supported in this version of SQL Server.
Yes, you can use the Elastic Query Features on SQL Azure.It's the only way you can perform the cross database Queries.
Here are the detailed Queries to follow:
Run the below Query in your DB1(Since you said like reading the TB1 from DB2 and insert those Data's into your TB2 in your DB1)
CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'STro*ngPaSSe0rD';
CREATE DATABASE SCOPED CREDENTIAL Login
WITH IDENTITY = 'Login',
SECRET = 'STro*ngPaSSe0rD';
CREATE EXTERNAL DATA SOURCE RemoteReferenceData
WITH
(
TYPE=RDBMS,
LOCATION='myserver.database.windows.net',
DATABASE_NAME='DB2',
CREDENTIAL= Login
);
CREATE EXTERNAL TABLE [dbo].[TB1]
(
[Columns] [DataTypes]
)
WITH (DATA_SOURCE = [RemoteReferenceData])
After these step, you can Query the external table like the Normal table. Though some limitations while using the External table, like you couldn't able to Insert Data's into a EXTERNAL TABLE(Reference table)
Azure supports this cross database query feature since 2015 but needs some extra setup to work and Elastic Query.
The first step is create a security credential:
CREATE MASTER KEY ENCRYPTION BY PASSWORD = '<password>';
CREATE DATABASE SCOPED CREDENTIAL DB2Security
WITH IDENTITY = '<username>',
SECRET = '<password>';
The "username" and "password" should be the username and password used to login into the DB2 database.
Now you can use it to define a external datasource, so DB1 can connect to DB2:
CREATE EXTERNAL DATA SOURCE DB2Access
WITH (
TYPE=RDBMS,
LOCATION='myservernotyours.database.secure.windows.net',
DATABASE_NAME='DB2',
CREDENTIAL= DB2Security);
Finally, you map the TB1 as a external table from the DB2 database, using the previous external datasource:
CREATE EXTERNAL TABLE dbo.TB1FromDB2(
ID int,
Val varchar(50))
WITH
(
DATA_SOURCE = DB2Access);
You can also accomplish this using the Azure SQL Data Sync, but the data are replicated in one single database and this feature are still a preview version (May/2018) and you always see oldest data (the minimal configurable interval for each synchronization is 5 minutes).
You can perform cross database queries using the elastic query features on SQL Azure.
You will have to create an external data source and an external table to be able to query tables on other SQL Azure databases. This article shows how to do it.
Hope this helps.
I have a multi-tenant database, each user has their own schema.
What is the best way to backup a single tenant (table schema)? As far as I know SQL Server does not support backup of a single schema (only the complete database).
I need to backup the structure and data. Also it needs to be automated (Ideally I should be able to call it from SSMS as well).
I was thinking exporting the ddl and data as sql statements. If there is some way to call the "Generate and Publish Scripts" wizard as stored proc I think it would work?
I am currently on Sql Server 2008 R2 but could upgrade.
A couple of ideas.
Using File Groups
Put the tables each tenant has into their own file group. SQL Server has the ability to backup and restore individual file groups. You can also perform some other operations such as taking indivudual tenants offline if required. For example:
CREATE TABLE tenant1.Table1
(Column1 INT, Column2, INT)
ON Tenant1FileGroup
Views & Separate Databases
Probably not the right way to go, but it will work. Have the tables for each tenant in their own database and reference them from the 'master' database with a view in the tenant schema. For example:
Tenant1DB
dbo.Table1
dbo.Table2
Tenant2DB
dbo.Table1
dbo.Table2
MasterDB
tenant1.Table1
tenant1.Table2
tenant2.Table1
tenant2.Table2
Where the objects mentioned above in the MasterDB database are views such as:
CREATE VIEW tenant1.Table1
AS
SELECT * FROM Tenant1DB.dbo.Table1
This way you can easily backup/restore individual tenant databases. Some other benefits of this strategy:
Individual tenants can be restored without bringing the main database into single user mode.
The system will scale out well as the tenant database can be moved to other servers.
SQL Server is not my strong point and I start to get lost when going past the basic Create Table, Select from Table etc.
I am attempting to set up a database synchronisation scenario with an Microsoft SQLCompact 3.5 database being Synced through a Proxy to and SQL 2008 R2 database. It is being synced through the Microsoft Sync Framework 2.1.
I got everything set up and working fine through the proxy when the user account is the db_owner of the database.
Of course this now needs to be locked down - so I have been trying to restrict the permissions to the bare minimum required for Synchronisation.
According to the Microsoft articles, I need to do the following...
Use the principle of least permission. Do not grant more permissions than are required to perform a specific task. For example, do not grant INSERT permissions for server database tables that are involved in download-only synchronization. Synchronization operations require the following permissions:
EXECUTE permissions on all the stored procedures that Sync Framework uses to read and write to metadata tables and base tables.
SELECT, INSERT, UPDATE, and DELETE permissions for metadata tables and any base tables that will be updated during a synchronization session.
When you provision SQL Server databases that use SqlSyncProvider, be aware of the following permissions requirements for provisioning:
CREATE TABLE permissions to enable creation of the metadata tables: scope_info and scope_config, and the tracking tables that are created for each base table.
ALTER TABLE to add triggers to the base tables.
CREATE PROCEDURE permissions to create the procedures that Sync Framework requires.
SELECT and INSERT permissions for scope_info and scope_config tables.
SELECT permissions for base tables.
I allowed the wizards in Visual Studio 2010 to create the Sync database and proxy for me.
As such - I am unable to find the scope_info and scope_config tables in SQL Server databases, and I am also unable to find the metadata tables so cannot set permissions on these tables. Also - where would I find the stored procedures that the Synchronisation framework is trying to use - I have looked but cannto find them.
Where would I find these and how would I go about setting the appropriate permissions?
I have granted datareader and datawriter, Insert, Update, Delete and Select as well as Execute permissions on the SQL Server database but the sync fails. I have also granted Create Table, Create Procedure and ALTER permissions on the database for the user as well- but still it fails.
If i enable the db_owner role for the user - it works.
The error I receive is:
Unable to initialize the client database, because the schema for table 'tblApplications, tblApplicationConfiguration, tblApplicationInstallProperties, tblApplicationPreRequisites, tblApplicationTypes, tblComputerApps, tblComputers, tblComputerTypes, tblDriveHWSerials, tblDrives, tblDriveTypes, tblFunctions, tblLocationApps, tblLocationComputers, tblLocationIPAddress, tblLocations, tblLocationUsers, tblPermissions, tblRegionLocations, tblRegions, tblRegisteredModules, tblRequestFormats, tblRequestStatus, tblRequestTypes, tblRoles, tblRoleUsers, tblSecurity, tblUsers, tblVehicle, tblVehicleLocationMap, tblVehicleMake, tblRequestProcessingStatus, tblDriveStatus, tblVideoViewTypes' could not be retrieved by the GetSchema() method of DbServerSyncProvider.
Make sure that you can establish a connection to the client database and that either the SelectIncrementalInsertsCommand property or the SelectIncrementalUpdatesCommand property of the SyncAdapter is specified correctly.
I am not able to use the db_owner role when its released.
there are two types of database providers in Sync Framework, the offline provider (SqlCeClientSyncProvider/DbServerSyncProvider) which is used by the Local Database Cache project item and the collaboration/peer-to-peer provider (SqlSyncProvider/SqlCeSyncProvider).
the offline providers dont use the scope_xxx tables as such you wont find it.
assuming you used the default SQL Server Change Tracking when you configured the sync via the wizard, try granting VIEW CHANGE TRACKING permission to the account you're using to connect to the database.
I'm not sure if it will help, but I found this:
Try to set UploadOnly on the SyncTable object instead. When you set it on the SyncAdapterBuidler, the select incremental commands are not generated by the builder. These commands are used to get the table schema from the database. Since these are missing you are getting the schema error.
Also, maybe this blog entry will help.
As JuneT Mentioned, you should turn on Change Tracking
ALTER DATABASE YourDataBaseName
SET CHANGE_TRACKING = ON
(CHANGE_RETENTION = 2 DAYS, AUTO_CLEANUP = ON)
I have a situation whereby an application we use has many databases used for storage, and creates new ones on the fly as needed (SQL Server 2008 R2).
ApplicationDatabase
ApplicationDatabase_Storage001
ApplicationDatabase_Storage002
ApplicationDatabase_Storage003
etc...
As needed the application will create a new storage database for itself.
My problem is that I have a sql server account that is used for the ApplicationDatabase, and I want to automatically give it permissions to the storage databases as they are created, but not to any other database that happens to be created in the same sql server instance. I have no control over the creation of the storage databases.
I read In the answer to this question that I can add the account in the model database however this appears to add the permissions for all new databases, when I only want it to apply to the databases mentioned above.
The best solution I could come up with is a SQL server job or external app that runs once a day or so and looks for the existence of each database, applying the permissions on each that it finds, but this does not seem ideal.
You can implement a DDL trigger that will be fired whenever a new database is created. Depending on the properties of the database, like name or storage definition, you can probably run additional scripts on the new database to set up the required security.
http://msdn.microsoft.com/en-us/library/ms186406.aspx
Here's a snippet from the article above:
IF EXISTS (SELECT * FROM sys.server_triggers
WHERE name = 'ddl_trig_database')
DROP TRIGGER ddl_trig_database
ON ALL SERVER;
GO
CREATE TRIGGER ddl_trig_database
ON ALL SERVER
FOR CREATE_DATABASE
AS
PRINT 'Database Created.'
SELECT EVENTDATA().value('(/EVENT_INSTANCE/TSQLCommand/CommandText)[1]','nvarchar(max)')
GO
DROP TRIGGER ddl_trig_database
ON ALL SERVER;
GO
Regards
Piotr