I submitted a new join in Azure Data Lake and got CQO: Internal Error - Optimizer internal error when I use left join or outer join in my U-SQL script. I tried to use inner join instead of left join and also tried to use left join in COSMOS. Both cases are working fine. Can anyone tell me why left join doesn't work in Azure Data Lake? Any help will be appreciated. Also attached screenshot with job link in case somebody needed. screenshot here
Related
I have a large query that can't go in memory. This query involves joining a few tables.. We are using a "DIRECT QUERY" to pull the data out of a database.
How would I write a query that has joins in the SQL with Qlikview direct discovery? Is it possible to do?
This guide outlines how to join in direct discovery. You can do a where = join or use the JOIN keyword.
https://help.qlik.com/sense/1.1/en-US/online/Subsystems/Hub/Content/DirectDiscovery/MultiTableSupportDirectDiscovery.htm
I have a big table on an external server, and I am trying do an update on another table on a local machine. The external server table is more than a couple of billion rows. When I run the following update as a stored procedure, it takes a very long time to give results. Is there a method to tweak this so that it can be broken into smaller datasets to query?
update C set C.[P_Flag]='Y' from [LocalMachine].dbo.O_C_Data C inner
join [ExternalServer].dbo.E_Extract E on C.[O_ID]=E.[B_ID] and
C.[P_Option]=E.[P_OPTID] and C.[P]=E.[PD_G] where C.[O_Flag]='Y'
EDIT Just as #usr pointed out, what I said below is not correct. You'd better know why it is slow by checking execution plan, which I didn't master when I faced the same problem. You query might trigger too many executions to remote server. That is to query data on remote server separately for every single row on local. However, I suggest to have a try with the first approach below. It did solve similar problem before. I suspect that you have the same root cause.
When you join local table to a table in remote server, SQL Server data engine will pull all data from table in remote and do JOIN in your local memory. That's why it's extremely slow.
I suggest creating a local temp table, inserting temp table with data pulled from remote server and updating local table with temp table. To achieve that, you still need to JOIN local table and remote table. There are 2 approaches:
Embed local table in query. In your case, can you write
INSERT INTO #TempTable
SELECT E.B_ID
FROM [ExternalServer].dbo.E_Extract E
WHERE E.[B_ID] IN (C_ID1, C_ID2, ...) AND E.[P_OPTID] IN (P_Option1, P_Option2) AND E.[PD_G] IN (P1, P2, ...)
Note, values after IN operator are constants.
Use REMOTE JOIN Hints. Take a look at MSDN
REMOTE
Specifies that the join operation is performed on the site of the right table. This is useful when the left table is a local table and the right table is a remote table. REMOTE should be used only when the left table has fewer rows than the right table.
If the right table is local, the join is performed locally. If both tables are remote but from different data sources, REMOTE causes the join to be performed on the site of the right table. If both tables are remote tables from the same data source, REMOTE is not required.
REMOTE cannot be used when one of the values being compared in the join predicate is cast to a different collation using the COLLATE clause.
REMOTE can be used only for INNER JOIN operations.
Is there a default report in Sql Server that will output the permissions/securables history for given user or login? If not, does anyone know how to craft such a query? We had an incident recently where a user mysteriously lost insert permissions on a table, and we'd like to find out exactly what caused it.
The only way to figure this out is to read transaction log that stores details on each transaction. If your database was in full recovery mode then the info is somewhere in there.
Unfortunately you can’t do this using standard tools because MS doesn’t support this.
You can either get yourself a commercial log reader or try to hack this using undocumented commands like fn_dblog.
Check these out for more details
Read the log file (*.LDF) in sql server 2008
SQL Server Transaction Log Explorer/Analyzer
How to view transaction logs in sql server 2008
Here is a query for finding the permissions on a database. We have been using a variant of this query to copy permissions from one table to another table.
select *
FROM sys.database_permissions AS dp
INNER JOIN sys.objects AS o ON dp.major_id=o.object_id
INNER JOIN sys.schemas AS s ON o.schema_id = s.schema_id
INNER JOIN sys.database_principals AS dpr ON dp.grantee_principal_id=dpr.principal_id
In crystal I am trying to do a full outer join however the option is disabled (greyed out) in the database expert. Does anyone know why or how to get this to work so I can select it?
I want to end up with a sort of Calendar that shows the different scheduled jobs and their steps (the exact command executed). But I"m having a hard time finding all the data. So my question is two-fold:
1) Does anyone know of a software (preferably open source or free) that can get this data from sql server 2000 in a spreadsheet or other easy to manipulate format?
2) Can anyone help me with the queries I'd need to get this data. How are SQL Agent Jobs organized in the database?
Any guidance on this issue would be welcome.
The information regarding jobs is stored in the MSDB database on your server.
Tables in the MSDB database will be of interest to you (names are self-explanatory)
sysjobs
sysjobsteps
The schedules are stored here
sysjobschedules
You can get execution history here
sysjobhistory
There are several built in procedures to help you with the details:
sp_help_job
sp_help_jobhistory
sp_help_jobschedule
Using this query to get the info I need:
SELECT j.name, database_name,Step_id, Subsystem, command, Sjs.enabled, SJS.freq_type,
sjs.freq_interval, sjs.freq_subday_type, freq_subday_interval, freq_recurrence_factor, next_run_date, next_run_time
FROM sysjobsteps SJ
INNER JOIN sysjobs J ON SJ.Job_id = J.Job_Id
INNER JOIN sysJobschedules SJS ON J.Job_ID = SJS.Job_id
order by J.name, SJ.step_id