Mystery Parametric SQL Query with joins + TClientDataSet blank fields - sql

Some times even obvious things seem to go wrong. There's a lot of obvious and a lot that can go definitely wrong when dealing with SQL components and even more when they are just a ring of a whole SQLQuery => Provider => ClientDataSet => DataSource => DataControl chain.
Today's example is so incredibly dumb, yet a time waster.
How to replicate it:
Drop a TZQuery (ZeosLib) with a simple parametric join in its SQL, example:
SELECT
pr.product_id AS product_id,
pr.model AS model,
pd.name AS name,
pr.image AS image,
pr.status AS status,
pr.date_added AS date_added,
pr.date_modified AS date_modified
FROM oc_product pr
LEFT JOIN oc_product_description pd
ON pr.product_id = pd.product_id AND pd.language_id = :language_id
WHERE
pr.status = 1
ORDER BY
pd.name
Of course we have one parameter, :language_id.
Then drop a TDataSetProvider linked to it, then a TClientDataSet, then a TDataSource and finally a TDBGrid, all linked each to the previous. Finally drop a TDBNavigator and link it as well.
Finally, add fields in the TClientDataSet, captions etc.
At program startup we assign the parameter to the TZQuery component with something like:
qryProduct.Params.ParamByName('language_id').AsInteger := 2;
Where 2 is a demo hardcoded value (in the real application it is determined by querying the Windows current language being used).
Now run the the application: perfect!
Now press "Refresh" on the TDBNavigator.
Either you get a nasty: "Key Violation" or a perfectly blank name column in the TDBGrid.
Why?

The solution is so simple yet not so immediately evident.
Looking at the generated refresh SQL (via a TZSQLMonitor) the joined table it's evident the parameter is not working:
2014-01-04 11:19:38 cat: Execute, proto: mysql-5, msg: SELECT pr.product_id AS product_id,
pr.model AS model, pd.name AS name, pr.image AS image, pr.status AS status, pr.date_added
AS date_added, pr.date_modified AS date_modified FROM oc_product pr LEFT JOIN
oc_product_description pd ON pr.product_id = pd.product_id AND pd.language_id = NULL
WHERE pr.status = 1 ORDER BY pd.name
The parameter bit is:
pd.language_id = NULL
A quick glance to the ClientDataSet component would reveal that the parameter name has been captured from the query component. But not the value.
Now, I don't know if this is a problem with the TZQuery component not sending the parameter value at runtime or if the functionality is just not implemented by the ClientDataSet but the errors it generates are far from immediately evident.
The fix, of course, is to manually set the parameter value both in the TZQuery and in the TClientDataSet components before Active is set to true.

Related

DBSQL_SQL_INTERNAL_DB_ERROR SQL error 2048

I have to join two tabled ACDOCA and BKPF. I have written the follow code for it.
SELECT a~rbukrs,
a~racct,
a~bldat,
a~blart,
a~kunnr,
a~belnr,
a~sgtxt,
b~xblnr,
a~budat,
a~hsl,
a~prctr
INTO TABLE #it_final
FROM acdoca AS a
LEFT OUTER JOIN bkpf AS b
ON a~rbukrs = b~bukrs
AND a~gjahr = b~gjahr
WHERE a~rbukrs IN #s_bukrs
AND a~Kunnr IN #s_kunnr
AND a~Budat IN #s_budat
AND a~Belnr IN #s_belnr
AND a~rldnr IN #s_rldnr
AND a~blart = 'DR' OR a~blart = 'ZK' OR a~blart = 'UE'.
Facing the following errors:----
Runtime error: DBSQL_SQL_INTERNAL_DB_ERROR
SQL error "SQL code: 2048" occurred while accessing table "ACDOCA".
Short Text: An exception has occurred in class "CX_SY_OPEN_SQL_DB"
How do I resolve this? please help.
A few things:
Selecting directly from the database tables is error prone (e.g. you'll forget keys while joining) and you have to deal with those terrible german abbreviations (e.g. Belegnummer -> belnr). Since quite some time there are CDS Views on top such as I_JournalEntryItem with associations and proper english names for those fields, if you can use them, I would (also they're C1 released).
As already pointed out by xQBert the query does probably not work as intended as AND has prescendence over OR, and as such your query basically returns everything from ACDOCA, multiplied by everything from BKPF which likely leads to the database error you've posted
With range queries you might still get a lot of results (like billions of entries, depending on your company's size), you should either limit the query with UP TO, implement some pagination or COUNT(*) first and show an error to the user if the result set is too large.
I would write that like this:
TYPES:
BEGIN OF t_filters,
company_codes TYPE RANGE OF bukrs,
customers TYPE RANGE OF kunnr,
document_dates TYPE RANGE OF budat,
accounting_documents TYPE RANGE OF fis_belnr,
ledgers TYPE RANGE OF rldnr,
END OF t_filters.
DATA(filters) = VALUE t_filters(
" filter here
).
SELECT FROM I_JournalEntryItem
FIELDS
CompanyCode,
GLAccount,
DocumentDate,
AccountingDocumentType,
Customer,
AccountingDocument,
DocumentItemText,
\_JournalEntry-DocumentReferenceID,
PostingDate,
AmountInCompanyCodeCurrency,
ProfitCenter
WHERE
CompanyCode IN #filters-company_codes AND
Customer IN #filters-customers AND
DocumentDate IN #filters-document_dates AND
AccountingDocument IN #filters-accounting_documents AND
Ledger IN #filters-ledgers AND
AccountingDocumentType IN ( 'DR', 'ZK', 'UE' )
INTO TABLE #DATA(sales_orders)
UP TO 100 ROWS.
(As a bonus you'll get proper DCL authorization checks)
2048 is/can be a memory allocation error: Too much data being returned. Given that, this line is highly suspect
AND a~blart = 'DR' OR a~blart = 'ZK' OR a~blart = 'UE'.
I'd consider this instead. Otherwise ALL blart ZK and UE records are returned regardless of customer, year, company et...
SELECT a~rbukrs,
a~racct,
a~bldat,
a~blart,
a~kunnr,
a~belnr,
a~sgtxt,
b~xblnr,
a~budat,
a~hsl,
a~prctr
INTO TABLE #it_final
FROM acdoca AS a
LEFT OUTER JOIN bkpf AS b
ON a~rbukrs = b~bukrs
AND a~gjahr = b~gjahr
WHERE a~rbukrs IN #s_bukrs
AND a~Kunnr IN #s_kunnr
AND a~Budat IN #s_budat
AND a~Belnr IN #s_belnr
AND a~rldnr IN #s_rldnr
AND a~blart IN ('DR','ZK','UE').
However, if you really did mean to return all blart ZK, UE records and only those that ar DR and in the defined parameters... you're simply asking for too much data from teh system and need to "LIMIT" your result set and somehow let the user know only a limited set is being returned due to data volume
I'd also ensure your join on keys is sufficient. Fiscal Year and company code represent an incomplete key to BKPF. I dont' know ACDOCA data table so I'm unsure if that's a proper join which may be leading to a semi-cartesean contributing to data bloat. I'd think in a multi-tenant db, you may need to join on mandt as well... possibly a doc number and some other values... again, this lookst to be an incomplete join on key.... so perhaps more is needed there as well.

ORA-01841 happens on one environment but not all

I have the following SQL-code in my (SAP IdM) Application:
Select mcmskeyvalue as MKV,v1.searchvalue as STARTDATE, v2.avalue as Running_Changes_flag
from idmv_entry_simple
inner join idmv_value_basic_active v1 on mskey = mcmskey and attrname = 'Start_of_company_change'
and mcentrytype = 'MX_PERSON' and to_date(v1.searchvalue,'YYYY-MM-DD')<= sysdate+3
left join idmv_value_basic v2 on v2.mskey = mcmskey and v2.attrname = 'Running_Changes_flag'
where mcmskey not in (Select mskey from idmv_value_basic_active where attrname = 'Company_change_running_flag')
I already found the solution for the ORA-01841 problem, as it could either be a solution similar to MSSQLs try_to_date as mentioned here: How to handle to_date exceptions in a SELECT statment to ignore those rows?
or a solution where I change the code to something like this, to work soly on strings:
Select mcmskeyvalue as MKV,v1.searchvalue as STARTDATE, v2.avalue as Running_Changes_flag
from idmv_entry_simple
inner join idmv_value_basic_active v1 on mskey = mcmskey and attrname = 'Start_of_company_change'
and mcentrytype = 'MX_PERSON' and v1.searchvalue<= to_char(sysdate+3,'YYYY-MM-DD')
left join idmv_value_basic v2 on v2.mskey = mcmskey and v2.attrname = 'Running_Changes_flag'
where mcmskey not in (Select mskey from idmv_value_basic_active where attrname = 'Company_change_running_flag')
So for the actually problem I have a solution.
But now I came into discussion with my customers and teammates why the error happens at all.
Basically for all entries of idmv_value_basic_activ that comply to the requirement of "attrname = 'Start_of_company_change'" we can be sure that those are dates. In addition, if we execute the query to check all values that would be delivered, all are in a valid format.
I learned in university that the DB-Engine could decide in which order it will run individual segments of a query. So for me the most logical explanation would be that, on the development environment (where we face the problem), the section " to_date(v1.searchvalue,'YYYY-MM-DD')<= sysdate+3” is executed before the section “attrname = 'Start_of_company_change'”
Whereas on the productive environment, where everything works like a charm, the segments are executed in the order that is descripted by the SQL Statement.
Now my Question is:
First: do I remember that right, since the teacher said that only once and at that time I could not really make sense out of it
And Second: Is this assumption of mine correct or is there another reason for the problem?
Borderinformation:
The Tool uses a kind of shifted data structure which is why there can be quite a few different types in the actual “Searchvalue” column of the idmv_value_basic_activ view. The datatype on the database layer is always a varchar one.
"the DB-Engine could decide in which order it will run individual segments of a query"
This is correct. A SQL query is just a description of the data you want and where it's stored. Oracle will calculate an execution plan to retrieve that data as best it can. That plan will vary based on any number of factors, like the number of actual rows in the table and the presence of indexes, so it will vary from environment to environment.
So it sounds like you have an invalid date somewhere in your table, so to_date raises an exception. You can use validate_conversion to find it.

What is the purpose of "#*" hash and star symbols before the parameters passed to a query?

I have the following query which gets called from ASP.NET application and creates a subset of rows within the same table "DETAILS" and the subset is defined by parameters $f2 and $f3 used for paging purposes.
INSERT INTO DETAILS (ID, UIN, ACTIVE_IND, UGUID, CREATED_BY, CREATED_DATE)
SELECT AF.ID, AF.UIN, AF.ACTIVE, AF.UGUID, AF.CREATED_BY, AF.CREATED_DATE FROM
(SELECT #*$f0 ID, DET.UIN, DET.ACTIVE_IND, DET.UGUID, DET.CREATED_BY, DET.CREATED_DATE,
DENSE_RANK() OVER (ORDER BY P.PRODUCT_ID) FG
FROM DETAILS DET
JOIN PRODUCTS P ON P.UIN = DET.UIN
WHERE ID = #*$f1 ) AF
WHERE AF.FG BETWEEN #*$f2 AND #*$f3
The ASP.NET c# code that calls this query looks like this
new SqlDataSource().ExecuteSql("InsertDetails",
new List<object>() {_subSetId, _mainSetId, start, end});
"InsertDetails" is the name of the above query and "start" "end" are the paging range.
My question is: What function or purpose have the "#*" before the parameters in this query??. I need to replicate this query for other tables but would like to know why are the parameters passed like this "#*$f0", "#*$f1", "#*$f2" and "#*$f3".
You asked me to put my comment as answer. It seems that I had the correct guess:
Sometimes such weird characters are used as place holders which are
replaced on string level before the call (kind of dynamic command
generation)... Neither beautifull nor clean, but sometimes - well -
you know...
You might use the profiler to monitor the statement which
is processed acutally
Happy Coding!

Long Text Field over 255 Characters gets truncated

Not sure why my field in my query is getting truncated upon the return of the result. The value is being stored in the field, but gets truncated by access to help with "performance". I have reviewed multiple forums and SO posts to no avail.
Problems listed at link do not apply, Aggregation, Uniqueness, Union, Format Property, Row Source
What is wrong with my query? Instructions field in the Customer table is the one that is getting truncated.
Here is the raw query generated by access:
SELECT Task.ID, Task.TaskID, Task.TaskName, Task.TypeID, TaskType.TaskTypeName, Task.CustomerID, Customer.CustomerName, Customer.OnHold, Customer.Blacklisted, Customer.CustomerEngagementRecieved, Customer.AutoEmail, Customer.SpecialInstructions, Customer.Instructions, Task.QuoteRequired, Task.PriorityID, Priority.Priority, Task.Min, Task.Max, Task.Projected, Task.DeadlineDate, Task.ResourceID, Resource.ResourceName, Resource.Email, Resource.Extension, Task.Description, Task.StatusID, Status.Status, Task.DeveloperLog, Task.TaskPOCID, POC.Phone, POC.Email, Task.OtherPOC, Task.OtherPOCPhone, Task.OtherPOCEmail, Task.FolderPath, Task.StopBilling, Task.Premium, Task.EntryDate, Task.CompleteDate, Task.AssignedBy, Task.SettingsID, Settings.AutoEmail
FROM TaskType
INNER JOIN (Status
INNER JOIN (Settings
INNER JOIN (Resource
INNER JOIN (Priority
INNER JOIN (Customer
INNER JOIN (Task
INNER JOIN POC ON Task.TaskPOCID = POC.POCID)
ON Customer.CustID = Task.CustomerID)
ON Priority.PriorityID = Task.PriorityID)
ON Resource.ResourceID = Task.ResourceID)
ON Settings.SettingsID = Task.SettingsID)
ON Status.StatusID = Task.StatusID)
ON TaskType.TTID = Task.TypeID;
`
Have a close read of this - http://allenbrowne.com/ser-63.html something in your set up will causing the truncation.
If it's when you cut and paste the query results that can also be mis-leading. When you say a Long Text are these linked tables?
I'd also rename your Min and Max fields as they are reserved words and may cause access to think you are aggregating your data.
So from the sounds of it, Access just sometimes will ALWAYS truncate the field no matter what the settings. There is a way to force access to show the entire field though, by using the DLOOKUP() function instead of using a Control Source.
Here is the Answer to my current Issue for reference,
=DLOOKUP("Instructions", "Customer", "CustID=" & [CustomerID])

Access query returns empty fields depending on how table is linked

I've got an Access MDB I use for reporting that has linked table views from SQL Server 2005. I built a query that retrieves information off of a PO table and categorizes the line item depending on information from another table. I'm relatively certain the query was fine until approximately a month ago when we shifted from compatibility mode 80 to 90 on the Server as required by our primary application (which creates the data). I can't say this with 100% certainty, but that is the only major change made in the past 90 days. We noticed that suddenly data was not showing up in the query making the reports look odd.
This is a copy of the failing query:
SELECT dbo_porel.jobnum, dbo_joboper.opcode, dbo_porel.jobseqtype,
dbo_opmaster.shortchar01,
dbo_porel.ponum, dbo_porel.poline, dbo_podetail.unitcost
FROM ((dbo_porel
LEFT JOIN dbo_joboper ON (dbo_porel.assemblyseq = dbo_joboper.assemblyseq)
AND (dbo_porel.jobseq = dbo_joboper.oprseq)
AND (dbo_porel.jobnum = dbo_joboper.jobnum))
LEFT JOIN dbo_opmaster ON dbo_joboper.opcode = dbo_opmaster.opcode)
LEFT JOIN dbo_podetail ON (dbo_porel.poline = dbo_podetail.poline)
AND (dbo_porel.ponum = dbo_podetail.ponum)
WHERE (dbo_porel.jobnum="367000003")
It returns the following:
jobnum opcode jobseqtype shortchar01 ponum poline unitcost
367000003 S 6624 2 15
The query normally should have displayed a value for opcode and shortchar01. If I remove the linked table dbo_podetail, it properly displays data for these fields (although I obviously don't have unitcost anymore). At first I thought it might be a data issue, but I found if I nested the query and then linked the table, it worked fine.
For example the following code works perfectly:
SELECT qryTest.*, dbo_podetail.unitcost
FROM (
SELECT dbo_porel.jobnum, dbo_joboper.opcode, dbo_porel.jobseqtype,
dbo_opmaster.shortchar01, dbo_porel.ponum, dbo_porel.poline
FROM (dbo_porel
LEFT JOIN dbo_joboper ON (dbo_porel.jobnum=dbo_joboper.jobnum)
AND (dbo_porel.jobseq=dbo_joboper.oprseq)
AND (dbo_porel.assemblyseq=dbo_joboper.assemblyseq))
LEFT JOIN dbo_opmaster ON dbo_joboper.opcode=dbo_opmaster.opcode
WHERE (dbo_porel.jobnum="367000003")
) As qryTest
LEFT JOIN dbo_podetail ON (qryTest.poline = dbo_podetail.poline)
AND (qryTest.ponum = dbo_podetail.ponum)
I'm at a loss for why it works in the latter case and not in the first case. Worse yet, it seems to work intermittently for some records and not for others (it's consistent about the ones it does and does not work for).
Do any of you experts have any ideas?
You definitely need to use subqueries for multiple left/right joins in Access.
I think it's a limitation of the Jet optimizer that gets confused if you're just chaining left/right joins.
You can see that this is a recurrent problem that surfaces often.
I'm always confused by Access' use of brackets in joins. Try stripping out the extra brackets.
FROM
dbo_porel
LEFT JOIN
dbo_joboper ON (dbo_porel.assemblyseq = dbo_joboper.assemblyseq)
AND (dbo_porel.jobseq = dbo_joboper.oprseq)
AND (dbo_porel.jobnum = dbo_joboper.jobnum)
LEFT JOIN
dbo_opmaster ON (dbo_joboper.opcode = dbo_opmaster.opcode)
LEFT JOIN
dbo_podetail ON (dbo_porel.poline = dbo_podetail.poline)
AND (dbo_porel.ponum = dbo_podetail.ponum)
OK the above doesn't work - Sorry I give up