How to union data from different databases? - sql

I came across the necessity to union two selects from different databases, namely paradox (in bde) and ms sql server.
Currently bde (through TQuery) is used only in this part of the programm (i.e. dbgrid). Now I need to add some data stored in ms sql server database (with which I usually use TADOQuery) to the same grid.
Although queries are executed over completely different tables, the result set of columns is named and typed similarly (I mean, if I had these tables, say, in ms sql server database, I could use a trivial union for that).
Is there any way to unite recordsets selected from these in delphi7 that I could use the result as a data source for a dbgrid?

You could use a clientdataset, created by the definitions of eg. the dataset of your SQL-Server dataset and add data of your paradox dataset. TFieldDefArray can be empty in your case.
type
TMyFieldDef = Record
Name: String;
Size: Integer;
DataType: TFieldType;
end;
TFieldDefArray = array of TMyFieldDef;
function GetClientDSForDS(ADataSet: TDataSet; AFieldDefArray: TFieldDefArray; AClientDataSet: TClientDataSet = nil; WithRecords: Boolean = true)
: TClientDataSet;
var
i: Integer;
Function NoAutoInc(ft: TFieldType): TFieldType;
begin
if ft = ftAutoInc then
Result := ftInteger
else
Result := ft;
end;
begin
if Assigned(AClientDataSet) then
Result := AClientDataSet
else
Result := TClientDataSet.Create(nil);
Result.Close;
Result.FieldDefs.Clear;
for i := 0 to ADataSet.FieldCount - 1 do
begin
Result.FieldDefs.Add(ADataSet.Fields[i].FieldName, NoAutoInc(ADataSet.Fields[i].DataType), ADataSet.Fields[i].Size);
end;
for i := 0 to High(AFieldDefArray) do
Result.FieldDefs.Add(AFieldDefArray[i].Name, AFieldDefArray[i].DataType, AFieldDefArray[i].Size);
Result.CreateDataSet;
for i := 0 to ADataSet.FieldCount - 1 do
begin
Result.FieldByName(ADataSet.Fields[i].FieldName).DisplayLabel := ADataSet.Fields[i].DisplayLabel;
Result.FieldByName(ADataSet.Fields[i].FieldName).Visible := ADataSet.Fields[i].Visible;
end;
if WithRecords then
begin
ADataSet.First;
while not ADataSet.Eof do
begin
Result.Append;
for i := 0 to ADataSet.FieldCount - 1 do
begin
Result.FieldByName(ADataSet.Fields[i].FieldName).Assign(ADataSet.Fields[i]);
end;
Result.Post;
ADataSet.Next;
end;
end;
end;
another attempt might be creating a linked server for paradox, I didn't try that...
http://www.experts-exchange.com/Microsoft/Development/MS-SQL-Server/SQL_Server_2008/Q_24067488.html

No problem with AnyDAC LocalSQL. You can execute SQL's with any DataSet, not only select SQL, insert, update, delete SQL too.

You can use the Built-in TClientDataSet functionality to union the data by appending the data from the second dataset to the data of the first one.
There are different ways to do it, my preferred one because the simple code would be to add two DataSetProviders and link it to each of your DataSets, for example
dspBDE.DataSet := MyTQuery;
dspADO.DataSet := MyAdoQuery;
Then, to open your DataSets, you can just do:
MyClientDataSet.Data := dspBDE.Data;
MyClientDataSet.AppendData(dspADO.Data, True);
To make this to work, both DataSets have to match the field number and data types. Since your structures are similar, you can work by typecasting in your SQL if this does not happen automatically.

BDE supports (or supported) heterogeneous queries
This allows queries to span more than one dataset, but with a limited SQL syntax.
IIRC I used a few over a decade ago for some quick'n'dirty datamerges, but I can't remember the specifics - I haven't touched BDE for years.

Several years ago (Delphi 7) i used TxQuery, but i'dont know if it is still in development
I have found this link

Related

Filling a dataset from a sql select string in Delphi

I am currently VERY new to delphi. It is older code, but I need to know if why my code is failing is what I think it is. all the data I need to read is in the tables I'm reading from, I know that much. but I'm getting the following error back on my SOAP call:
There is no row at position 0.
This tells me that although the data is in the DB, I know I'm not able to read it out. am I using the correct way to read sql data from a db table, in delphi, using the following code:
FXMLDocument := XMLDocument.Create;
FXMLDocument.LoadXML(strXML);
// Get WebService URL - needed for saving COG Assessments
strUrl := GetURL;
// Fill Applicant and Configuration DataSets
FApplicant := TApplicant.Create;
FConfiguration := TConfiguration.Create;
FApplicant.Username := FXMLDocument.GetElementsByTagName('appid').item(0).FirstChild.Value;
strFormID := FXMLDocument.GetElementsByTagName('formid').item(0).FirstChild.Value;
SqlConDB.Close;
selApp.CommandText := 'SELECT AppID, FirstName, LastName, Age, Gender, UserPassword, ProgToRun, ReportLang, CompanyID FROM Users WHERE AppID = '''+ FApplicant.Username +'''';
selConfig.CommandText := 'SELECT * FROM Preferences';
SqlConDB.Open;
daApp.Fill(dsApp, 'Applicant');
daConfig.Fill(dsConfig, 'Configuration');
SqlConDB.Close;
FConfiguration := CASConfigRoutines.FillConfigurationFromDB(dsConfig);
// Get Program to Run from Data
FApplicant.ProgToRun := dsApp.Tables[0].Rows[0].Item['ProgToRun'].ToString;
FApplicant.Gender := dsApp.Tables[0].Rows[0].Item['Gender'].ToString;
FApplicant.ReportLang := dsApp.Tables[0].Rows[0].Item['ReportLang'].ToString;
// Get Company ID from Data for FNB check
FApplicant.Company := dsApp.Tables[0].Rows[0].Item['CompanyID'].ToString;
Should these two lines:
daApp.Fill(dsApp, 'Applicant');
daConfig.Fill(dsConfig, 'Configuration');
Not rather be:
daApp.Fill(selApp, 'Applicant');
daConfig.Fill(selConfig, 'Configuration');
as it is selApp that is the "select" statement? I don't know enough about Delphi to say if it is right or wrong, but can someone maybe assist in telling me why I'm getting a record count of 0 if I know for a fact, my data is in the DB?
Thanks!

Returning multiple values using function causing multiple query runs

We have kiosks for customers to check their purchase volume for two different categories of items. They will input their mobile number, which will send an OTP to their mobile numbers and they will input it back to authenticate, the system has to check the data and display for them. As a developer, the kiosk supplier has provided us with a limited functionality development kit by which we can execute select statement on the database and display the returned values on the kiosk.
I have created an object type as follows:
CREATE OR REPLACE TYPE rebate_values
AS
OBJECT (ASales_total number,
ACurrent_Rebate_Percent number,
ANeeded_Sales number,
ANext_Rebate_Percent number,
BSales_total number,
BCurrent_Rebate_Percent number,
BNeeded_Sales number,
BNext_Rebate_Percent number);
A function to which I will pass customers' mobile to get their sales and rebate information:
CREATE OR REPLACE FUNCTION AA_rebate_function (P_phone IN NUMBER)
RETURN rebate_values
IS
A_P_Sales_total NUMBER;
A_P_Current_Rebate_Percent NUMBER;
A_P_Needed_Sales NUMBER;
A_P_Next_Rebate_Percent NUMBER;
B_P_Sales_total NUMBER;
B_P_Current_Rebate_Percent NUMBER;
B_P_Needed_Sales NUMBER;
B_P_Next_Rebate_Percent NUMBER;
P_CODE VARCHAR (10);
BEGIN
SELECT CC_CODE
INTO P_CODE
FROM CUSTOMERS
WHERE C_MOBILE = P_phone;
FOR OUTDATA
IN (
--My Query to retrieve the data
Select ................
)
LOOP
IF OUTDATA.CLASS = 'X'
THEN
A_P_Sales_total := OUTDATA.SALES_TOTAL;
A_P_Current_Rebate_Percent := OUTDATA.CURRENT_REBATE_PERCENT;
A_P_Needed_Sales := OUTDATA.NEEDED_SALES_FOR_HIGHER_REBATE;
A_P_Next_Rebate_Percent := OUTDATA.NEXT_HIGHER_REBATE_PERCENT;
END IF;
IF OUTDATA.CLASS = 'Y'
THEN
B_P_Sales_total := OUTDATA.SALES_TOTAL;
B_P_Current_Rebate_Percent := OUTDATA.CURRENT_REBATE_PERCENT;
B_P_Needed_Sales := OUTDATA.NEEDED_SALES_FOR_HIGHER_REBATE;
B_P_Next_Rebate_Percent := OUTDATA.NEXT_HIGHER_REBATE_PERCENT;
END IF;
END LOOP;
RETURN rebate_values (A_P_Sales_total,
A_P_Current_Rebate_Percent,
A_P_Needed_Sales,
A_P_Next_Rebate_Percent,
B_P_Sales_total,
B_P_Current_Rebate_Percent,
B_P_Needed_Sales,
B_P_Next_Rebate_Percent);
END;
/
The query takes 27 seconds to retrieve the values for each customer. Each customer will have 2 rows, so that's why I have used LOOP to collect the values.
When I execute the function:
SELECT AA_rebate_function (XXXXXXXXXX) FROM DUAL;
I get data as follows in a single column within 27 seconds:
(XXXX, X, XXXX, X, XXXX, X, XXXX, X)
But when I execute the function to get the values in different columns, it takes 27 x 8 seconds = 216 seconds, i.e., approximately 3.6 minutes which is a big issue as the customer cannot wait for 3.6 minutes on the kiosk to view the data.
SELECT x.c.ASales_total,
x.c.ACurrent_Rebate_Percent,
x.c.ANeeded_Sales,
x.c.ANext_Rebate_Percent,
x.c.BSales_total,
x.c.BCurrent_Rebate_Percent,
x.c.BNeeded_Sales,
x.c.BNext_Rebate_Percent
FROM (SELECT AA_rebate_function (XXXXXXXXXX) c FROM DUAL) x;
I have tried using stored procedure with OUT values but it doesn't fit in my environment as I cannot program to execute stored procedures from the kiosk development toolkit because it only supports select statements, checked with the supplier and they don't have any plan to add that support in near future.
I tried converting the single field into multiple columns using REGEXP_SUBSTR but I get a type conversion error as it is an array.
The query is very complex and has to calculate data for the last 10 years and has millions of rows, 27 seconds is actually the optimum time to get the desired results.
Interesting! I didn't realize that when you query a function that returns an object, it runs the function once for each column you reference the object in. That's awkward.
The easiest solution I could find for this is to switch your function to be PIPELINED. You'll need to create a nested table type to do this.
create type rebate_values_t is table of rebate_values;
/
CREATE OR REPLACE FUNCTION AA_rebate_function (P_phone IN NUMBER)
RETURN rebate_values_t PIPELINED
IS
... your code here ...
PIPE ROW (rebate_values (A_P_Sales_total,
A_P_Current_Rebate_Percent,
A_P_Needed_Sales,
A_P_Next_Rebate_Percent,
B_P_Sales_total,
B_P_Current_Rebate_Percent,
B_P_Needed_Sales,
B_P_Next_Rebate_Percent));
RETURN;
END;
/
SELECT x.ASales_total,
x.ACurrent_Rebate_Percent,
x.ANeeded_Sales,
x.ANext_Rebate_Percent,
x.BSales_total,
x.BCurrent_Rebate_Percent,
x.BNeeded_Sales,
x.BNext_Rebate_Percent
FROM TABLE(AA_rebate_function (XXXXXXXXXX)) x;
For some reason, this should only execute the function once, and take 27 seconds.

Delphi ClientDataSet Sorting by changing IndexName

I've been learning about the ClientDataSet in delphi and how it can help sort my SQL database. The data is showing fine in my TDBGrid and i've enabled sorting by clicking on a header by changing the IndexField of the ClientDataset. I want to make it go descending on sorts sometimes though so have been trying to use 2 IndexNames outlined here https://stackoverflow.com/a/13130816/4075632
However, when I swap the IndexName from DEFAULT_ORDER to CHANGEINDEX, all data in my DBGrid disappears. I'm pretty new to all of this, and I know it will depend on my situation, but what are some of the ways this happens and I will try to troubleshoot them.
I have 1 TSQLConnection connected to TSQLQuery, and that connected to TDataSetProvider, and that connected to my ClientDataSet which leads to a TDataSource to the TDBGrid. Why might the ClientDataSet which is usually fine cause problems when I change its name? Please bear in mind that most of the settings are default because I'm not too sure about these components. Thanks, I hope you can provide some useful help and I'm sorry it may be difficult to se my situation.
Toby
I use the following code to build indexes for a clientdataset:
Procedure BuildIndices (cds: TClientDataSet);
var
i, j: integer;
alist: tstrings;
begin
with cds do
begin
open;
logchanges:= false;
for i:= 0 to FieldCount - 1 do
if fields[i].fieldkind <> fkCalculated then
begin
j:= i * 2;
addindex ('idx' + inttostr (j), fieldlist.strings[i], [], '', '', 0);
addindex ('idx' + inttostr (j+1), fieldlist.strings[i], [ixDescending], '', '', 0);
end;
alist:= tstringlist.create;
getindexnames (alist);
alist.free;
close;
end;
end;
As a result, there is an index 'idx0' for sorting column 0 ascending and 'idx1' for sorting column 0 descending; 'idx2' and 'idx3' for column 1, etc.
Then, in the grid's OnTitleClick event, I have the following
procedure Txxx.DBGrid1TitleClick(Column: TColumn);
var
n, ex: word;
begin
n:= column.Index;
try
dbgrid1.columns[prevcol].title.font.color:= clNavy
except
end;
dbgrid1.columns[n].title.font.color:= clRed;
prevcol:= n;
directions[n]:= not directions[n];
ex:= n * 2;
if directions[n] then inc (ex);
with clientdataset do
try
disablecontrols;
indexname:= 'idx' + inttostr (ex);
finally
first;
enablecontrols
end;
end;
In each form, I define an array of booleans ('directions'), one element per grid column. These elements track whether a column should be sorted ascending or descending.
The ClientDataSet comes with two predefined indexes: DEFAULT_ORDER and CHANGEINDEX, which are of no real use for your task, because you cannot tweak them to your needs. So you have to create your own indexes. A comprehensive description by Cary Jensen can be found in this article as well as in his highly recommended book about ClientDataSets.

Variable structure for database results

A lot of times when we query the database, we just need one column with varchar.
So I've made a nice function for querying the database and putting the results in a stringlist:
function Getdatatostringlist(sqlcomponent, sqlquery: string): TStringlist;
What I'm looking for now is basically the same function but for results with multiple columns where you don't know in advance what type the data is, be it varchar, int, datetime.
What kind of datastructure would be good to use here.
The reason I want this is that I try not to work on open datasets. I like much more to fetch all results into a temporary structure, close the dataset and work on the results.
After Kobiks reply about using in Memory datasets I came up with the following, it's fast put together to test the concept:
procedure TForm1.Button2Click(Sender: TObject);
var
MyDataSet : TAdoDataSet;
begin
MyDataSet := GetDataToDataSet('SELECT naam FROM user WHERE userid = 1', ADOConnection1);
try
Form1.Caption := MyDataSet.FieldByName('naam').AsString;
finally
MyDataSet.free;
end;
end;
function TForm1.GetDataToDataSet(sSql: string; AdoConnection: TADOConnection): TAdoDataSet;
begin
Result := TAdoDataSet.Create(nil);
Result.LockType := ltBatchOptimistic;
Result.Connection := AdoConnection;
Result.CommandText := sSql;
Result.Open;
Result.Connection := nil;
end;
I think this is something to build on.
You should use any disconnected in-memory TDataSet descendant, such as TClientDataSet.
Do not attempt to re-invent the wheel by storing a record-set in some new "Variant" structure. A TClientDataSet already contains all features you need to manipulate a "temporary" data structure.
Here is how you create a TClientDataSet structure:
cds.FieldDefs.Add('id', ftInteger);
cds.FieldDefs.Add('name', ftString, 100);
// ...
// create it
cds.CreateDataSet;
// add some data records
cds.AppendRecord([1, 'Foo']);
cds.AppendRecord([2, 'Bar']);
Many TDataSets has an ability to be used as an in-memory (client) datasets depending on the provider and LockType, for example a TADODataSet with LockType=ltBatchOptimistic could fetch results-set from the server, and then remain disconnected.
For exchanging Data with Excel this structure is usefull, might be useful for other purposes.
Function GetDatasetasDynArray(Ads: TDataset; WithHeader: Boolean = true): Variant;
// 20130118 by Thomas Wassermann
var
i, x, y: Integer;
Fields: Array of Integer;
begin
x := 0;
y := Ads.RecordCount;
if WithHeader then
inc(y);
SetLength(Fields, Ads.FieldCount);
for i := 0 to Ads.FieldCount - 1 do
if Ads.Fields[i].Visible then
begin
Fields[x] := i;
inc(x);
end;
SetLength(Fields, x);
Result := VarArrayCreate([0, y - 1 , 0, length(Fields) - 1], VarVariant);
y := 0;
if WithHeader then
begin
for i := Low(Fields) to High(Fields) do
begin
Result[y, i] := Ads.Fields[Fields[i]].DisplayLabel;
end;
inc(y);
end;
try
Ads.DisableControls;
Ads.First;
while not Ads.EOF do
begin
for i := Low(Fields) to High(Fields) do
begin
Result[y, i] := Ads.Fields[Fields[i]].Value;
end;
Ads.Next;
inc(y);
end;
finally
Ads.EnableControls;
end;
end;
procedure TForm1.Button1Click(Sender: TObject);
var
DynArray:Variant;
begin
DynArray := GetDatasetasDynArray(Adodataset1,true);
//DynArray[0,x] Header or First row
//DynArray[1,x] First row or SecondRow
Excel.Range.Value := DynArray;
end;
Why don't you like working with open datasets? They usually do not block the server. Copying the data from the dataset to whatever you want is extra overhead which is most likely not necessary.
A dataset provides exactly the functionality you want: A matrix with variable columns and rows.
EDIT: However, if you have iterate through the dataset often, you should consider creating a class holding the relevant information and then copy the data into a generic list, dictionary, tree or whatever you need as fast lookup structure.
Of course you could think of building something smart which can be as flexible as a dataset but: The more general things get, the poorer the performance (usually).

Type mismatch in expression in Delphi 7 on SQL append

I have a code, that checks the current date when a form is loaded, performs a simple calculation, and appends a SQL in Delphi. It works on Windows 7 with Delphi 7, and on another computer with Xp, but doesn't on 3 other computers with Xp. When the form is loaded it shows a "Type mismatch in expression", and points to the line after the append. What could be the problem?
procedure TfmJaunumi.FormCreate(Sender: TObject);
var d1, d2: TDate;
begin
d1:= Date;
d2:= Date-30;
With qrJaunumi do
begin
Open;
SQL.Append('WHERE Sanem_datums BETWEEN' + #39 + DateToStr(d1) +
#39 + 'AND' + #39 + DateToStr(d2) + #39);
Active := True;
end;
end;
As robsoft says, it is probably the internationalisation settings. You could use parameterised queries instead - they are generally simpler if using dates and times.
Also, the Open after the with's begin isn't needed - in fact it will open the query without the WHERE clause you are adding.
procedure TfmJaunumi.FormCreate(Sender: TObject);
var d1, d2: TDate;
begin d1:= Date; d2:= Date-30; With
qrJaunumi do
begin
SQL.Append('WHERE Sanem_datums BETWEEN :StartDate AND :EndDate');
// exact expression will vary according to DB connection type.
// Example is for TADOQuery.
Parameters.ParamByName('StartDate').Value := d1;
Parameters.ParamByName('EndDate').Value := d2;
Active := True;
end;
end;
You may use a prepared statement to overcome any localization problems with date time values. DateToStr depends on the client side. FormatDateTime can fail if the server's localization doesn't accept the date format.
procedure TfmJaunumi.FormCreate(Sender: TObject);
var
d1, d2: TDate;
begin
d1:= Date;
d2:= Date - 30;
//qrJaunumi.SQL.Clear; removed because it would remove the "SELECT ... FROM ..." part
qrJaunumi.SQL.Add('WHERE Sanem_datums BETWEEN :StartDate AND :StopDate ');
qrJaunumi.Prepared := True;
qrJaunumi.ParamByName('StartDate').AsDateTime := d1;
qrJaunumi.ParamByName('StopDate').AsDateTime := d2;
qrJaunumi.Open; // = qrJaunumi.Active := True;
end;
The space after ":StopDate" is important because Delphi has a bug in the parameter parser unless they fixed it in newer versions.
It's almost certainly to do with the local internationalisation settings on those computers - DateToStr will return a string in the local date format (possibly MM/DD/YYYY or DD/MM/YYYY) - and depending on where you are this might not be what you're expecting.
I suspect you'll find that the computers it's not working on think they're in a different country/use a different internationalisation setting to the computers where it's working.
A better solution would be to use FormatDateTime to get the date into a standard format that your SQL Server installation will accept, so there's no chance that any local 'internationalisation' settings can interfere like this.
Unfortunately none of the above worked, but the solution was to replace the Format with "yy.mm.dd." instead of "yyyy.mm.dd.", and adding single quotes. Weird, it says the format is "yyyy.mm.dd." everywhere.
The code looks like this now:
procedure TfmJaunumi.FormCreate(Sender: TObject);
var d1, d2: TDate; d3, d4, atd: String;
begin
d1:= Date;
d3:= FormatDateTime('yy.mm.dd.',d1);
d2:= Date-30;
d4:= FormatDateTime('yy.mm.dd.',d2);
atd := '''';
With qrJaunumi do
begin
Open;
SQL.Append('WHERE Sanem_datums BETWEEN'+ atd+d4+atd +'AND'+ atd+d3+atd+';');
Active := True;
end;
end;
Hi I had same error and found different solution from MS Support:
SQL.Text:='Delete * from TableName where((kno='+(inttostr(userNo))+')and(Sanem_datums >= # '+(FormatDateTime('mm-dd-yy',d1))+' # ))'; ExecSQL;
https://support.microsoft.com/en-us/kb/175258