I am wondering if anyone can explain why I get different results for the same query string between using the ExecuteSQL function in FM versus querying the database through a database browser (I'm using DBVisualizer).
Specifically, if I run
SELECT COUNT(DISTINCT IMV_ItemID) FROM IMV
in DBVis, I get 2802. In FileMaker, if I evaluate the expression
ExecuteSQL ( "SELECT COUNT(DISTINCT IMV_ItemID) FROM IMV"; ""; "")
then I get 2898. This makes me distrust the ExecuteSQL function. Inside of FM, the IMV table is an ODBC shadow, connected to the central MSSQL database. In DBVis, the application connects via JDBC. However, I don't think that should make any difference.
Any ideas why I get a different count for each method?
Actually, it turns out that when FM executes the SQL, it factors in whitespace, whereas DBVisualizer (not sure about other database browser apps, but I would assume it's the same) do not. Also, since the TRIM() function isn't supported by MSSQL (from what I've seen, at least) it is necessary to make the query inside of the ExecuteSQL statement something like:
SELECT COUNT(DISTINCT(LTRIM(RTRIM(IMV_ItemID)))) FROM IMV
Weird, but it works!
FM keeps a cache of the shadow table's records (for internal field-id-mapping). I'm not sure if the ExecuteSQL() function causes a re-creation of the cache. In other words: maybe the ESS shadow table is out of sync. Try to delete the cache by closing and restarting the FM client or perform a native find first.
You can also try a re-connect to the database server via the Open File script step.
HTH
Related
We are using the new DataStream feature introduced in NetScaler 9 (we're on v10) to do content switching (described here: http://support.citrix.com/proddocs/topic/netscaler/ns-dbproxy-wrapper-con.html). We have a read-only virtual server that balances across several read-only MySql slaves. We use our Content Switching to send all "Selects" over to the read-only server.
the policy is configured as such:
mysql.req.query.command.contains("select")
our users send multi-part queries to our database server. Most often they are simple, like:
use database;
select col1 from table1;
Sometimes they will put comments at the head of the query. for example:
-- this is my query
select col1 from table1;
What we've found is that if the query simply starts with a select, everything works swimmingly. However, in the cases where there is a use statement or comments preceding the query, the content swticher fails to detect that this is a select query and it bypasses our read-only virtual server.
I am about to tell all of our developers that they must fully alias every table in every query and avoid use statements (yes, this is a good thing anyway), and also that they cannot use comments in their sql (that's just silly).
Does anyone know how I can configure my NetScaler DataStream Content Switching to ignore comments and use statements?
The decision on where to send the query is done on the first line received after successful authentication... so ignoring the comment won't work.
You could setup a responder policy which sends back an error message saying "Please don't use SQL Comments in commands sent to the Load Balanced VIP". A bit draconian, but your devs would get the message fairly quick.. but there's no way to ignore the comment, but still base a decision on the select statement. However, I was under the impression that the select statement is up to the first semi colon... so in your example above, it should (in theory) still find the select statement. I'd need to test that to be certain of the behaviour however.
Also - the USE statement is critical. This is the DB on which all subsequent commands are issued.
It would be best practice to NOT use the USE statement, but instead, change the select statement to:
select col1 from database.table1;
Once the USE statement is seen, it prevents any subsequent commands being pipelined down the same connection... So if there are a lot of Use statements, you will not get to enjoy the connection multiplexing functionality that comes with DataStream.
We learned that Block Level comments are acceptable, but single line comments are not.
This is properly ignored:
/* my comment */
These comment styles are treated as part of the query:
-- my comment
# my comment
kind of ridiculous when having SET autocommit=0 is perfectly reasonable. What about in that situation.
I'm getting a little confused about using parameters with SQL queries, and seeing some things that I can't immediately explain, so I'm just after some background info at this point.
First, is there a standard format for parameter names in queries, or is this database/middleware dependent ? I've seen both this:-
DELETE * FROM #tablename
and...
DELETE * FROM :tablename
Second - where (typically) does the parameter replacement happen? Are parameters replaced/expanded before the query is sent to the database, or does the database receive params and query separately, and perform the expansion itself?
Just as background, I'm using the DevArt UniDAC toolkit from a C++Builder app to connect via ODBC to an Excel spreadsheet. I know this is almost pessimal in a few ways... (I'm trying to understand why a particular command works only when it doesn't use parameters)
With such data access libraries, like UniDAC or FireDAC, you can use macros. They allow you to use special markers (called macro) in the places of a SQL command, where parameter are disallowed. I dont know UniDAC API, but will provide a sample for FireDAC:
ADQuery1.SQL.Text := 'DELETE * FROM &tablename';
ADQuery1.MacroByName('tablename').AsRaw := 'MyTab';
ADQuery1.ExecSQL;
Second - where (typically) does the parameter replacement happen?
It doesn't. That's the whole point. Data elements in your query stay data items. Code elements stay code elements. The two never intersect, and thus there is never an opportunity for malicious data to be treated as code.
connect via ODBC to an Excel spreadsheet... I'm trying to understand why a particular command works only when it doesn't use parameters
Excel isn't really a database engine, but if it were, you still can't use a parameter for the name a table.
SQL parameters are sent to the database. The database performs the expansion itself. That allows the database to set up a query plan that will work for different values of the parameters.
Microsoft always uses #parname for parameters. Oracle uses :parname. Other databases are different.
No database I know of allows you to specify the table name as a parameter. You have to expand that client side, like:
command.CommandText = string.Format("DELETE FROM {0}", tableName);
P.S. A * is not allowed after a DELETE. After all, you can only delete whole rows, not a set of columns.
I have used mysql for some projects and recently I moved to postgresql. In mysql when I alter a table or a field the corresponding query will be displayed in the page. But such a feature was not found in postgresql(kindly excuse me if I'm wrong). Since the query was readily available it was very helpful for me to test something in the local database(without explicitly typing the query), copy the printed query and run it in the server. Now it seems like I've to manually do all the trick. Even though I'm familiar with the query operations,at times it can be pretty time consuming process. Can anybody help me? How can I get the corresponding query to get displayed in postgresql(like in mysql) whenever a change is made to the table?
If you use SELECT * FROM ... there should not be any reason for your output to not include newly added columns, no matter how you get your results - would that be psql in command line, PgAdmin3 or any other IDE.
After you add new columns, it is possible that these changes are still in open transaction in other window or SQL command - be sure to COMMIT such transaction. Note that your changes to data or schema will not be visible to any other database clients until transaction commits.
If your IDE still does not show changes, maybe you need to refresh list of tables or if that option is not available, restart your IDE. If that does not work still, maybe you should use better IDE.
If you have used SELECT field1, field2, ... FROM ... then you must add new fields into your SELECT statement(s) - but this would be true for any other SQL implementation, MySQL included.
You could use the LISTEN / NOTIFY mechanism in PostgreSQL to notify your client on altering the database schema.
I'm attempting to run a simple statement against an Access DB to find records.
Data validation in the records was horrible, and I cannot sanitize it. Meaning, it must be preserved as is.
I need to be able to search against a string with white space and hyphen characters removed. The following statement will work in Access 2010 direct:
select * from dummy where Replace(Replace([data1],' ',''),'-','') = 'ABCD1234';
Running it from an ODBC connection via PHP will not. It produces the following error:
SQL error: [Microsoft][ODBC Microsoft Access Driver] Undefined function 'Replace' in expression., SQL state 37000 in SQLExecDirect
Creating a query in the database that runs the function and attempting to search its values indirectly causes the same error:
select * from dummy_indirect where Expr1 = 'ABCD1234';
I've attempted to use both ODBC drivers present. ODBCJR32.dll (03/22/2010) and ACEODBC.dll (02/18/2007). To my knowledge these should be current as it was installed with the full Access 2010 and Access 2010 Database Engine.
Any ideas on how to work around this error and achieve the same effect are welcome. Please note, that I cannot alter the database in way, shape, or form. That indirect query was created in another mdb file that has the original tables linked from the original DB.
* Update *
OleDB did not really affect anything.
$dsn= "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=c:\dummy.mdb;";
I'm not attempting to use it as a web backend either. I'm not a sadomasochist.
There is a legacy system that I must support that does use Access as a backend. Data gets populated there from other old systems that I must integrate into more modern systems. Hence, the creation of an API with Apache/PHP that is running on the server supporting the legacy system.
I need to be able to search a table that has an alphanumeric case identifier to get a numeric identifier that is unique and tied to a generator (Autonumber in access). Users have been using it a trash box for years (inconsistent data entry with sporadic notations) so the only solution I have is to strip everything except alphanumeric out of both the field value and the search value and attempt to perform a LIKE comparison against it.
If not replace() which is access supported, what ODBC compatible functions exist that I can use do the same kind of comparison?
Just to recap, the Access db engine will not recognize the Replace() function unless your query is run from within an Access application session. Any attempt from outside Access will trigger that "Undefined function" error message. You can't avoid the error by switching from ODBC to OleDb as the connection method. And you also can't trick the engine into using Replace() by hiding it in separate query (in the same or another Access db) and using that query as the data source for your main query.
This behavior is determined by Access' sandbox mode. That linked page includes a list of functions which are available in the default sandbox mode. That page also describes how you can alter the sandbox mode. If you absolutely must have Replace() available for your query, perhaps the lowest setting (0) would allow it. However, I'm not recommending you do that. I've never done it myself, so don't know anything about the consequences.
As for alternatives for Replace(), it would help to know about the variability in the values you're searching. If the space or dash characters appear in only one or a few consistent positions, you could do a pattern match with a Like expression. For example, if the search field values consist of 4 letters, an optional space or dash, followed by 4 digits, a WHERE clause like this should work for the variations of "ABCD1234":
SELECT * FROM dummy
WHERE
data1 = 'ABCD1234'
OR data1 Like 'ABCD[- ]1234';
Another possibility is to compare against a list of values:
SELECT * FROM dummy
WHERE
data1 IN ('ABCD1234','ABCD 1234','ABCD-1234');
However if your search field values can include any number of spaces or dashes at any position within the string, that approach is no good. And I would look real hard for some way to make the query task easier:
You can't clean the stored values because you're prohibited from altering the original Access db in any way. Perhaps you could create a new Access db, import the data, and clean that instead.
Set up the original Access db as a linked server in SQL Server and build your query to take advantage of SQL Server features.
Surrender. :-( Pull in a larger data set to your PHP client code, and evaluate which rows to use vs. which to ignore.
I'm not sure you can do this with ODBC and your constraints. The MS Access driver is limited (by design; MS wants you to use SQL Server for back ends).
Can you use OLEDB? that might be an option.
I'm trying to pull user data from 2 tables, one locally and one on a linked server, but I get the wrong results when querying the remote server.
I've cut my query down to
select * from SQL2.USER.dbo.people where persId = 475785
for testing and found that when I run it I get no results even though I know the person exists.
(persId is an integer, db is SQL Server 2000 and dbo.people is a table by the way)
If I copy/ paste the query and run it on the same server as the database then it works.
It only seems to affect certain user ids as running for example
select * from SQL2.USER.dbo.people where persId = 475784
works fine for the user before the one I want.
Strangely I've found that
select * from SQL2.USER.dbo.people where persId like '475785'
also works but
select * from SQL2.USER.dbo.people where persId > 475784
brings back records with persIds starting at 22519 not 475785 as I'd expect.
Hope that made sense to somebody
Any ideas ?
UPDATE:
Due to internal concerns about doing any changes to the live people table, I've temporarily moved my database so they're both on the same server and so the linked server issue doesn't apply. Once the whole lot is migrated to a separate cluster I'll be able to investigate properly. I'll update the update once this happens and I can work my way through all the suggestions. Thanks for your help.
The fact that LIKE operates is not a major clue: LIKE forces integers to string (so you can say WHERE field LIKE '2%' and you will get all records that start with a 2, even when field is of integer type). Your incorrect comparisons would lead me to think your indexes are corrupt, but you say they work when not used via the link... however, the selected index might be different depending on the use? (I seem to recall an instance when I had duplicate indexes and only one was stale, although that was too long ago to recall the exact cause).
Nevertheless, I would try rebuilding your index using the DBCC DBREINDEX (tablenname) command. If it turns out that doing so fixes your query, you may want to rebuild them all: here is a script for rebuilding them all easily.
Is dbo.people a table or a view? I've seen something similar where the underlying table schema had been changed and dropping and recreating the view fixed the problem, although the fact that the query works if run directly on the linked server does indicate something index based..
Is the linked server using the same collation? Depending on the index used, I could see something like this perhaps happening if the servers were not collation compatible, but the linked server was set up with collation compatible (which tells Sql Server it can run the query on the remote server).
I would check the following:
Check your definition on the linked server, and confirm that SQL2 is the
server you expect it to be
Check and compare the execution plans both from the remote and local servers
Try linking by IP address rather than name, to ensure you have the proper machine
Put the code into a stored procedure on the remote machine, and try calling that instead
Sounds like a bug to me - I;ve read of some issues along these lines, btu can't remember specifically what. What version of SQL Server are you running?
select * from SQL2.USER.dbo.people where persId = 475785
for a PersID which fails how does:
SELECT *
FROM OpenQuery(SQL2, 'SELECT * FROM USER.dbo.people WHERE persId = 475785')
behave?