I have a stored procedure which searches by text passed by a parameter. I noticed that if the text is not in English (i.e. Hebrew, Arabic), the query finishes without returning any rows. I am sure there is data to be found
I dont know which SQL server is being used : it is whatever is provided by GoDaddy on shared Windows hosting plan.
The thing is, I have a asp.net site which can search and fetch the data from this column encoded as UTF-8. The path which does not work is the MS SQL Management Studio. Even when I manually run same stored procedures which work from inside asp.net - they dont manage to find non English characters. The parameters I pass to the query are prefixed by N qualifier.
Try using the N character like the sample :
Select * from students where name like N'%بیژن%'
and as mentioned in the comment the column data type should be Nvarchar.
Related
I have a db2 database where I store names containing special characters. When I try to retrieve them with an internal software, I get proper results. However when I tried to do the same with queries or look into the db, the characters are stored strangely.
The documentation says that the encoding is utf-8 latin1.
My query looks something like this:
SELECT firstn, lastn
FROM unams
WHERE unamid = 12345
The user with the given ID has some special characters in his/her name: é and ó, but the query returns it as Ă© and Ăł.
Is there a way to convert the characters back to their original form with using some simple SQL function? I am new to databases and encoding, trying to understand the latter by reading this but I'm quite lost.
EDIT: Currently sending queries via SPSS Modeler with a proper ODBC driver, the database lies on a Windows Server 2016
Per the comments, the solution was to create a Windows environment variable DB2CODEPAGE=1208 , then restart, then drop and re-populate the tables.
If the applications runs locally on the Db2-server (i.e. only one hostname is involved) then the same variable can be set. This will impact all local applications that use the UTF-8 encoded database.
If the application runs remotely from the Db2-server (i.e. two hostnames are involved) then set the variable on the workstation and on the Windows Db2-server.
Current versions of IBM supplied Db2-clients on Windows will derive their codepage from the regional settings which might not always render Unicode characters correctly, so using the DB2CODEPAGE=1208 forces the Db2-client CLI drivers to use a Unicode application code page to override this.
with t (firstn) as (
values ('éó')
--SELECT firstn
--FROM unams
--WHERE unamid = 12345
)
select x.c, hex(x.c) c_hes
from
t
, xmltable('for $id in (1 to string-length($s)) return <i>{substring($s, $id, 1)}</i>'
passing t.firstn as "s" columns tok varchar(6) path '.') x(c);
C C_HEX
- -----
é C3A9
ó C3B3
The query above converts the string of characters to a table with each character (C) and its hex representation (C_HEX) in each row.
You can run it as is to check if you get the same output. It must be as described for a UTF-8 database.
Now try to comment out the line with values ('éó') and uncomment the select statement returning some row with these special characters.
If you see the same hex representation of these characters stored in the firstn column, then this means, that the string is stored appropriately, but your client tool (SPSS Modeller) can't show these characters correctly due to some reason (wrong font, for example).
I need to read in Delphi the output from a SQL Server procedure that returns a XML as output (#MSG1)
PROCEDURE dbo.PROC_ZUND_XML
(#LOTEPC VARCHAR(10),
#MSG1 NVARCHAR(MAX) OUTPUT)
I've tried to get this output using a TFDStoredProc component as well as a TFDQuery, however, it doesn't matter the variable type I use in the parameters configuration in Delphi (ftWideString, ftWideMemo etc.) it always store just the first 8000 characters of the output. A count statement in the SQL Command section of the Delphi's FireDAC Query Editor shows that the output of the procedure have more than 8000 characters.
So, the problem seems to be occurring internally, when the value is assigned to the variable. Does anybody have an idea about how to fix it? Maybe a different approach to the problem... Thanks in advance.
I'm using SQL Server 2008 and Delphi 10.2
If you need to export this xml file to a specific folder, I advise you to use SQL SERVER itself to export this file to a SHARED NETWORK FOLDER.
With BCP, you can generate your XML, and save directly to an XML file, in the directory you want ..
Some useful tips:
If necessary, I recommend using the -w argument to avoid confusion
with Wrong CHARACTERS (À, É, È, Ó, Ò)
Useful Documentations:
http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=171529
https://learn.microsoft.com/en-us/previous-versions/sql/sql-server-2012/ms162802(v=sql.110)
https://sqlsouth.wordpress.com/2014/05/23/export-xml-from-sql-server-using-bcp/
A part of a project with one of our clients deals with creating full text indexes in Hebrew 1037 (Database has both english and hebrew in it) that will correspond with a UI system written in Angular and C# . The database itself is collated to Latin1_General_100_CI_AS.
The stored procedure gets a parameter for search that suppose to get single word input and the code inside the SP uses that parameter for different full text search queries.
first weird thing that happens is that when i pass a string with or without quotation marks, everything works fine. I read that this is a known bug and Microsoft do not intend to fix it anytime soon.
The real problem lies with the actual string passed.
When i pass a string in Hebrew or a numeral value - everything works fine and the SQL Server fetches the relevant data (Through UI and SSMS).
When i pass a string in English through SSMS, SQL Server replies with a relevant data but when the UI system passes a parameter it gets no data back.
The following SQL code is the code im using which is very basic:
ALTER PROCEDURE dbo.SearchByFreeText
#StringToSearch NVARCHAR(500) = NULL
AS
SELECT *
FROM SampleTable
WHERE CONTAINS(SampleColumn, #StringToSearch)
OR
FREETEXT(SampleColumn, #StringToSearch)
I will very appreciate some help on the matter.
Thanks!
Try To understand below diffrent :
SELECT * FROM content WHERE freetext(description,"SQL MGT")
Although this appears to search on the string "SQL MGT," it actually searches on "SQL" or "MGT."
SELECT * FROM content WHERE CONTAINS(description, 'NT NEAR great')
This search on the Description column finds all rows where "NT" is near "great".
Try setting the language_term explicitly when calling CONTAINS or FREETEXT. From the docs:
CONTAINS (... [ , LANGUAGE language_term ] )
language_term is the language to use for word breaking, stemming, thesaurus expansions and replacements, and noise-word (or stopword) removal as part of the query.
language_term can be specified as a string, integer, or hexadecimal value corresponding to the LCID of a language. If language_term is specified, the language it represents will be applied to all elements of the search condition. If no value is specified, the column full-text language is used.
Looks like you do have a FTS language mismatch and SSMS somehow manages to determine the correct one when you run your query in SSMS.
So, when passing English string from UI SQL Server probably tries to check it against Hebrew index. Not as final solution but try to hard code the English language code (1033) into SQL query and pass English search term from UI to see if anything will be found:
SELECT *
FROM SampleTable
WHERE CONTAINS(SampleColumn, #StringToSearch, LANGUAGE 1033)
OR
FREETEXT(SampleColumn, #StringToSearch, LANGUAGE 1033)
If it does find results then you will probably have to determine the language of the search term on the fly and enable the correct LANGUAGE param via Dynamic SQL or IF (#lang = 1033) block.
HTH
I was tasked with moving an installation of Orchard CMS to a different server and domain. All the content (page content, menu structure, links, etc.) is stored in an MSSQL database. The good part: When moving the physical files of the Orchard installation to the new server, the database will stay the same, no need to migrate it. The bad thing: There are lots and lots of absolute URLs scattered all over the pages and menus.
I have isolated / pinned down the tables and fields in which the URLs occur, but I lack the (MS)SQL experience/knowledge to do a "search - replace". So I come here for help (I have tried exporting the tables to .sql files, doing a search-replace in a text editor, and then re-importing the .sql files to the database, but ran into several syntax errors... so i need to do this the "SQL way").
To give an example:
The table Common_BodyPartRecord has the field Text of type ntext that contains HTML content. I need to find every occurance of the partial string /oldserver.com/foo/ and replace it with /newserver.org/bar/. There can be multiple occurances of the pattern within the same table entry.
(In total I have 5 patterns that will need replacing, all partial string / substrings of urls, domains/paths, etc.)
I usually do frontend stuff and came to this assignment by chance. I have used MySQL back in the day I was playing around with PHP related stuff, but never got past eh basics of SQL - it would be helpful if you could keep your explainations more or less newbie-friendly.
The SQL server version is SQL Server 9.0.4053, I have access to the database via the Microsoft SQL Server Management Studio 12
Any help is highly appreciated!
You can't manipulate the NTEXT datatype directly, but you can CAST it to VARCHAR(MAX), then use the REPLACE function to perform the string replacement, then CAST it back to NTEXT. This can all be done in a single UPDATE statement.
update MyTable
set MyColmun = cast(replace(cast(MyColumn as nvarchar(max)), N'/oldserver.com/foo/', N'/newserver.org/bar/') as ntext)
where cast(MyColumn as nvarchar(max)) LIKE N'%/oldserver.com/foo/%'
The WHERE clause in the UPDATE statement below is used to prevent SQL Server from making non-changes, i.e. if the value does not need to be changed then there is no need to update it to itself.
The CAST function is used to change the data type of a value. NTEXT is a legacy data type used for storing large character values, NVARCHAR(MAX) is a new and more versatile data type for storing large character values. The REPLACE function can not operate on NTEXT values, hence the need to CAST it to NVARCHAR(MAX) first, do the replace, then CAST it back to NTEXT afterwards.
I'm attempting to run a simple statement against an Access DB to find records.
Data validation in the records was horrible, and I cannot sanitize it. Meaning, it must be preserved as is.
I need to be able to search against a string with white space and hyphen characters removed. The following statement will work in Access 2010 direct:
select * from dummy where Replace(Replace([data1],' ',''),'-','') = 'ABCD1234';
Running it from an ODBC connection via PHP will not. It produces the following error:
SQL error: [Microsoft][ODBC Microsoft Access Driver] Undefined function 'Replace' in expression., SQL state 37000 in SQLExecDirect
Creating a query in the database that runs the function and attempting to search its values indirectly causes the same error:
select * from dummy_indirect where Expr1 = 'ABCD1234';
I've attempted to use both ODBC drivers present. ODBCJR32.dll (03/22/2010) and ACEODBC.dll (02/18/2007). To my knowledge these should be current as it was installed with the full Access 2010 and Access 2010 Database Engine.
Any ideas on how to work around this error and achieve the same effect are welcome. Please note, that I cannot alter the database in way, shape, or form. That indirect query was created in another mdb file that has the original tables linked from the original DB.
* Update *
OleDB did not really affect anything.
$dsn= "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=c:\dummy.mdb;";
I'm not attempting to use it as a web backend either. I'm not a sadomasochist.
There is a legacy system that I must support that does use Access as a backend. Data gets populated there from other old systems that I must integrate into more modern systems. Hence, the creation of an API with Apache/PHP that is running on the server supporting the legacy system.
I need to be able to search a table that has an alphanumeric case identifier to get a numeric identifier that is unique and tied to a generator (Autonumber in access). Users have been using it a trash box for years (inconsistent data entry with sporadic notations) so the only solution I have is to strip everything except alphanumeric out of both the field value and the search value and attempt to perform a LIKE comparison against it.
If not replace() which is access supported, what ODBC compatible functions exist that I can use do the same kind of comparison?
Just to recap, the Access db engine will not recognize the Replace() function unless your query is run from within an Access application session. Any attempt from outside Access will trigger that "Undefined function" error message. You can't avoid the error by switching from ODBC to OleDb as the connection method. And you also can't trick the engine into using Replace() by hiding it in separate query (in the same or another Access db) and using that query as the data source for your main query.
This behavior is determined by Access' sandbox mode. That linked page includes a list of functions which are available in the default sandbox mode. That page also describes how you can alter the sandbox mode. If you absolutely must have Replace() available for your query, perhaps the lowest setting (0) would allow it. However, I'm not recommending you do that. I've never done it myself, so don't know anything about the consequences.
As for alternatives for Replace(), it would help to know about the variability in the values you're searching. If the space or dash characters appear in only one or a few consistent positions, you could do a pattern match with a Like expression. For example, if the search field values consist of 4 letters, an optional space or dash, followed by 4 digits, a WHERE clause like this should work for the variations of "ABCD1234":
SELECT * FROM dummy
WHERE
data1 = 'ABCD1234'
OR data1 Like 'ABCD[- ]1234';
Another possibility is to compare against a list of values:
SELECT * FROM dummy
WHERE
data1 IN ('ABCD1234','ABCD 1234','ABCD-1234');
However if your search field values can include any number of spaces or dashes at any position within the string, that approach is no good. And I would look real hard for some way to make the query task easier:
You can't clean the stored values because you're prohibited from altering the original Access db in any way. Perhaps you could create a new Access db, import the data, and clean that instead.
Set up the original Access db as a linked server in SQL Server and build your query to take advantage of SQL Server features.
Surrender. :-( Pull in a larger data set to your PHP client code, and evaluate which rows to use vs. which to ignore.
I'm not sure you can do this with ODBC and your constraints. The MS Access driver is limited (by design; MS wants you to use SQL Server for back ends).
Can you use OLEDB? that might be an option.