A part of a project with one of our clients deals with creating full text indexes in Hebrew 1037 (Database has both english and hebrew in it) that will correspond with a UI system written in Angular and C# . The database itself is collated to Latin1_General_100_CI_AS.
The stored procedure gets a parameter for search that suppose to get single word input and the code inside the SP uses that parameter for different full text search queries.
first weird thing that happens is that when i pass a string with or without quotation marks, everything works fine. I read that this is a known bug and Microsoft do not intend to fix it anytime soon.
The real problem lies with the actual string passed.
When i pass a string in Hebrew or a numeral value - everything works fine and the SQL Server fetches the relevant data (Through UI and SSMS).
When i pass a string in English through SSMS, SQL Server replies with a relevant data but when the UI system passes a parameter it gets no data back.
The following SQL code is the code im using which is very basic:
ALTER PROCEDURE dbo.SearchByFreeText
#StringToSearch NVARCHAR(500) = NULL
AS
SELECT *
FROM SampleTable
WHERE CONTAINS(SampleColumn, #StringToSearch)
OR
FREETEXT(SampleColumn, #StringToSearch)
I will very appreciate some help on the matter.
Thanks!
Try To understand below diffrent :
SELECT * FROM content WHERE freetext(description,"SQL MGT")
Although this appears to search on the string "SQL MGT," it actually searches on "SQL" or "MGT."
SELECT * FROM content WHERE CONTAINS(description, 'NT NEAR great')
This search on the Description column finds all rows where "NT" is near "great".
Try setting the language_term explicitly when calling CONTAINS or FREETEXT. From the docs:
CONTAINS (... [ , LANGUAGE language_term ] )
language_term is the language to use for word breaking, stemming, thesaurus expansions and replacements, and noise-word (or stopword) removal as part of the query.
language_term can be specified as a string, integer, or hexadecimal value corresponding to the LCID of a language. If language_term is specified, the language it represents will be applied to all elements of the search condition. If no value is specified, the column full-text language is used.
Looks like you do have a FTS language mismatch and SSMS somehow manages to determine the correct one when you run your query in SSMS.
So, when passing English string from UI SQL Server probably tries to check it against Hebrew index. Not as final solution but try to hard code the English language code (1033) into SQL query and pass English search term from UI to see if anything will be found:
SELECT *
FROM SampleTable
WHERE CONTAINS(SampleColumn, #StringToSearch, LANGUAGE 1033)
OR
FREETEXT(SampleColumn, #StringToSearch, LANGUAGE 1033)
If it does find results then you will probably have to determine the language of the search term on the fly and enable the correct LANGUAGE param via Dynamic SQL or IF (#lang = 1033) block.
HTH
Related
I have a stored procedure which searches by text passed by a parameter. I noticed that if the text is not in English (i.e. Hebrew, Arabic), the query finishes without returning any rows. I am sure there is data to be found
I dont know which SQL server is being used : it is whatever is provided by GoDaddy on shared Windows hosting plan.
The thing is, I have a asp.net site which can search and fetch the data from this column encoded as UTF-8. The path which does not work is the MS SQL Management Studio. Even when I manually run same stored procedures which work from inside asp.net - they dont manage to find non English characters. The parameters I pass to the query are prefixed by N qualifier.
Try using the N character like the sample :
Select * from students where name like N'%بیژن%'
and as mentioned in the comment the column data type should be Nvarchar.
Can anybody tell me what does the error mean? Whenever I open the query builder it will prompt with an error indicating that SQL syntax errors were encountered.
https://msdn.microsoft.com/en-us/library/ms189012.aspx
I looked at the following page in MSDN but I don't understand what it means...
For instance, what do these bullet points from the MSDN article mean?
The SQL statement is incomplete or contains one or more syntax errors.
The SQL statement is valid but is not supported in the graphical panes (for example, a Union query).
The SQL statement is valid but contains syntax specific to the data connection you are using.
USER (which you've apparently decided is an appropriate table name) is a SQL Server reserved word.
The best solution is to rename your table, so you don't have to escape the table name every time you want to query it and to make it clear it's your user data (hey, there's a table name suggestion - userdata).
The other option is to escape the name by surrounding it with square brackets:
SELECT * FROM [users]
Note that it will get old fast having to do this with every query. Again, the best solution would be to rename the table to something that isn't a reserved word.
Consider below query containing both Persian(a right to left language) and English(a left to right language):
SELECT 'نرم افزار SQL سرور'
the required result is this string :
سرور SQL نرم افزار
Is there any function or any other way to converting string from ltr to rtl??
It is required to add N before string literal: SELECT N'نرم افزار SQL سرور'. This is needed to correctly interpret contained Unicode characters. (Source)
Important: In some cases, please avoid using standard copy-paste in order to put SELECT into SSMS command window. This could affect the RTL/LTR order. Instead, try to open correctly composed file using File > Open.
And regarding your comment:
the result should be : سرور SQL نرم افزار`
I admit I understand RTL writing system only partially, but from what I can see, Persian words are put to the output exactly in order as you entered them (even if reading right to left). Could you show me based on Unicode Bidirectional Algorithm or similar standards document why the word order should be changed by SQL Server? Shouldn't be change you expect made by preprocessing on another place, sending expected string form SELECT N'سرور SQL نرم افزار'? I see no point why just SQL SELECT should perform the change. If it did, what would happen if you feed result of such a SELECT into another SELECT? Another transformation? I have reasons to think that SQL server is interpreting your input technically correctly.
Hint: maybe you can try to surround your RTL text by different Directional formatting characters.
Please try the same SELECT with MySQL server at SQL Fiddle. Different server and technology, but the same result as Microsoft SQL Server gave.
Result from SSMS with MS SQL Server:
Conclusion: in order to get expected result, please form the input accordingly.
Related: Transformation of word order you expected can be done by appropriate settings in user interface.
When we add digit with english this will again not work following solution will work
SELECT nchar(8234)+ N' 33-M ' + N'کلینک کمرہ نمبر' +nchar(8236) + N'میں تشریف لائیں'
I was tasked with moving an installation of Orchard CMS to a different server and domain. All the content (page content, menu structure, links, etc.) is stored in an MSSQL database. The good part: When moving the physical files of the Orchard installation to the new server, the database will stay the same, no need to migrate it. The bad thing: There are lots and lots of absolute URLs scattered all over the pages and menus.
I have isolated / pinned down the tables and fields in which the URLs occur, but I lack the (MS)SQL experience/knowledge to do a "search - replace". So I come here for help (I have tried exporting the tables to .sql files, doing a search-replace in a text editor, and then re-importing the .sql files to the database, but ran into several syntax errors... so i need to do this the "SQL way").
To give an example:
The table Common_BodyPartRecord has the field Text of type ntext that contains HTML content. I need to find every occurance of the partial string /oldserver.com/foo/ and replace it with /newserver.org/bar/. There can be multiple occurances of the pattern within the same table entry.
(In total I have 5 patterns that will need replacing, all partial string / substrings of urls, domains/paths, etc.)
I usually do frontend stuff and came to this assignment by chance. I have used MySQL back in the day I was playing around with PHP related stuff, but never got past eh basics of SQL - it would be helpful if you could keep your explainations more or less newbie-friendly.
The SQL server version is SQL Server 9.0.4053, I have access to the database via the Microsoft SQL Server Management Studio 12
Any help is highly appreciated!
You can't manipulate the NTEXT datatype directly, but you can CAST it to VARCHAR(MAX), then use the REPLACE function to perform the string replacement, then CAST it back to NTEXT. This can all be done in a single UPDATE statement.
update MyTable
set MyColmun = cast(replace(cast(MyColumn as nvarchar(max)), N'/oldserver.com/foo/', N'/newserver.org/bar/') as ntext)
where cast(MyColumn as nvarchar(max)) LIKE N'%/oldserver.com/foo/%'
The WHERE clause in the UPDATE statement below is used to prevent SQL Server from making non-changes, i.e. if the value does not need to be changed then there is no need to update it to itself.
The CAST function is used to change the data type of a value. NTEXT is a legacy data type used for storing large character values, NVARCHAR(MAX) is a new and more versatile data type for storing large character values. The REPLACE function can not operate on NTEXT values, hence the need to CAST it to NVARCHAR(MAX) first, do the replace, then CAST it back to NTEXT afterwards.
The website i worked was recently attempted to be hacked by the following SQL injection script
boys' and 3=8 union
select 1,
concat(0x232425,ifnull(`table_name`,0x30),char(9),ifnull(`table_rows`,0x30), char(9),0x252423),
3,4,5,6,7,8,9
from `information_schema`.`tables`
where table_schema=0x62646B3032 limit 44,1 -- And '8'='8
This injection returned the mysql table name. This was reported by the error reporting system on that website and we managed to fix that part however I am not able to understand what does the above injection mean?
Anyone can explain this?
Penuel
They're using a select from the Information Schema views in mysql server :
http://dev.mysql.com/doc/refman/5.0/en/information-schema.html
They use some clever hacks to rout out simple sql injection prevention techniques.
According to this the MySQL concat()
Returns the string that results from
concatenating the arguments. May have
one or more arguments. If all
arguments are nonbinary strings, the
result is a nonbinary string. If the
arguments include any binary strings,
the result is a binary string. A
numeric argument is converted to its
equivalent binary string form
So 0x232425 is converted to #$% which is simply added to the begining and end of the table_name field. Maybe just to make it easier for them to pull out the Table names later using Regex.
Later on the char(9) is equivalent to a tab as you can see here and is just there to format the output nicer.
The 3,4,5,6,7,8,9 is just there so that the columns match the boys table that they are performing the Union on.
This injection returned the mysql table name.
Do you mean that your website displayed the table name when you gave it this input, or that the query returns that when run from the mysql client? If it showed on your website, then the attacker has the ability to inject much more harmful queries. Check your data.