SSMS 2014 - DB Collation for unicode/ multiple languages - sql

In SSMS 2014, I have a DB with collation set as Chinese_Simplified_Stroke_Order_100_CI_AI, in which I create a table for a regular process with about 50 columns in it.
It basically contains sales data of some products. Few of these columns have integers as values and others contain English text.
Two of these columns, however contain values in the form of text which is in Chinese. A sample script that I update the column with is as below:
ALTER TABLE table_xyz
ALTER COLUMN comments NVarchar (max)
COLLATE Chinese_Simplified_Stroke_Order_100_CI_AI
UPDATE table_xyz
SET comments =
CONCAT('以下的邮件是专为您的最终客户所准备。', Account_Name)
This has been working fine till now.
I have a similar table in a second DB with collation set as Japanese_CI_AS_KS_WS and accordingly the comments in this table are in Japanese. Sample update statement as below:
ALTER TABLE table_abc
ALTER COLUMN comments NVarchar (max) COLLATE Japanese_CI_AS_KS_WS
UPDATE table_xyz
SET comments =
CONCAT('次の電子メールは顧客のためのテンプレートです', Account_Name)
Now, I have been tasked to transfer these tables to an existing DB that has SQL_Latin1_General_CP1_CI_AS collation set as default. The problem is I whenever I update the above two tables in this new DB, all I get in the output is '???????'
I have tried searching for solutions and have observed the below:
Many suggestions include converting the datatype to unicode.
A few people at my workplace suggested changing the collation of the column.
Use UTF-8 as default character set
As per my knowledge the first two are already taken care of when I run the Alter table statement. The third point seems valid for MySQL and not SQL Server.
Also, if I import the table from the respective DBs directly along with the data, the column values are displayed correctly (in Chinese and Japanese text). However, when I truncate and try to load the data I face the problem. I would be unable to import data in this way, since the end objective is to keep all tables in a single DB and purge the remaining DBs.
Hope I've made the problem statement clear.

You need to use N at the beginning of string literal containing UNICODE characters.
The code in your first example should be like this.
CONCAT(N'以下的邮件是专为您的最终客户所准备。', Account_Name)

Related

Problem with data into MariaDB using the SELECT clause in WHERE section

I don't know how to explain but I'll try, into my database in a table, I have one record with many fields.
The username field, for example, contains the value = 'any-user-test' but if I execute a "SELECT" clause and in the WHERE section I compare username='any-user-test' the result does not contain the record.
But if I compare using username LIKE '%any-user-test' the record is returned.
And as further proof using:
WHERE CONVERT(username USING ASCII) = 'any-user-test'
the record is returned too.
The database is MariaDB in a server Ubuntu using encryption and CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci.
Any idea how to identify the problem?

Server 2012 R2 SQL Database Change Column From ntext to nvarchar

I have a predefined SQL data base that we have to work correctly with a reporting software we have purchased. When ever we pull a column of data with the reporting software we get system.indexoutofrangeexception error. On the first table we replaced all Semi Colons ';' with space within the data and this corrected the issue. This column does not have any other special characters within the data only semi colons.
However the data in the second column we need to query contains all different kinds of characters that are probably invalid. The column type is ntext and would like to either change the data directly in the sql database everytime there is a new entry or would changing the format to nvarchar(max) or nvarchar(1024) be suffice?
Thanks for the support I am beyond green at sql.
Your problem is most likely not related to the datatype in your database but the data itself.
Your reporting software seems to have specific requirements that your data does not meet.

Hearts change to question mark when I insert data into the database. How do I change the charset of the table?

There is a video file named ♥-You-Got-Me-♥[www.savevid.com].mp4. But as the file with this name is inserted into a SQL Server 2005 database the hearts change to ?.
So the name turns into ?-You-Got-Me-?[www.savevid.com].mp4.
I don't know how to change the character set of the database? How do I change the char set of my table so that it can over all the characters ?
It will be great if along with the command,graphical method to do so is included in the answer.
You don't need to change the character set of the database. As long as you are using the NVARCHAR type, you should be good on the database side. However, you have to make sure that however you get the data into the table takes Unicode into account:
DECLARE #VAR VARCHAR(100) = N'♥-You-Got-Me-♥[www.savevid.com].mp4'
, #NVAR NVARCHAR(100) = N'♥-You-Got-Me-♥[www.savevid.com].mp4'
, #oops NVARCHAR(100) = '♥-You-Got-Me-♥[www.savevid.com].mp4'
SELECT
#VAR
, #NVAR
, #oops;
Returns:
?-You-Got-Me-?[www.savevid.com].mp4 ♥-You-Got-Me-♥[www.savevid.com].mp4 ?-You-Got-Me-?[www.savevid.com].mp4
The last declaration omits the N in front of the literal. There are similar ways to mess this up in your front end. Even if the DB stores Unicode, you have to make sure that everything between input and the DB, and then back out to your UI, handles multi-byte characters properly.
It isn't a CHARSET problem but a datatype problem in SQL Server. SQL Server doesn't have CHARSET as such like MySQL and Collations are for code page, sorting and comparing
You need to use nvarchar to store unicode (basically non-latin) data properly.
The problem is likely using VARCHAR, if possible changing to a NVARCHAR type should resolve the problem for new entries. If you cannot change the column type it may get more complicated.
SQL Fiddle
Change the type of the field storing your filename from varchar to nvarchar.
For example:
Table:
FilenameId INT IDENTITY
Filename NVARCHAR(200)
SQL to insert data:
INSERT INTO TestTable ([Filename]) VALUES(N'♥-You-Got-Me-♥[www.savevid.com].mp4')

SQL converting nvchar to ntext

What I have is a staging table that is all nvarchar (so i can load it easily). In my live table i have a bunch of ntext items. I have the following:
obviously this isnt the whole query:
update
SLTDS_C69_Stdtable
set
[AARIssue] = convert(ntext, st.[AARIssue]),
[AttachmentIDs] = convert (ntext, st.[AttachmentIDs])
I get this error returned:
types ntext and nvarchar are incompatible in the equal to operator.
ANy idea how to fix this?
Do you want to replace the text or add to it?
INthe first case you don't need to convert at all, just set the filed to the value of the filed inthe other table. INthe second case you need to use UPDATE text.
However, you have a problem in that ntext is deprecated, you should consider converting these fields to nvarchar(max) as soon as possible unless you are still running SQL Server 2000.
Is this a repeat of this: How to update a text or ntext field in SQL Server 2000
The live table should use UPDATETEXT?

How to insert Arabic characters into SQL database?

How can I insert Arabic characters into a SQL Server database? I tried to insert Arabic data into a table and the Arabic characters in the insert script were inserted as '??????' in the table.
I tried to directly paste the data into the table through SQL Server Management Studio and the Arabic characters was successfully and accurately inserted.
I looked around for resolutions for this problems and some threads suggested changing the datatype to nvarchar instead of varchar. I tried this as well but without any luck.
How can we insert Arabic characters into SQL Server database?
For the field to be able to store unicode characters, you have to use the type nvarchar (or other similar like ntext, nchar).
To insert the unicode characters in the database you have to send the text as unicode by using a parameter type like nvarchar / SqlDbType.NVarChar.
(For completeness: if you are creating SQL dynamically (against common advice), you put an N before a string literal to make it unicode. For example: insert into table (name) values (N'Pavan').)
Guess the solation is first turn on the field to ntext then write N with the value. For example
insert into eng(Name) values(N'حسن')
If you are trying to load data directly into the database like me, I found a great way to do so by creating a table using Excel and then export as CSV. Then I used the database browser SQLite to import the data correctly into the SQL database. You can then adjust the table properties if needed. Hope this would help.