I have a database encoded in utf8 and I want to export it to a dump file. The problem is that when I do it the data in the dump file are not encoded in utf8. Is there a way to define the encoding when creating the dump file ?
Your DB when you created it, may have been using another form of encoding aside from UTF8. You may want to refer to this article about how to change encoding settings. Hopefully once that has been changed you will be able to export.
https://dev.mysql.com/doc/refman/5.0/en/charset-applications.html
This doc will show you how to encode per table, as well as how to change your encoding via CLI.
Cheers.
Related
I'm publishing my CSV files to CKAN using the API. I want to make my data easy to open in Brazilian Excel, so it must have:
semicolon ";" separated columns
coma "," as a decimal separator
use encoding cp-1252
I'm using Data Store and Data Pusher.
My problem is that if I upload my data with encoding cp1252, Data Pusher sends it as is to the Data Store that expects the data as UTF-8. The data preview doesn't display the accents correctly. In the image below Março were the correct value to display:
I want to have my user downloading the data as cp-1252, so it opens easily in Excel, but also have CKAN displaying it correctly. I must specify the encoding of the file while uploading the file.
I couldn't specify the encoding directly, but taking a look at Data Pusher source I saw that it uses the Messy Tables library. MT obey the environment locale set of the host machine, so I configure it to pt_BR.UTF8 and my accents worked fine.
Now I can publish my data with commas as a decimal separator and using encoding Windows-1252. The data correctly opens in Excel when it is downloaded, and is also correctly displayed in the Data Explorer.
Here is a scenario.
We are collecting programmatically some data into Javascript object which contain Polish characters.
Then we are converting it to string with JSON2CSV library and sending to Azure blob with uploadBlockBlob method from #azure/storage-blob library.
After a while we are using Azure Functions triggered by blob storage trigger. We get "Myblob" property with string with CSV content. Then we are using Papaparse library to convert it back to object. And finally we are using content of the object to update database through mssql library.
In the process we are loosing polish characters.
JSON2CSV when converting to string does not seem to have "encoding" property exposed. Nor the uploadBlockBlob. With Papaparse enforcing encoding with "UTF-8" does not have any effect on the process (changing to cp1250 also does not help). The original content is scraped from a web page with software running on WIndows machine.
Any ideas how to preserve the encoding throughout the pipeline?
Closing the question as it appeared to be my mistake. I pushed Varchar type into NVarchar column - that is why non-latin characters got wrong.
I am using CGI DSPSTMF command to display stmf file on web browser. I am copying a spool file to a stmf file using CPYSPLF *STMF option. Once copied i am passing IFS location to DSPSTMF command but it is going to download automatically and when i open the download file i am getting all Junk data any idea why?
Also, i noticed it is using CONTTYPES file in CGILIB and on my server it is empty. What should be the values in it and what should i do show correct data instead of junk. I tried to use different methods to copy the file to IFS like used cpytostmf instead of cpysplf but on IFS file looks correct not the download version.
What CCSID is the resulting stream file tagged with?
use WRKLNK and option 8=Display attributes
If 65535, that tells the system the data is binary and it won't try to translate the EBCDIC to ASCII.
The correct fix is to properly configure your IBM i so that the stream file is tagged with it's correct CCSID.
Do a WRKSYSVAL QCCSID ... if your system is still set to 65535, that's the start of your problem. But this isn't programming related, you can try posting to Server Fault but you might get better responses on the Midrange mailing list
I'm having trouble saving BigQuery table data having special characters as json with the proper encoding. This issue is only via the UI. When I save from the CLI the character encoding is proper. In the table the data is correctly encoded. I'm just clicking 'Save Result' and save as JSON local file. While loading data to the table the data was UTF-8 and properly encoded too. Sample special char: Bié
How to save table data with special characters from the UI correctly? I can use cli but its easier to use the UI if I can get this to export correctly.
As discussed in the comments, I could not reproduce the same problem. Since the results returned by the BigQuery Console and CLI should be the same I suggested that this could be a problem in the text editor which was confirmed.
We run a little java web application on a hsqldb 2.4 database, and apparently the encoding of the .lobs file where blobs are persisted depends on the OS holding it. For instance, it is encoded in Ansi when the application runs on windows, which is problematic when we want to send on a Linux system (via http) the files represented by its blobs.
Do anyone know how to specify the encoding of this .lobs file ?
HSQLDB does not encode the data in blobs. It stores the data exactly as inserted. If you need to encode the files represented by the blobs in a different character set, you have to do it in your application.