Arabic and English text in PostgreSQL database - sql

Question

I need to insert both English and Arabic text into a PostgreSQL database.
I'm running the following command via a .bat script:
psql.exe --echo-all --username=postgres --dbname=dbname -f populate.sql
populate.sql has statements like this:
insert into table1 (column1, column2) VALUES (2, 'المستخدم ');
If I do this via pgadmin, it works. The thing is I need to do this via some .sql population scripts that are ran once the application is started.
In that case I get gibberish, like this:
العرض
I created the scripts in Notepad++ using the Encode in UTF-8 without BOM option, since the normal encoding in UTF-8 adds an extra character to the start of the file and some of the inserts are not made.
I'm assuming this is an encode problem, but I have yet to figure out exactly what is wrong.
The databse is in UTF-8.
Thanks in advance!

Answer 1

The Windows console doesn't speak Unicode of any form by default, it speaks a "native" codepage. Which codepage depends on your Windows install's language settings.
If you chcp 65001 then it'll switch into utf-8.
Overall, though, text encoding handling in batch/cmd files and the Windows command line is absolutely awful. I generally recommend that you instead put anything that isn't basically 7-bit ASCII into a separate .sql file and execute it via psql -f with a suitable client_encoding; the PGCLIENTENCODING environment variable is useful for this.
So try:
SET PGCLIENTENCODING=utf-8
psql.exe --username=postgres --dbname=dbname -f populate.sql