I am lost on this. I have a web site that I work on offline, then I upload to live server, import MySQL database to live server, and make live in general. Problem is that when I export database from LIVE server and import back into LOCAL server, I get unknown characters in my XHTML output, the black diamond with the white question mark in the middle, you know the one.
The command I used to export my database from LOCAL MySQL installation is as follows:
mysqldump --verbose -hlocalhost -uxxx -pxxx --databases xxx --add-drop-table --default-character-set UTF8 > somedir/xxx.sql
Then, when importing that data to the LIVE database I used a simple command as follows:
mysql -hlocalhost -uxxx -pxxx somedatabase < xxx.sql
Great, all is well, data is imported and there are NO unknown characters on site. However, when I do the same in reverse (dump on live server using same command as in windows, import in windows using same mysql command) that is when all the unknown characters appear.
Is it a collation issue? Am I using the incorrect --default-character-set ?
I have never had this happen before, and I would love some feedback or a nudge in the right direction.
UPDATE: I have completely dropped my local one, exported the ENTIRE live db and imported. Still the same error :/ This is driving me mad!
Many thanks,
Simon
Try on both sides:
SET names UTF-8
Probably the defaults differ.
I suspect your live site has a different database schema, could it be latin1?
Related
I have spent quite a bit of time scouring the overflow forums and using other resources to try and figure this out without success. My apologies if this has been answered elsewhere. But, to my knowledge it hasn't
Here is a little background. I have some experience with SQL, Visual basic and Auto Hot Key (awesome program). Currently I am trying to learn more than the basic stuff I know about SQL (little rusty too).
For my previous and current job I work(ed) a lot with IBM's iSeries (or 'Mainframe' as some veterans like to call it.) Specifically it's version: IBM DB2 PE 9.7 FP5 SRM R1 on the DB2 i5/OS.
As you may know, iSeries related emulators have a feature that allows its users to record macros for later playback. This feature of course has its limitations. I have written some from scratch in Visual Basic that are more flexible in the operations that they can perform. However, you can only do so much with VB.
Regardless of how you look at it, iSeries is slow IMO. In order to pass/retrieve information to/from its database/server its users are limited to the speed of the program (among other things). VB macros are subject to smart pauses/timeouts as these are needed for scripts to work (like auto hot key)
iSeries does have a feature where you can query information, interact with libraries (schemas if I have that right?) and tables using the i5/OS Query manager. It also has an FTP feature (which has its uses I guess).
I have started playing around with Powershell and SSH/Qshell, I have read a lot of documenation in IBMs extensive support section on their site regarding iSeries and all things related.
So here are the questions:
Is qshell/SSH an efficient way to retrieve information from a database, or more specifically, the libraries that 'DB2 for iSeries' uses (.lib/.file extension files?)?
Wouldn't writing SQL scripts and executing them via SSH/Qshell be faster than using the iSeries emulator itself?
Is this only possible via port forwarding or tunneling?
How do I find out more about the server via SSH commands? I can navigate the servers directories pretty easily and view files that I otherwise cannot see within iSeries. Its database files are .lib/.file/.mbr file extensions that I cannot view using the 'cat' command. This would require using SQL commands I am assuming?
Do I need to log on as a root user in order to have read/write access privelages and perform anything SQL related?
As mentioned above I have played around with ssh/qshell but I am having difficulty bringing up the mysql/sql prompt to do anything (I am using openSSH 4.7p1 / a unix CLI?) which I believe changes from $ to > when you successfully do this. I have been able to do this but am unable to execute any SQL related commands so I am not sure what I am doing wrong.
Below is an example of me logging into the server over ssh via powershell and trying to execute something SQL related:
PS H:> ssh -1 myusername#blabla.something.com "mysql -u myusername -e 'show tables;'"
enter password: password
connected
Could not chdir to home directory /home/myusername: A file or directory in the path name does not exist
bsh: mysql: execute permission denied
bsh: cannot execute
Connection closed to blabla.something.com
This is an example of just logging in over ssh as normal:
ssh -1 myusername#blabla.something.com
Could not chdir to home directory /home/myusername: A file or directory in the path name does not exist
$ Mysql
mysql: execute permission denied
$ "mysql
'show databases;'"
mysql
'show databases;'" not found
Any insight as to what I am doing wrong or feedback on my questions is greatly appreciated. I know there are alternatives to what I am trying to do using PuTTY (not really an option at work), ODBC drivers (can't really download an IBM iSeries fixpack to get the drivers or repair the installation of iSeries as I can't anyways I don't have Windows admin privs)
UPDATE
First, I want to thank you all for your comments and your insight.
Warren:
Thank you, for you insight. IBM DB2 PE 9.7 FP5 SRM R1 is indeed DB2 LUW, or "COBRA". This particular emulator is an adaption of IBM's 3270 emulator. This knock off brought to you by attachmate (they added a couple bells and whistles and no longer offer support, ODBC drivers etc). Within this version, I have the option to utilize IBM DB2 Query Manager for i5/OS. I would agree that the server is partitioned (I think?) as I have been able to call different prompts like DB2/SQLJ/JAVA, just haven't been too successful with using them lol. Perhaps my attempts below will reveal a little more.
James:
Thank you for your input, I never thought to do this from powershell directly. As for the redbook, have it! I'm still reading it though. This may be a little more advanced than where I currently stand with powershell and ADO.NET. I will need to take a more in depth look at your coding example to gain a better understanding of it.
Buck:
I appreciate your point by point feedback. Not sure what your setup is, but IBM i in general (especially anything after i5) is insanely fast. The company I work for.. or at least the immediate people I work with, know very little in regards to i/vba/qsh/etc. To put it politely. So, much of what I have learned has been through reading and applying what I learn. This, actually, is only my second post. On any website, ever. I don't like to just post away because I can't figure something out in a minute. I'm stubborn and will spend hours if I have to make a script work.
As for STRSQL, the equivalent I use is STRQRY. From reading IBM's i5/OS Query Manager PDF (300 pgs -_-). I would agree that this is a very powerful tool. The only drawback is 1. you have to make a form that will format the query and 2. You have to create the Query using SQL . This is no big deal whatsoever. Unfortunately, there are hundreds of schemas, and sometimes thousands of .FILE extensions within any given LIB (among other extension types). Luckily I have narrowed this down to the primary LIB that I access most. Unfortunately, there are something like 4000+ files extensions within this library and files within some of those files (︻╦╤─ ^ _ ^). Going through all of them is a little time consuming to say the least.
There are some "macros" that perform a query in batch and then prints the data to a formatted file onto the server (which takes erm... like 30 mins for 500-600 pages). Is there any way I could view the parameters of this batch run? "Work with queries/query forms?"
The primary reason I am more so interested in using QSH>SQL to perform a query and navigate a database is due to the fact I would like to improve my SQL skillz (beginner) but I would like to be able to apply this knowledge to other environments in the future.
God, I write too much s***
My recent efforts: I am sorry. My formatting sucks
After I log in via ssh this is what I attempted:
$ db2 "select * from sysibm.sysdummy1"
db2: cannot execute
$ cd /usr/bin
$ ls
ajar
qsh
sqlj
db2
db2profc
dataq
java
javadoc
javah .....
$ db2 "select from qsys.lib
++++++CLI ERROR++++++
SQLSTATE 42704
LIB in QSYS type *file not found // there are tons of .file extensions within btw
$ cd ..
$ cd qsys.lib
$ ls
A crap load of .lib .menu .file (*file?) extensions
$ db2 "select * from 1234abcd.lib"
++++++CLI ERROR++++++
SQLSTATE 42601
NATIVE ERROR CODE: -104
Token not valid bla bla VALID TOKENS: FOR SKIP FETCH ORDER GET // success! well still fail
Questions
So my problem now is I just need to use valid tokens. Does anybody know of a good example? (im going to attempt on my own still)
For the life of me I cannot replicate the process above, to where I at least get a cli error / SQLSTATE. Everything now returns:
"db2: you should just give up lol"
I try to log everything I do, guess I missed something important ay? Somewhere between...
$ cd /usr/bin
$ ls
and
$ db2 "select * from iforget.lib"
Something happened to where I could at least execute SQL statements. Now, nothing.
db2 : t(ಠ益ಠt)
Its late. So tired
Any feedback = Appreciation
The qshell (and DB/2) equivalent to the mysql cli command is db2:
db2 "select * from sysibm.sysdummy1"
If you have the IBM Access .NET data provider installed you can query directly from PowerShell:
# Assembly name from \\HKCR\Installer\Assemblies\Global
$an = 'IBM.Data.DB2.iSeries,Version="12.0.0.0",PublicKeyToken="9CDB2EBFB1F93A26",Culture="neutral"'
# Connection string
$cs = 'Data-Source=10.0.0.50;UserID=QPGMR;Password=****'
Add-Type -AssemblyName $an
$cn = New-Object IBM.Data.DB2.iSeries.iDB2Connection($cs)
$cn.Open()
$cmd = $cn.CreateCommand()
$cmd.CommandText = "SELECT * FROM SYSIBM.SYSDUMMY1"
$da = New-Object IBM.Data.DB2.iSeries.iDB2DataAdapter($cmd)
$ds = New-Object System.Data.DataSet
$cnt = $da.Fill($ds)
Write-Host "$cnt records selected."
$cmd.Dispose()
$cn.Close()
foreach ($dt in $ds.Tables) {
$dv = New-Object System.Data.DataView($dt)
$dv | Format-Table -AutoSize
$dv.Dispose()
}
$ds.Dispose()
For more information see the IBM Redbook Integrating DB2 Universal Database for iSeries with Microsoft ADO .NET .
1) Is qshell/SSH an efficient way to retrieve information from a
database, or more specifically, the libraries that 'DB2 for iSeries'
uses (.lib/.file extension files?)?
The synopsis was helpful; it appears you are interested in doing ad hoc queries rather than writing an end-user application. qshell is just as efficient as the IBM i command STRSQL in terms of issuing SQL statements and seeing the results immediately in an emulator window. I just queried a 2 million row table on a column without an index and got the first page of results in .23 seconds.
2) Wouldn't writing SQL scripts and executing them via SSH/Qshell be
faster than using the iSeries emulator itself?
It is difficult to understand what faster might mean. It seems you are using the emulator solely to play back macros; this is not particularly efficient. Typically, one would use the emulator to get to a command line, issue the STRSQL command and run ad hoc SQL queries from there. That is very fast in terms of getting the results to display.
3) Is this only possible via port forwarding or tunneling?
This depends entirely on how the network was configured.
4) How do I find out more about the server via SSH commands? I can
navigate the servers directories pretty easily and view files that I
otherwise cannot see within iSeries. Its database files are
.lib/.file/.mbr file extensions that I cannot view using the 'cat'
command. This would require using SQL commands I am assuming?
Whoever hired you must have given you some basic information to get you started, so you probably already know the libraries (schemas) and files (tables) which make up the production environment. So I'm assuming that you aren't asking about SQL catalogs like SYSCOLUMNS and SYSTABLES but rather things which are more specific to IBM i. The reference material for IBM i is located in the Infocenter.
5) Do I need to log on as a root user in order to have read/write
access privelages and perform anything SQL related?
No system demands root privileges in order to perform read/write access. Your employer will need to provide you with a user profile having privileges to the tables you want to access.
If you need a light weight terminal emulator, look at the tn5250 project on Sourceforge. You won't need Windows admin privileges to install or run it.
I have an OpenEdge Progress v10.1C linux server with a bunch of data on it. We're migrating this data to an SQL server. I just need to get the data off of the server somehow, be it a bunch of CSVs of tables or an sql file or whatever - I just need the raw data.
I have no experience with this server and I can't seem to figure out how to even connect to it or anything. I just know where the data directory is and that I have a bunch of files in /usr/dlc/bin which are for management (like _progres, pro, ...)
I would really appreciate any help extracting this data!
At a command line execute $DLC/bin/showcfg -- this will reveal what you have available for licenses which will have an impact on what options you have to export the data.
If your license allows, to get simple text files you can use the "data dictionary dump". To run that start a session from a command line like this:
mpro dbName -p dict.p
If your licenses are good that will bring up the data dictionary. Navigate to Admin -> Dump Data and Definitions -> Table Contents if you get that far then go ahead and dump what you need.
If the database is large this may take a while. Maybe even a long while.
If you cannot navigate to that point then then you do not have an appropriate license. In that case if you have SQL installed and configured then the simplest thing may be to just extract the data with that. But since you are asking this question that seems unlikely -- none the less... type "ps -ef | grep sql" and see if anything shows up. If it does you should be able to connect an ODBC client.
If all else fails you can try to get someone to write you a custom extract program. That will need to be done by someone with a compiler license and they will need to provide you with r-code. That sort of thing is usually provided as a commercial service.
I am developing a app where i need to keep the replica of entire database from the web server to the IOS native sq-lite database, How can i keep updated version of db or import at loading of the app in one go, What should be my approach to go with this problem, Any help would be really appreciated,
Thanks
Deepesh
Looks like this is a popular problem. According to this answer,
Not every DB schema can be converted. MySQL is more complex and feature-rich than SQLite. However, if your schema is simple enough, you could dump it into an SQL file and try to import it / load it into an SQLite DB.
To do this, you can either use this mysql2sqlite.sh script like so
./mysql2sqlite.sh myDbase | sqlite3 database.sqlite
or use this answer and simply do
mysqldump database > database.sql
sqlite3 database < database.sql
I'm having trouble copying data from a production MySQL server to a development SQLite3 file (so that I have real data to play with on development machine). I've found tons of resources around the 'net on how to convert from MySQL to SQLite3, most of which were bash scripts with elaborate sed filters, but none worked (the most common problem was syntax issues upon import).
Anyway, then I stumbled upon YamlDB, and I thought "Why, of course! Let Rails do the conversion for me!" Well, this doesn't work either because all of the NULL fields (which are represented in the YAML file as !!null) end up being imported into the SQLite3 database exactly as "--- !!null" instead of actual NULLs. I seem to be the only person with this problem, as there were no mentions of it in the GitHub issues queue.
I even tried the workaround for using syck instead of psych (found in this SO question), but it made no difference.
So my question is this: does ANYONE know of a SIMPLE way to export data from one rails database for importing into another, regardless of database kind? And by "simple", I mean a few commands at the console, or whatever.
Look into taps # http://github.com/ricardochimal/taps
It will dump your MySQL db into a local sqlite db and is relatively simple to use.
From the comments: If you get an error stating schema parsing returned no columns, table probably doesn't exist then you need to specify an absolute path to the sqlite3 db instead of a relative one
currently i'm working with informix db server which is not in local, where can't connect from outside office or virtual lan, so there is any tools so i can copy all table and work locally,
thanks in advance
You can export database to text using dbexport and import it locally using dbimport. There are other way of migrating database, but for small databases it should work. The data is in text so it may be easy to change something (for example data format), or even use such export to import data to other database. Have a look at: http://publib.boulder.ibm.com/infocenter/idshelp/v10/index.jsp?topic=/com.ibm.mig.doc/mig138.htm