Does anyone know if it's good solution to use SQLite in multi-thread environment.
I want to replace SQL Server with more simple and built-in database as there is no need to feed such big server DB. The supposed max size of DB would be 4 gigabyte after 4-5 years of usage. Is it normal for built-in DB? Could it affect performance?
It depends on the type of queries you would use. If the queries are simple selects with plain joins, then SQLite could do fine but I think you would still be better off with e. g. Firebird 2.5 when the stable release gets out (RC3 is available now). You would have somewhat richer SQL to work with. I don't know how much bulk loads are important for you, but neither SQLite nor Firebird are very strong in this area. If you need good bulk insert performance and low cost, then you should look at PostgreSQL or MySQL. There is also a very interesting looking database I happened to stumble upon recently called CUBRID. I have only installed it so far, so I can't tell how good or bad it is but it certainly seems worth a look.
You might also want to look at this wikipedia article:
http://en.wikipedia.org/wiki/Comparison_of_relational_database_management_systems
I don't know which distro you're talking about here. I've only used SQLite.NET and I know it works well on multithreaded applications.
It can also be deployed on client-server systems so you need not worry at all.
Considering Vinko's statement of 'real' databases, you can ignore him. SQLite is really worth it's salt.
If you're working with .NET, you might find this link useful:
http://sqlite.phxsoftware.com
According to the documentation SQLite is thread safe but there are caveats.
You can use SQLite in a multithreaded environment, but if and only if you build a special version of it (and find out if the library you'll be using it supports it and tweak it if it doesn't.) So, assuming your library supports multithreaded SQLite, if you really need a high level of concurrency to the database you may prefer to use a 'real' database. Be it MSSQL or any other falls out of the scope of the question.
Consider MySQL and SQL Server Express, for example.
If your concurrency level is low, SQLite can cope with it.
I would also suggest you to take a look at the CUBRID database. It has nice optimizations for Web applications and it is easy to learn.
Related
A client wants a database converting from MS SQL to MySql.
Personally I wouldn't do this, I prefer MS SQL for larger databases and like the tools available when working with MSSQL.
However I do use MySql on smaller projects. I've never moved a database from one to the other, or made much use of sprocs when using mysql.
Is there a lot of work involved in rewriting the sprocs, considering I have no prior knowledge of the ms sql database itself, so will need to first gain an understanding of what each sproc does.
This isn't a discussion on which to use as that's already been decided by the client for a few reasons.
have you seen this?
I haven't done this before, but I can only assure you mysql is good for big projects, too.
Can someone recommend a great IDE/Database Developer Tool for Postgres 8.4. We are currently using EMS SQL Manager but it has so many bugs and shortcomings that its getting on my nerves. I reported the bugs to EMS but they have not been fixed yet.
I wonder what do people use for PostgresSQL development? Windows/Linux, commercial/free, standalone/eclipse plug-in, it does not matter as much. I just want a reliable tool that works and does not blow up in my face.
On PostgreSQL DB we do and would like to have IDE support for.
Tables, views, indexes, triggers
SQL, plpgsql, and python procedures that return single values, set of values or tables
SQL query and performance monitoring
DB role management
Good transaction support straight from GUI as well as support for BEGIN, COMMIT, etc
Any recommendations are welcome.
Edit: After many months we are still using EMS SQL Manager. I got used to certain shortcomings, some bug were fixed. I tried every possible PostgreSQL management program I could get my hands on and still in my opinion EMS SQL Manager is still the best.
pgAdmin is a popular one. It's free, and it runs on Windows and Linux. It's been a while since I have used it but I found it to be quite stable. Wasn't fond of the procedure code editor - it worked, I just didn't like it (but now I can't remember why, it was probably a minor thing).
Lightning Admin, commercial, postgreSQL and mysql, $50. The developer is very pro postgreSQL.
http://lightningadmin.com/
Is there a cross platform solution for sql? My prototype was in sqlite. I am switching to a server that offers tsql and i was considering mysql in the past for my webservers(maybe i should stick to tsql and sqlite). I am wondering if theres a .NET lib that allows me to write sql compatible with all.
Some annoyance i had was in create table. I thought primary keys auto increased but they dont. I have to write identity(1,1). When i ported my sqlite code to mysql i had issues also with create table but i am sure there will be other places once my sql statements get more complex. So i thought trying a lib may be a good idea.
I personally use SubSonic as an ORM. It supports both SQL-Server and MySQL, as well as SQLite. Database creation is always a tricky question, but you generally don't perform it often. I have chosen SQLite, as it is very portable, and should also be available when I port code using Mono. TSQL (SQL-Server) is a windows-only product, and is not portable. My databases also won't be very large (< 100 Mb) and this may also need to figure in your choice.
I would recommend (for the moment, until VS 2010 is out) that you don't use Entity Framework, as it is a traditional v1.0 MS product, and very problematic. Essential features are not present, and artificial restrictions make it less useful than other ORMs. No doubt the next version will be better, but the current version is not worth your time compared to something like SubSonic.
An ORM like NHibernate or Entity Framework will support all those sql backends, and more. It will also save you from writing all your SQL queries yourself.
You might want to take a look at ADO.NET Entity Framework. It supports a lot of different backends.
It would seem these days that everyone just goes with MySQL because that's just what everyone goes with. I'm working on a web application that will be handling a large quantity of incoming data and am wondering if I should "just go with MySQL" or if I should take a look at other open-source databases or even commercial databases?
EDIT: Should have mentioned, am looking for optimal performance, integration with ruby + rails running on debian 5 and money is tight although if it will save money in the long run I would consider making an investment into something more expensive.
I've posted this before, but I have no reason to change this advice:
MySQL is easier to start using.
Nicer UI tools. Faster, if you don't use ACID. More tolerant of invalid data. Autoincrement columns are as easy as typing autoincrement. Permissions aren't as tied to the file systems and OS users. Setting a delimiter is easier than using PG's "dollar sign quoting" when writing a stored proc. In MySQL, you connect to all databases, not just one at a time.
Postgres (PG) is much more standards compliant, but it's uglier and more complicated, especially from a UI perspective. It used to require manual vacuuming, and actually enforces referential integrity (which is a great thing that can be a pain in the ass). Autoincrement is much more flexible, but requires sequences (which can me masked by using serial), and wait, what's an OID?
So if you don't really know or care much about databases, data validity, ACID compliance, etc, but you do care about ease and speed, you tend to go with MySQL.
Too many (not all, but many) "web programmers" know a lot about "web 2.0" or PHP or Java, but don't know much about database theory or practice ("an index? what's that?"). They tend to see a database as just a fancy hashtable or bag of data, and indeed one that's not anywhere as dynamically changeable or forgiving as a hashtable.
For these folks, MySQL -- because until 5.0 it wasn't really an RDBMS, and in many ways still is not -- is a godsend. It's "faster" than the competition, and doesn't "waste time" on "esoteric" database stuff a web programmer doesn't want, understand, or see the value of.
For somebody with a database background, on the other hand, MySQL is a minefield: stuff that should work (complicated views, group bys, order bys in group bys) may work or may if you're lucky crash the server, or if you're unlucky just give results with incorrect data.
I've spent days working around some of these things in admittedly complicated by not extraordinarily complex views and group bys.
And MySQL isn't really faster. If you're using InnoDb tables for ACID (or just because at more than 30 Million rows, MyISAM tables tend to get crappy), yes a straight one-table select is probably faster than in PG. But add in joins, and PG is suddenly significantly faster. (MySQL is especially bad at outer joins.)
In summary: if to you the database is a bag, if you never intend to do data mining or reporting, if you're mostly interested in serving up big hunks of text with few relations or updates -- that is, if you're using a database to power a blog, MySQL is a great choice.
But if you're actually managing data, if you understand that data lives longer and is more valuable to a business than front-end programs and middle-tier business rules , if you need the features of a real database, use PG.
A "web programmer" who has decided all his table structures can be auto-generated by Hibernate (or some other ORM) looks at that and says, "too complicated" and "I bet complicated means more cost and slower speed" and so he goes with MySQL.
As I said, PG is far superior, and I hate mucking with MySQL's bizarre bugs, and I think that overall PG performance is probably better than MySQL for any even slightly complicated query.
But MySQL makes things look (deceptively) simple, so you get a lot of people who don't really understand database design figuring that MySQL is a great choice.
Use PG. It's consistent, it's reliable, it's standards-compliant, it's faster on (even moderately) complicated queries, it doesn't completely throw off your schedule with weird bugs.
I think PostgreSQL is a very viable alternative to MySQL. It's much more Oracle-like.
Personally, I try to avoid MySQL whenever I can for the following reasons:
The default storage engine MyISAM lacks Foreign Key support. Innodb does, however it does not support Foreign Keys to MyISAM tables for obvious reasons.
If I attempt to insert invalid data, MySQL will happily change it for me.
Illegal DateTime, Date, or Timestamp values are convert to "zero": http://dev.mysql.com/doc/refman/5.1/en/datetime.html
Varchar and Char type columns: http://dev.mysql.com/doc/refman/5.1/en/char.html
Numeric Datatypes depending on SQL Strict Mode: http://dev.mysql.com/doc/refman/5.1/en/numeric-types.html
These things may not be of any importance to some people, but when it comes to quality of data, I would rather use something else.
Well, there are may differences between the RDBMS of the world.
Take a look at http://en.wikipedia.org/wiki/Comparison_of_relational_database_management_systems#Fundamental_features
Using this as a guide you should be able to narrow down your choices a lot.
A few things to keep in mind:
SQL Server 2005 Express is limited to a 4GB file size, but has excellent support int the .NET and Java languages.
MySQL will run on both Windows and Linux, and many languages support it (including .NET and Java) with external libraries.
SQLite is supported by effectively every operating system, and can be distributed as an integrated part of your application.
Firebird the open sourced and branched version of Borland's Interbase is pretty good, it works happily on most (all?) flavours of Linux and is very performant. I am not a RoR guy so I don't know the details, but I do have a friend in NZ who uses Firebird with all his RoR projects, it definitely works with RoR, and works well.
Edit: Found a link to a Firebird Rails Adapter here
Please be a little wary of the idealistic points-of-view put forth by people who may have axes to grind. MySQL is a capable, robust and scalable solution for many problems, but this is usually not so right out-of-the-box as its defaults are extremely conservative. Like any database product, you will need to spend time tuning its installation for the desired performance. You will also need to spend some time accomodating its limitations, which is also true for any database you choose.
MySQL's popularity is partly self-generated: a lot of hosting providers provide it, so a lot of people use it.
Mysql is great, and mssql is great. I haven't used anything else. I would say if you are completely on the fence, go with the technology stack you are strongest with. I have a good amount of c#, asp.net, and other Microsoft stack experience, so it is pretty natural for me to specialize in mssql. If you are more familiar with *nix, php, etc, you may be more at home sticking to an open source stack. You can certainly mix and match the two stacks, but sticking to one world or the other can avoid some pain for you.
Like duffymo I also recommend PostgreSQL. It is very nice from a developer perspective:
work on many platforms (both unix-based and Windows), has vary stable interfaces to any laguage/environment I worked (Windows, Linux, Delphi, Java, Perl, Python). Stored procedure language: PLPGSQL is also easy and powerful. User support (newsgroups, lists, SO) is nice and helpful.
"It would seem these days that everyone just goes with MySQL because that's just what everyone goes with." If MySQL is the only thing that people use then why are Oracle and MSSQL still around?
The debate as to which database engine to go for can be talked about until the cows come home. I personally have always found one constant in choosing a databse engine. The one you can afford is generally the one you go for.
If you can justify spending XXX on a database then you probably know the reasons already for choosing it.
MySQL is easy to install and it works fine without special settings. With the proper approach MySQL can flexibly adjust to your needs. But there are also some pitfalls: in some cases it may slow down your project, no matter how well you have tuned the DBMS and the data structure.
MySQL is for you if:
you do not want to delve into DBMS settings;
you think structurally;
integration with MySQL is in any programming language, framework, CMS, CMF and so on;
you need DBMS to manage small structural data (up to 1 or 2 gigabytes).
You mentioned, that you deal with large quantity of incoming data. Probably this guide would be helpful
It seems to me, from both personal experience and SO questions and answers, that SQL implementations vary substantially. One of the first issues for SQL questions is: What dbms are you using?
In most cases with SQL there are several ways to structure a given query, even using the same dialect. But I find it interesting that the relative portability of various approaches is frequently not discussed, nor valued very highly when it is.
But even disregarding the likelihood that any given application may or not be subject to conversion, I'd think that we would prefer that our skills, habits, and patterns be as portable as possible.
In your work with SQL, how strongly do you prefer standard SQL syntax? How actively do you eschew propriety variations? Please answer without reference to proprietary preferences for the purpose of perceived better performance, which most would concede is usually a sufficiently legitimate defense.
I vote against standard/vendor independent sql
Only seldom the database is actually switched.
There is no single database that fully conforms to the current sql standard. So even when you are standard conform, you are not vendor independent.
vendor differences go beyond sql syntax. Locking behaviour is different. Isolation levels are different.
database testing is pretty tough and under developed. No need to make it even harder by throwing multiple vendors in the game, if you don't absolutly need it.
there is a lot of power in the vendor specific tweaks. (think 'limit', or 'analytic functions', or 'hints' )
So the quintessence:
- If there is no requirement for vendor independence, get specialised for the vendor you are actually using.
- If there is a requirement for vendor independence make sure that who ever pays the bill, that this will cost money. Make sure you have every single rdbms available for testing. And use it too
- Put every piece of sql in a special layer, which is pluggable, so you can use the power of the database AND work with different vendors
- Only where the difference is a pure question of syntax go with the standard, e.g. using the oracle notation for (outer) joins vs the ANSI standard syntax.
We take it very seriously at our shop. We do not allow non-standard SQL or extensions unless they're supported on ALL of the major platforms. Even then, they're flagged within the code as non-standard and justifications are necessary.
It is not up to the application developer to make their queries run fast, we have a clear separation of duties. The query is to be optimized only by the DBMS itself or the DBAs tuning of the DBMS.
Real databases, like DB2/z :-), process standard SQL plenty fast.
The reason we enforce this is to give the customer choice. They don't like the idea of being locked into a specific vendor any more than we do.
In my experience, query portability turns out to be not so important. We work with various data sources (mainly MSSQL and MySQL), but we know which data is stored where and can optimize accordingly. Since we control the systems, we decide when - if ever - structures are moved and queries need to be rewritten.
I also like to use certain other server-specific functionality, such as query notification in SQL Server, which MySQL doesn't offer. So there, again, we use it when we can and don't worry about portability.
Furthermore, parts of our apps need to query schema information and act on it. Here, again, we have server-specific code for the different systems, instead of trying to restrict ourselves to the lowest common denominator.
There is no clear answer whether SQL portability is desirable or not - it really depends a lot on the situation, such as the type of application.
If the application is going to be a service - ie there will only ever be you hosting it, then obviously nobody but you will care whether your SQL is portable enough, so you could safely ignore it as long as you have no specific plans to drop support for your current platform.
If the application is going to be installed at a number of sites, which each have their own established database systems, obviously SQL portability is very important to people. It lets you widen your potential market, and may give a bit of piece of mind to clients who are on the fence in regards to their database system. Whether you want to support that, or you are happy selling only to, for instance, Oracle customers, or only to MySQL/PostgreSQL customers, for example, is up to you and what you think your market is.
If you are coding in PHP, then the vast majority of your potential customers are probably going to expect MySQL. If so, then it's not a big deal to assume MySQL. Or similarly if you are in C#/.NET then you could assume Microsoft SQL Server. Again, however, there is a flip side because there may exist a small but less competitive market of PHP or .NET users who want to connect to other database systems than the usual.
So I would largely regard this as a market research question, unless as in my first example you are providing a hosted service where it doesn't matter to users, in which case it is for your own convenience only.