I am looking for good embedded database that i can use for application developed using Qt. The applications target desktop users from various sites of a single large company. The database should be able to store data separately at each site and the data shall be merged with other sites as and when it is required.
Besides SQLite, any of the following will work with Qt as an embedded database. Qt already has drivers for most of them, and you can find drivers for others. In terms of merging data "with other sites", it all depends on what you mean by this. Replication solutions for SQLite and MySQL Embedded may not be great (or exist at all). I would probably go with Firebird or Berkley.
Firebird
MySQL Embedded
Berkley DB - Qt plugin can be found at http://sourceforge.net/projects/qbdb/
If interested you can find more information on various replication support at the following links:
http://www.firebirdfaq.org/faq249/
https://docs.oracle.com/cd/E17276_01/html/programmer_reference/rep.html
http://hrivnac.web.cern.ch/hrivnac/Activities/Packages/Octopus/
Related
In my CS program, I was told I should learn SQL for my databases.
If I'm using PostgreSQL, do I also need a SQL server to go along with it? Is PostgreSQL a language, a server, or both? Is there even a SQL language or is it only servers?
Background: I downloaded Postgres because hey, that has SQL in the name, it works and I'm under the impression it's a pretty good choice anyway. But I couldn't figure out through their website if it needs a companion server, so I went looking for one and found AWS RDS.
The impression I have is that Postgres is the language and AWS RDS is the server, and they serve different functions. But I'm not sure about any of that.
Seems you're learning too many new topics at the same time.
Ok. I'll try to answer.
SQL stands for 'Structured Query Language', and serves as a 'standard' for many vendors that in much ways respects its fundamentals. Oracle, MySQL (now owned by Oracle), MariaDB and PostreSQL are some vendors.
Main thing with SQL code I would recommend you to identify every time you look at it, is to understand if it belongs to DML or DDL. DML stands for 'Data Manipulation Language' and refers to SQL instructions which 'modifies' data. DDL stands for 'Data Declaration Language' which defines or 'alter' de structure on which data will be stored.
Another important concept is atomicity of data manipulation. You can confirm a change or roll it back before it is persisted. This thing corresponds to 'commit' changes or do a 'rollback'. It's some kind of advanced concept, but generally happens "automatically" with standard client configurations. Later, you would have to know about it while programming some system module which interacts with databases.
When you think of the SQL 'server', it refers to the software configured/installed which has the responsability of manage persistence of data within some kind of 'instance' of persistence, allocated in some system with data storage capabilities. AWS implements this service in the cloud, and RDS is the product which supports many kind of SQL flavors to choose (Oracle, Postgresql, etc.)
If you are comfortable with Docker, I recomend you learn the basics which would help you setup and destroy databases many times, which is useful to develop and test locally. Next command, let you start a Postgresql database configured with open port 5432. You can see the server log through docker and use some SQL client to get connected. When you press Ctrl+C everything will be deleted. Of course there are other ways to keep data persistent, but this command would be an easy starting point.
$ docker run --rm -p 5432:5432 --name some-postgres-container-name -e POSTGRES_PASSWORD=mysecretpassword postgres:13.3
Side note: it's better to get used to work with specific docker image versions always (not 'latest').
More details of it usage here: https://hub.docker.com/_/postgres/
if I'm using PostgreSQL, do I also need a SQL server to go along with
it? Is PostgreSQL a language, a server, or both? Is there even a SQL
language or is it only servers lol? I'm genuinely trying to figure
this out myself, but basically everything I read is beyond my scope of
competence and confuses me more. I'm learning the syntax of SQL well
enough, but I'm so confused about everything on the most fundamental
level.
By the way "SQL Server" is Microsoft's SQL flavor, just another one. Don't be confused with the concept of having some SQL server configured.
Yes, you can think of PostgreSQL as a language too, which shares most of its syntax and semantics with other SQL vendors. Yes, there is a 'basic' SQL language shared and compatible between all vendors; some share more aspects than others. In terms of Venn diagrams, you can think of many circles representing each one, Microsoft's SQL Server, Oracle SQL, PostgreSQL, MySQL, etc. sharing the very most of its elements, where each element is a SQL instruction.
When dealing with Databases in general, keep in mind that they helps to modelate situations of 'real world' scenarios or software systems. SQL allows to 'talk' to implementation of "Relational Databases" wich is one kind of database modeling, but there are others too. ER Diagrams helps to represent the 'structure' of a database in a conceptual manner. I like DBeaver because it has an integrated ER diagram generator wich helps to understand the structure of a given database instance.
I have used Postgres and it is an excellent product (and free).
I would install it standalone first. It does come with its own client tools, which you use to communicate with the database server, which runs independently as a service. However, you might be better off installing something like SqlWorkbench as a client tool (which I use). In the config you specify the machine Postgres is running on (which can be your local computer for testing purposes) and the port to connect on. Essentially, the client sends your instructions to Postgres server and the server returns the resultsets associated with your instructions. The client also formats the resultsets into a nice readable "spreadsheet" format with rows and columns.
First I'll try to answer the questions you asked. There is a SQL language, but in practice it is not strictly standardized. There are many offerings for databases and database servers. Many of these are discussed below.
Any database you pick will give you the chance to learn basics of SQL queries and this knowledge will serve you well even if you switch to a different database later.
Specifically, when it comes to PostgreSQL, it is a Relational Database Management System. It is a software that operates as a server. You can install it on your personal computer running Windows, Linux, or MacOS. You can also install in on a dedicated server computer where you'll get better performance and uptime. Further, there are many companies that offer PostgreSQL hosting including Amazon RDS and Google Cloud but they're not free.
For a CS student, PostreSQL installed on your personal computer might be a reasonable choice. But you have lots of options. Read on....
For a CS program, your choice of database will depend on:
what degree of portability you need
how much data you have
how many users will connect to database
what kinds of jobs you might pursue after graduation
Portability
If you think you want to ship your database with your application, then your best bet is probably SQLite. By some accounts it can handle several million rows worth of data and still be performant. However, it's not great if you need for multiple users to connect to the same database. Your data can get corrupted in many multi-user scenarios.
How Much Data and Users
For large data and large users, you'll want to consider the client/server heavy hitters:
PostgreSQL
MySQL/MariaDB
Oracle
SQL Server
These databases will support large quantities of data any many simultaneous connections. But if you want to distribute the database with your application, it's not a good idea. Or if you want to demonstrate your app, you need to ensure that a connection to a server will be available. All of these databases come with a free version, but the last two will have the most restrictions.
After Graduation
Now you're looking to the future and possibly what kind of skills you want to put on your resume. If you think you'll end up in a corporate environment that is already well established, they will likely already have a preferred database and it could be any of the ones listed here (SQLite or the "heavy hitters"). If you want to position yourself as developing apps with low overhead cost, you'll gravitate towards SQLite/PostgreSQL/MySQL. If you think you're going to be some kind of database administrator working in a buttoned-up corporate environment, those companies tend to favor SQL Server and Oracle.
Good luck. Any choice you make will probably be fine. Knowing some flavor of SQL is useful for your future endeavors.
SQL is a language like any other language but working on database. It is called SQL because it works on structured data like table (i.e rows and columns). After reading the documentation of PostgreSQL, I think we do not need any separate server installation. You can download it from here. If you are facing any issues with it I suggest using MySQL workbench. Although installation may take longer time, but its easy to understand.
Team Foundation Server version-control of Web Applications
I feel let down :( ...
Checking-in and out .mdf files, branching and merge all work well in TFVC, however there is no data conflict resolution (conflict resolution for code is great!), only a choice is offered between whole source or target files.
I am not suggesting that data-tables are displayed side-by-side, because scripts in DAC/ 'database projects' can be compared for changes in table structure and data differences compared by stored procedures.
Some method must be commonly used, but apparently that MS expect the code and data versioning to be managed iteratively/ separately? (I would like to avoid trialling additional proprietary software like Red-Gate Source-Control.)
FULL version control is required as new feature branches will change the DB, but core/testing data needs to be retained.
So PLEASE!! Help me with pointers to straight-forward FULL version-control practices for web applications that include sql versioning with intelligent merge and roll-back capabilities.
Many thanks!
You could try Red Gate Deployment Manager to manage your deployments, which also comes with a free community edition. However, although not strictly mandatory we would recommend that you do this in conjunction with SQL Source Control, which would allow you to specify static data tables to put in version control. Although this is third party software, the database objects are saved as plain text .sql files, and not a proprietary format.
I've yet to see an application like MS Access that will let you link to tables on remote servers, of a variety of technologies, and then perform queries that include joins across these disparate sources. I find it somewhat amazing that Access can link up to a table on Sql Server, another on Oracle, or even a local file-based DB, and then allow queries to be performed joining them.
So my programming related question is: If one were to develop software that could do this, how should one go about doing this? Does Access use some inbuilt feature of existing DB technology to do this? Ie, does OleDB or ODBC or some technology have this built in, or is Access doing a lot of proprietary heavy lifting on its own?
If Access is doing all this work itself, it seems this would be a rather difficult thing to reproduce in a new project. I'm just curious what level of complexity we're talking about here, and whether some existing layer of DB tech already has this inbuilt.
The task of joining tables from different data sources is harder than it looks even if you just restrict yourself to a single database API (like ODBC). ODBC has nothing built in to accomplice this. Look at ODBC's SQLGetInfo, SQLGetConnectAttr and SQLGetStatementAttr APIs and you'll see the possible differences between ODBC drivers.
There is an SQLEngine for Windows which will join different ODBC data sources which can all be to different databases at ODBC-ODBC Join Engine
UPDATE: Disclosure - I work for Easysoft.
I currently have a filesystem path I would like to index into a SQL database. I need to access the data so that I can do queries against files based on modified times, or partial names, or many other items.
Is there a way to somehow sync a filesystem to a database automatically, or even access a filesystem in a sql-like interface, without having to crawl through folders recursively?
I was checking the Microsoft Sync Framework 2.0, since it supports SQL databases now, however it doesn't appear to support syncing files to databases.
I'm sure other vendors do something similar, such as Microsoft for the database of files in Media Center, or programs like TVersity storing a database of the files as well.
You don't mention a programming language but here is how I would do it and I think it's the way most media apps maintaining a library do it (although they might be writter in different languages and use the win32 api).
Using .net to get your initial data I would once recursively scan the directory and then add all information to the database. Then I would run a service using a FileSystemWatcher object to be notified about changes and process the events accordingly.
It sounds like what you want is Windows Search. (or the Windows 7 Library feature if you're feeling bleeding edge).
Ultimately though something has to crawl the disk to pull the info out, whether it's you or a third party service.
As an aside SQL may not be the best tool for that particular job, depending on exactly what you want to search for. Tree structures are notoriously tricky to represent in relational databases efficiently - your searches could get quite expensive!
When developing whether its Web or Desktop at which point should a developer switch from SQLite, MySQL, MS SQL, etc
It depends on what you are doing. You might switch if:
You need more scalability or better performance - say from SQLite to SQL Server or Oracle.
You need access to more specific datatypes.
You need to support a customer that only runs a particular database.
You need better DBA tools.
Your application is using a different platform where your database no longer runs, or it's libraries do not run.
You have the ability/time/budget to actually make the change. Depending on the situation, the migration could be a bigger project than everything in the project up to that point. Migrations like these are great places to introduce inconsistencies, or to lose data, so a lot of care is required.
There are many more reasons for switching and it all depends on your requirements and the attributes of the databases.
You should switch databases at milestone 2.3433, 3ps prior to the left branch of dendrite 8,151,215.
You should switch databases when you have a reason to do so, would be my advice. If your existing database is performing to your expectations, supports the load that is being placed on it by your production systems, has the features you require in your applications and you aren't bored with it, why change? However, if you find your application isn't scaling, or you are designing an application that has high load or scalability requirements and your research tells you your current database platform is weak in that area, or, as was already mentioned, you need some spatial analysis or feature that a particular database has, well there you go.
Another consideration might be taking up the use of a database agnostic ORM tool that can allow you to experiment freely with different database platforms with a simple configuration setting. That was the trigger for us to consider trying out something new in the DB department. If our application can handle any DB the ORM can handle, why pay licensing fees on a commercial database when an open source DB works just as well for the levels of performance we require?
The bottom line, though, is that with databases or any other technology, I think there are no "business rules" that will tell you when it is time to switch - your scenario will tell you it is time to switch because something in your solution won't be quite right, and if you aren't at that point, no need to change.
BrianLy hit the nail on the head, but I'd also add that you may end up using different databases at different levels of development. It's not uncommon for developers to use SQLite on their workstation when they're coding against their personal development server, and then have the staging and/or production sites using a different database tool.
Of course, if you're using extensions or capabilities specific to a certain database tool (say, PostGIS in PostGreSQL), then obviously that wouldn't work.