I'm looking for an IDE or "visual editor" for some basic table manipulation.
I have a few tables, ~100K entries each. Most of them share two columns that together compose a UNIQUE PRIMARY KEY. These tables are static (they are just old record data), so no "online" or code interface is needed.
To be honest I only wish Excel or something like that could handle so many rows, since I want to perform simple tasks (e.g. erase a column, sort by column). What tool in your experienced is the most "Excel-like" for static tables?
Try Microsoft Access.
You can import or link to external data sources and access has lots of tools available to work with the data.
Using this method you will be able to do the following
View the data so that you can filter & sort the data.
write custom queries against the data (using a visual designer or SQL).
Add, edit and delete data (providing you have edit, delete privileges on the data source)
Write reports using the linked data.
Also, tables in Access 2007 and upwards look very much like Excel spreadsheets and as this is in the Microsoft office suite there are plenty of tools to export the data between Access and Excel.
phpmyadmin imo can serve as a good visual editor for what you need (though its 'online' and needs a running webserver)
DBDesignerFork is open source, free, and can reverse engineer your database to build the model.
You can then switch in to Query Mode and it will help you build queries from the table diagram.
You mention you want to be able to "erase a column, sort by column" these are two very different things. The sort is easily handled using an OREDER BY in your SELECT statements. Dropping columns can also be done in SQL using the ALTER TABLE command but remeber there is no easy "undo" unless you start wrapping your changes in transactions.
To sum up, you should forget the Excel comparison and learn SQL :)
Related
I'm new to CDS/Dataverse, coming from the SQL Server world. I created a new Dataverse table and there are over a dozen columns in my "new" table (e.g. "status", "version number"). Apparently these are added automatically. Why is this?
Also, there doesn't seem to be a way to view a grid of data (like I can with SQL Server) for quick review/modification of the data. Is there a way to view data visually like this?
Any tips for a new user, coming from SQL Server, would be appreciated. Thanks.
Edit: clarified the main question with examples (column names). (thanks David)
I am also new to CDS/Dataverse, so the following is a limited understanding from what I have explored so far.
The idea behind Dataverse is that it gives you a pre-built schema that follows best-practice for you build off of, so that you spend less time worrying about building a comprehensive data schema, creating tables, and how to relate them all together, and more time building applications in Power Apps.
For example, amongst the several dozen tables it generates from the get-go is Account and Contact. The former is for organisational entities and the latter is for single-person entities. You can go straight into adding your user records in one of these tables and take advantage of bits of Power Apps functionality already hooked up to these tables. You do not have to spend time thinking up column names, creating the table, making sure it hooks up to all the other Dataverse tables, testing whether the Power Apps functionality works with it correctly etc.
It is much the same story with the automatically generated columns for new tables: they are all there to maintain a best-practice schema and functionality for Power Apps. For example, the extra columns give you good auditing with the data you add, including when a row was created, modified, who created the row etc. The important thing is to start from what you want to build, and not get too caught up in the extra tables/columns. After a bit of research, you'll probably find you can utilise some more tables/columns in your design.
Viewing and adding data is very tedious -- it seems to take 5 clicks and several seconds to load the bit of data you want, which is eons in comparison to doing it in SQL Server. I believe it is how it is due to Microsoft's attempt to make it "user friendly".
Anyhow, the standard way to view data, starting from the main Power Apps view is:
From the right-hand side pane, click Data
Click Tables
From the list of tables, click your table
Along the top row, click Data
There is an alternative method that allows you to view the Dataverse tables in SSMS – see link below:
https://www.strategy365.co.uk/using-sql-to-query-the-common-data-service/
To import data in bulk:
Click on Data from the top drop-down menu > Get data.
Importing data from Excel is free. To import from other sources, including SQL Server, I believe is a paid service (although I think you may be able to do this on the free Community Plan).
I've been handed an interesting question in that an Apple centric user would be keen to run databases on Filemaker Pro and we already have several running on MS SQL.
FM Pro is visually stunning and as a front end to work with customers would look good, but I'm more SQL at heart.
Does anybody use both?
Can you easily run tasks between SQL and FM Pro to update data to FM Pro (say overnight)?
Has anybody made the change from SQL to FM Pro for any purpose and found it to be ok?
Thanks in advance
To expand on user4166144's answer a bit, you can add MS SQL as an external data source to FileMaker using ODBC. (See "Using FileMaker Pro, I want to create a live connection to a MS SQL Server, Oracle or MySQL data source.")
This will let you base layouts on an MS SQL table just as though it was a native FileMaker table. That is, the data will be "live", with no need for over-night copying about.
There are some limitations to ODBC connections, which will probably be irrelevant in your case. Mostly, ODBC data sources in FileMaker don't get all the FileMaker goodies in Manage Database. Tables from ODBC sources are "shadow tables". For example, if you delete a field ("column") in FileMaker, it doesn't get deleted in the SQL database. However, creating, editing, and deleting records all work as normal. You can even add tables from ODBC sources to the relationship graph, which is the primary way that you get data from multiple tables in FileMaker.
FileMaker is a little hard to wrap your head around coming from an SQL background. It's meant for rapid application development, and as such it has certain paradigms in mind. Here are a few things to know that I hope will help:
Every user interface ("Layout") in FileMaker is based on a table occurrence. The body of a layout represents a single record in that table occurrence. Every script, calculation and related piece of data is calculated from the perspective of that single record in that single table occurrence. That is, a layout is a "cursor".
There is no (sane) FileMaker way to do the equivalent of an SQL "OR" when it comes to the Relationship Graph.
FileMaker 12 has two features with very similar names. It has a calculation function "ExecuteSQL", which allows you to run SELECT statements on table occurrences in FileMaker; that includes ODBC sources. It also has a script step called "Execute SQL", which is handy for running arbitrary SQL against an ODBC data source. This latter is probably going to be very useful for you.
It's somewhat hard to get the results of SQL queries onto FileMaker layouts in any kind of elegant way. Generally, you need to write the results to a global field, a global variable, or a regular field. If you want to display tabular data from an SQL query in a decent kind of way, you will need to generate HTML and spit it into data url in a Web Viewer element on a layout (i.e., prefix the HTML with "data:text/html,")
FileMaker, since version 9, includes the ability to connect to a number of SQL databases without resorting to using SQL, including MySQL, SQL Server, and Oracle. This requires installation of the SQL database ODBC driver to connect to a SQL database. SQL databases can be used as data sources in FileMaker’s relationship graph, thus allowing the developer to create new layouts based on the SQL database; create, edit, and delete SQL records via FileMaker layouts and functions; and reference SQL fields in FileMaker calculations and script steps. It is a cross platform relational database application.
Versions from FileMaker Pro 5.5 onwards also have an ODBC interface.
FileMaker 12 introduced a new function, ExecuteSQL, which allows the user to perform an SQL query against the FileMaker database to retrieve data, but not for modification or deletion, or schema changes.
Firstly, let me apologize for the title, as it probably isn't as clear as I think it is.
What I'm looking for is a way to keep sample data in a database (SQL, 2005 2008 and Express) that get modified every so often. At present I have a handful of scripts to populate the database with a specific set of data, but every time the database is changed all the scripts have to be more or less rewritten and I was looking for some alternatives.
I've seen a number of tools and other software for creating sample data in a database, some free and some not. Are there any other methods I haven’t considered?
Thanks in advance for any input.
Edit: Also, if anyone has any advice at all in dealing with keeping data in sync with a changing application or database, that would be of some help as well.
If you are looking for tools for SQL server, go visit Red Gate Software, they have the best tools. They have a data compare tool that you can use to keep lookup type tables up-to-date and a SQL compare tool that you can use to keep the tables synched up between two datbases. So using SQL data compare, create a datbase with all the sample data you want. Then periodically refresh your testing db (or your prod db if these are strictly lookup type tables) using the compare tool.
I also like the alternative of having a script (you can use Red Gate's tool to create scripts) because that means you can store this info in your source control and use it as part of a deployment package to other servers.
You could save them in another database or the same db in different tables distinguished by the name, like employee_test
Joseph,
Do you need to keep just the data in sync, or the schema as well?
One solution to the data question would be SQL Server snapshots. You create a snapshot of your initial configuration, so any changes to the "real" database don't show up in the snapshot. Then, when you need to reset the table, select from the snapshot into a new table. I'm not sure how it will work if the schema changes, but it might be worth a try.
For generation of sample data, the Database project in Visual Studio has functionality that will create fake/random data.
Let me know if this make sense.
Erick
I have 3 computers having the same sql server 2005 database, I would like to gather the data from the 3 computers to another computer which has the same database. Please help me.
This is called "data conversion" and a lot of your work will be to determine uniqueness on each one of them and coming up with strategies to prevent collisions, mainly primary keys that likely are the same across these databases. No simple answer here, it can be a project in itself.
It might be difficult without any manual data transformation. It depends on your database and type of the data. For example what do you use as a keys? If you have sequential integers as a primary/foreign keys, then you will have to do some manual data transformation. IF you use GUIDS, it will get slightly easier, but you still have to ensure that for example some lookup tables doesn't have different guid keys for same items etc.. But there is no took for doing this automatically.
Maybe if you have some very simple data without any relations to other tables (like table with one column with text messages etc) you can script the data with SQL Server Database Publishing Wizard, and then execute the scripts against your target database.
You need to backup your databases by right clicking in Enterprise Manager and choosing backup before choosing the location etc.
After backing up you can then restore to your local Sql Server by right clicking and choosing restore.
After you have the data locally you will need to write queries to transfer the data to your local database.
Alternatively you can use something like Red Gates Sql Data Compare to compare and transfer data using a visual interface. Although this costs money.
Redgate SQL Toolbelt may be able to help you. You would first copy database to that another computer and then compare it with Sql Data Compare against 3 databases always copying data only one way (to your new database). However I am not 100% sure if it will work like i think it would. You would have to verify it yourself.
Like other people suggested some things like primary keys etc may be problematic.
Is there an automatic way in SQL Server 2005 to create a database from several tables in another database? I need to work on a project and I only need a few tables to run it locally, and I don't want to make a backup of a 50 gig DB.
UPDATE
I tried the Tasks -> Export Data in Management studio, and while it created a new sub database with the tables I wanted, it did not copy over any table metadata, ie...no PK/FK constraints and no Identity data (Even with Preserve Identity checked).
I obviously need these for it to work, so I'm open to other suggestions. I'll try that database publishing tool.
I don't have Integration Services available, and the two SQL Servers cannot directly connect to each other, so those are out.
Update of the Update
The Database Publishing Tool worked, the SQL it generated was slightly buggy, so a little hand editing was needed (Tried to reference nonexistent triggers), but once I did that I was good to go.
You can use the Database Publishing Wizard for this. It will let you select a set of tables with or without the data and export it into a .sql script file that you can then run against your other db to recreate the tables and/or the data.
Create your new database first. Then right-click on it and go to the Tasks sub-menu in the context menu. You should have some kind of import/export functionality in there. I can't remember exactly since I'm not at work right now! :)
From there, you will get to choose your origin and destination data sources and which tables you want to transfer. When you select your tables, click on the advanced (or options) button and select the check box called "preserve primary keys". Otherwise, new primary key values will be created for you.
I know this method can hardly be called automatic but why don't you use a few simple SELECT INTO statements?
Because I'd have to reconstruct the schema, constraints and indexes first. Thats the part I want to automate...Getting the data is the easy part.
Thanks for your suggestions everyone, looks like this is easy.
Integration Services can help accomplish this task. This tool provids advanced data transformation capabilities so you will be able to get exact subset of data that you need from large database.
Assuming that such data is needed for testing/debugging you may consider applying Row Sampling to reduce amount of data exported.
Create new database
Right click on it,
Tasks -> Import Data
Follow instructions