I'm looking for some help designing a better summary report. Right now we publish and send everything (execution% by modules, defects etc) in an excel and I was hoping if we could use that excel data to generate a live dashboard that would be accessible by a URL.
To add, the execution data comes from QTest and defects from JIRA. At this point we are even ok with filling data in excel manually and using that as a source for any reporting tool.
If a free tool is available, even more better.
Any leads, helps, feedback is appreciated.
Thanks,
MD
Sounds like you need Microsoft's Power BI. We've done a lot of reporting from JIRA using this free tool (Desktop). If you need to share it with others "real time", you'll prefer the online experience for about $10/user/month. But if you're looking to stay "free", you can simply share the Power BI file with your stakeholders.
I recommend AGAINST using the already built in JIRA APP. It seems to want to pull back all your issues. Instead, use a REST API Call like this:
https://domain/rest/api/2/search?jql=filter=22605&fields=id,key,summary,description
If you get more issues back than your Issue Search is configured for, the pagination can be a little tricky. Also multiple values in a custom field need special handling.
Or if you're on premise and know your JIRA DB, direct SQL is an efficient way to go.
We use both mechanisms... (REST and SQL). SQL let us add logic in the view of the data that JIRA itself doesn't report on easily. (Parent-Child-subchild relationships and roll up of effort, story points, etc)
The best part of the Power BI solution is you should be able to integrate the data from JIRA and your test tool. (We pull from JIRA and our time tracking system).
We're just starting Sharepoint development, and one of my first tasks is to build a data collection tool. It will be used across multiple sites, so there will be an admin area, and each site will pull in it's related questions, and record the data. I've gone through a bunch of tutorials on development, and have a fairly good idea of how to start. I just want to make sure I understand one thing. Do lists basically take the place of your database? If this was a regular app, I would create a question table, a link table that tells which questions are connected to which site, a table that stores the answer, linking to the site and question table.
Is this the basic pattern you follow, or should I be doing things differently for Sharepoint applications?
If the thought is to use an external databse, can anyone point me to some info on this?
In our Sharepoint project we stared with Lists. It was good to some point - till DB had only few relations between data. After adding tables and relations performance falls a lot and we had to switch to use standard DB in MS SQL Server. So I recomend to use DB.
Disadventages: you cannot use sharepoint controls to edit/view data and cannot restrict access to data from sharepoint level
Adventages: much faster access to data
I often find myself writing one off queries to either answer someone's question or trouble shoot something and I would like to be able to quickly expose the on demand refreshable results of the query graphically so that I can share these results to others without having to go through the process of creating an SSRS report and publishing it to a reporting services server.
I have thought about using excel to do this or maybe running a local SSRS server but both of these options are still labor intensive and I cannot justify the time it would take to do these since no one has officially requested that I turn this data into a report.
The way I see it the business I work for has invested money in me creating these queries that often return potentially useful data that other people in the organization might want but since it isn't exposed in any way and I don't know that this data is something they want and they may not even realize they want this data, the potential value of the query is not realized. I want to increase the company's return on investment on all these one off queries that I and other developers write by exposing their results graphically so that they can be browsed by others and then potentially turned into more formalized SSRS reports if they provide enough value to justify the development of the report.
What is the fastest way for me to take a query and turn it into a refreshable graph of the results set?
Why dont you simply use what you may already have. Excel...you can import data via an ODBC / Oracle / SQL Connection. Get Data..and bam you can run the query and format it right in the spreadsheet and provide sorting etc. All you need to supply is the database name and user name and password to connect to the db.
JonH is right regarding Excel's built in ODBC support, but I have had tons of trouble with this. In my case, the ODBC connection required the client software to be installed so that it could use the encryption methods, etc. Also, even if that were not the case, the user (I believe) would still have to manually install and set up an ODBC connection.
Now if you just want something on your machine to do the queries and refresh them, JohH's solution is great and my caveats are probably irrelavent. But if you want other users to have access, you should consider having a middle-man app (basically a PHP script, assuming a web server is an option for you), that does a query, transforms the results into XML, and outputs it as "report-xyz.xml". You can then point anybody running a newer version of Excel to that address and they can very easily import the data into Excel with no overhead. (basically a kind of web service).
Keep in mind, I don't think you should have a web script that will allow users to make queries to your Database server! You would have some admin page where you make pass the query in and a new xml file with the results gets made. So my idea is also based on the idea that you want to run the same queries over and over without any specifics passed in. (if that were the case, I'd look into just finding a pre-built web services bridge for your database that already has security features built in. Then you could have users make the limited changes allowed.)
I'm considering undertaking a project to migrate a very large MS Access application to a new system based on SQL Server. The existing system is essentially an ERP application with a couple of dozen users, all sharing the Access database over the network. The database has around 300 tables and lots of messy VBA code. This system is beginning to break down (actually, it's amazing it has worked as long as it has).
Due to the size and complexity of the Access application, a 'big bang' approach is not really feasible. It seems sensible to rope off chunks of functionality and migrate them piecemeal to the new system. During the migration process, which I expect to take several months, there may be a need for both databases to be in operation and be able to query and modify data in both systems.
I have considered using something like the ADO.NET Entity Framework to implement a data abstraction layer to handle this, but as far as I can tell, the Entity Framework has no Access provider.
Does my approach seem reasonable? What other strategies have people used to accomplish similar goals?
You may find that the main problem is using the MS Access JET engine as the backend. I'm assuming that you do have an Access FE (frontend) with all objects except tables, and a BE (backend - tables only).
You may find that migrating the data to SQL Server, and linking the Access FE to that, would help alleviate problems immediately.
Then, if you don't want to continue to use MS Access as the FE, you could consider breaking it up into 'modules', and redesign modules one by one using a separate development platform.
We faced a similar situation a few years ago, but we knew from the beginning that we'll have to swich one day to SQL SERVER, so the whole code was written to work from an Access client to both Access AND SQL server databases.
The idea of having a 'one-step' migration to SQL server is certainly the easier way to manage this on the database side, and there are many tools for that. But, depending on the way your client app talks to the database, your code might then not work properly. If, for example, your code includes a lot of SQL instructions (or generates them on the fly by, for example, adding filters to SELECT instructions), your syntax might not be 'SQL server' compatible: access wildcards, dates, functions, will not work on SQL server.
In addition to this, and as said by #mjv, the other drawback of a one time switch to MS SQL is that you will inheritate many of the problems from the original database: wrong or inapropriate field names, inapropriate primary/foreign key policies, hidden one-to-many relations that you'd like to implement in the new database model, etc.
I'll propose here some principles and rules to implement a 'soft transition' solution, which clearly best fits you. Just to say that it's not going to be easy, but it's definitely very interesting, paticularly when dealing with 300 tables! Lucky you!
I assume here that yo have the ability to update the client code, and you'd prefer to keep at all times the same client interface. It is of course possible to have at transition time two different interfaces, one for each database, but this will be very confusing for the users, and a permanent source of frustration for them.
According to me, the best solution strongly depend on:
The original connection technology,
and the way data is managed in your
client's code: Access linked tables,
ODBC, ADODB, recordset, local
tables, forms recordsources, batch
updating, etc.
The possibilities to split your
tables and your app in 'mostly
independant' modules.
And you will not spare the following mandatory activities:
setup up of a transfer
procedure from Access database to SQL server. You
can use already existing tools (The
access upsizing wizard is very poor,
so do not hesitate to buy a real
one, like SSW or EMS SQL Manager,
very powerfull) or build your own
one with Visual Basic. If your plan
is to make some changes in Data
Definition, you'll definitely have
to write some code. Keep in mind
that you will run this code
maaaaaany times, so make sure that
it includes all time-saving
instructions that will allow you to
restart the process from the start
as many times as you want. You will
have to choose between 2 basic data
import strategies when importing data:
a - DELETE existing record, then INSERT imported record
b - UPDATE existing record from imported record
If you plan to switch to new Primary\foreign key types, you'll have to keep track of old identifiers in your new database model during the transition period. Do not hesitate to switch to GUID Primary Keys at this stage, especially if the plan is to replicate data on multiple sites one of these days.
This transfer procedure will be divided in modules corresponding to the 'logical' modules defined previously, and you should be able to run any of these modules independantly (keeping of course in mind that they'll probably have to be implemented in a specific order, where the 'customers' module has to run before the 'invoicing' module).
implement in your client's code the possibility to connect to both original ms-access database and new MS SQL server. Ideally, you should be able to manage from within your code both connections for displaying and validating data.
This possibility will be implemented by modules, where you will have, for each of them, a 'trial period', ie the possibility to choose at testing time between access connection and sql connection when using the module. Once testing is done and complete, the module can then be run in exclusive SQL server mode.
During the transfer period, that can last a few months, you will have to manage programatically the database constraints that exist between 'SQL server' modules and 'Access' modules. Going back to our customers/invoicing example, the customers module will be first switched to MS SQL. Before the Invoicing module can be switched, you'll have to implement programmatically the one to many relations between Customers and Invoices, where each of the tables will be in a different database. Such a constraint can be implemented on the Invoice form by populating the Customers combobox with the Customers recordset from the SQL server.
My proposal is to build your modules following your database model, allways beginning with the 'one' tables or your 'one-to-many' relations: basic lists like 'Units', 'Currencies', 'Countries', shall be switched first. You'll have a first 'hands on' experience in writting data transfer code, and managing a second connection in your client interface. You'll be then able to 'go up' in your database model, switching the 'products' and 'customers' tables (where units, countries and currencies are foreign keys) to the new server.
Good luck!
I would second the suggestion to upsize the back end to SQL Server as step 1.
I would never go to the suggested Step 2, though (i.e., replacing the Access front end with something else). I would instead suggest investing the effort in fixing the flaws of the schema, and adjusting the Access app to work with the new schema.
Obviously, it is never the case that everything just works hunky dory when you upsize -- some things that were previously quite fast will be dogs, and some things that were previously quite slow will be fast. And I've found that it is often the case that the problems are very often not where you anticipate that they will be. You can only figure out what needs to be fixed by testing.
Basically, anything that works poorly gets re-architected, or moved entirely server-side.
Leverage the investment in the existing Access app rather than tossing all that out and starting from scratch. Access is a fine front end for a SQL Server back end as long as you don't assume it's going to work just the same way as it would with a Jet/ACE back end.
...thinking out loud... I think this may work.
I appears that the complexity of the application resides in the various VBA modules rather than the database table/schema themselves. A possible migration path could therefore be to first migrate the data storage to SQL server, exactly as-is, as follow:
prevent any change to the data for a few hours
duplicate all tables to the SQL server; be sure to create the same indexes as well.
create linked tables to ODBC Source pointing to the newly created tables on SQL Server
these tables should have the very same name as the original tables (which therefore may require being renamed, say with a leading underscore, for possible reference).
Now, the application can be restarted and should be using the SQL tables rather than the Access tables. All logic should work as previously (right...), possible slowness to be expected, depending on the distance between the two machines.
All the above could be tested in about a day's work or so; the most tedious being the creation of the tables on SQL server (much of that can be automated, I'm sure). The next most tedious task is to assert that the application effectively works as previously, but with its storage on SQL.
EDIT: As suggested by a comment, I should stress that there is a [fair ?] possibility that the application would not readily work so smoothly under SQL server back-end, and could require weeks of hard work in testing and fixing. However, and unless some of these difficulties can be anticipated because of insight into the application not expressed in the question, I propose that attempting the "As-is" migration to SQL Server should be considered; after all, it may just work with minimal effort, and if it doesn't, we'd know this very quickly. This is therefore a hi-return, low risk proposal...
The main advantage sought with this approach is that there will be a single storage during the [as the OP expects] longer period during which the old Access application will co-exist with the new application.
The drawback of this approach, is that, at least at first, the schema of original database is reproduced verbatim, i.e. including some of its known quirks and legacy-herited idiosyncrasies. These schema issues (and the underlying application logic) can be in time corrected, but this is of course less easy than if the new application starts ab initio, with its own, separate, storage, and distinct schema.
After the storage is moved to SQL server, the most used and/or the most independent modules of the Access application can be re-written in the new application, and as significant portions of the original application is ported, effective usage, by select beta testers or by actual users can start to be switched to the new application.
Possibly, some kind of screen-scraping based logic or some other system could be used to produce an hybrid application which would provide the end users with a comprehensive application, which sometimes work from new logic, and sometimes from the original MS-Access program.
Good Afternoon,
A customer has provided me with a spreadsheet file his team uses to provide quotes for a product. The spreadsheet has extensive data in underlying sheets and calls those sheets as part of the numerous formulas on the quote sheet.
I've been tasked with web-enabling this tool such that the quotes can be generated on the client's website, hosted in a shared environment, running ASP.net 3.5 webforms. The quote web form will be in a Restricted portion of the website controlled by ASP.net membership API.
Are there 3rd party tools that do well with this sort of thing?
Thanks,
Sid
I've once been asked to do something similar. I refused. Someone else tried this and it became a complete failure. That's because your design is based upon the structure of a spreadsheet.
When I was called back to help on this project, I kicked the previous code into a large bin, which will stay closed until Hell freezes over! I then started analysing the Excel sheets, extracting the business logic and writing them down as documentation. This allowed me to create a good design to set up the first version of a new project, which wasn't based upon the Excel sheet, just on the business logic in the sheet.
Don't be tempted to fall into this trap! It sounds too easy but in reality, it's way too expensive as a solution because of all the problems that you will encounter!If your customer really needs to have something to use real fast, start with .NET (VS2008 with latest service packs or better) and SQL Server. First create a simple database structure to contain the quotes. Add as many relations as needed. Then, using VS2008, create a new project based on the "Dynamic Data Entities Web Application" which will use the Entity Framework to connect to the database. Make sure the entity framework is connected to your database and all data is provided to it. (By adding an "ADO.NET Entity Data Model" to it.) Compile it, put it on the web server and version 0.1 should be ready. It will allow your customer to continue data entry, although in a bit primitive way, while maintaining the data integrity of it all. It still isn't good enough to expose to the outside World but it makes it easier to get rid of the data part of the Excel spreadsheet. (And technically, it should be possible to also add a RESTful service around the entity model within an hour to allow Excel to read data from a web service instead.)
When this start page has been created, your customer can get used to data entry through web pages while you have some time to analyse their spreadsheet a bit more to optimize the data model and to create custom logic and better pages for data entry.
Basically, this is the approach that I used. My customer had data in Excel sheets which was exported to an Access database which was used as read-only data for users of his application. He used Excel to keep the data up-to-date. But when multiple users started to maintain the data in the same Excel sheet, he got in big trouble, losing the integrity and data of the Excel sheets multiple times.
I first created a simple data model around the Excel data with the Entity framework/DDS for easier data entry and then I could optimize the database structure while the customer just continued to modify the data. Occasionally, I'd update the database structure, adding more relations and restructuring the data to be more optimized and at one point I added the RESTful service functionality so the customer could get the data as XML through a simple web request. Now the customer has completely forgotten about the Excel sheet and is using the website full-time. He has accepted the fact that turning it into some fancy website will cost a lot of time but the current situation is quite acceptable.
This approach succeeded where previous attempts failed simply because I only focused on the data, nothing more. Now I can improve it all using little steps. But step one: analyze the Excel sheet and dump the data in a database. The Entity framework will allow you to quickly build simple sites around this without much effort.