Which ETL tool do you recommend? [closed] - pentaho

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
I am using Oracle as a database. We have decided to use another database and need to use ETL tool to transform the data from Oracle to that database.
I have never worked in any ETL tool and also didn't know about it. After doing some web searching, I found some tools like Pentaho, Clover, Talend etc. But, even after reading on these, I am not able to decide which one to use.
Which is the better tool to get start with, easiest to use, and has good performance?

It's impossible to answer your question without arguing on the migration requirements. Generally speaking, however, the tools your cited are all nice and focus on a particular capabilities set which could help you during the choice:
Talend: it's eclipse-based; this means that it's really powerful, stable and customizable (creating custom components or even entire eclipse plugins, if you want) and standard (ie. the project structure follows the eclipse stack). It's quite-well embeddable, as it produces java code, but this could be an hassle for the newcomer anyway. The drawback is the learning curve: if you don't manage java at all, your learning curve will be VERY steep, but if you have a java developer in your team, Talend would be a great choice.
Penthao: it's another well-estabilished solution. Its ETL tool (named Kettle) is just a component of their wider Business Intelligence open platform, which is really good if you need reporting services, OLAP, data mining and so. It's java-based, but the language is completely hidden, so you don't need to be a java developer to use it efficiently. The major drawback is that Kettle is much harder to extend than Talend. This means: poor connectors/components out-of-the-box, few community-made collections. Integration with existing java application would be REALLY a pain in the neck, too!
CloverETL: I don't use it very often, as it's mostly the younger brothers of the others. Its major advantages are: it's light, easily embeddable and easy to learn. But it's really much less powerful than Talend and even than Kettle.

Related

Is there any static analysis tool that can be used as a "tutorial" towards this concept? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
I know the basic concept of static analysis, dynamic analysis and have some experiences using some tools on dataflow analysis, taint analysis and some model checking work.
Currently I am trying to get a deeper understanding of static analysis by reading some implementation of static analysis tools.
Is there any recommendation..?
Tools that work on dynamic typing language are especially welcomed:)
Thank you!
Static analysis tools come in wide variety, having to do with differences in:
Purpose of the tool (find dead variables? find subscript errors? determine dynamic variable type?)
Method to achieve it (heuristic? conservative algorithm? theorem prover?)
Target languages and properties analyzed
Representations processed (text, ASTs, byte code, mixtures)
User annotations to help guide the analysis (when facing Turing-hard analyses, sometimes a hint is huge help)
User interface, IDE integration, and reporting
...
This is far too much to learn about by reading some implementation. You virtually never learn anything significant by reading source code. Worse, 90% of the source code handles junk that is unrelated to the above issues ("open file", "parse lexeme", "print unicode string").
You are better off getting a textbook on program analysis and reading that to understand at least some of the above issues.
I'd suggest to go and look at WALA and Soot. They're very academic implementations, but they have very interesting algo implemented.
If you want to start with lightweight (intraprocedural, no global dataflow, etc.) analyses, you can look at Findbugs.
Concerning the dynamically typed languages, WALA had some support for JavaScript, but AFAIK it is fairly limited (though haven't looked at it deeply enough). There are academic tools for JavaScript though, such as TAJS. Google closure compiler is also interesting in that regard.

Software development tools - from user goals to tests [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
I'm beginning a new project of about 1 year of development (for the first version) done with multiple developers, testers, etc.
I'm wondering if something exist that could help me do the following:
List all user goals
Associate functions to these user goals
Associate requirements to these functions
Associate design activities to these requirements
Associate development tasks to these requirements
Associate tests to these requirement
Qualify tests (system test, regression test, developer test, automated or not)
This way, I could:
Track if the program developed fulfill all user goals
Track if all functions are tested
Do a test matrix traceability to know if each requirements is tested
Track tests to do if a function is to be changed
Track the time needed to develop a function (it can serve later to estimate the time needed to modify it or to add a similar function to the program)
List all system tests to do when a new version is shipped
List all regression tests to do
List all developer test to do when there is a change in the function
List all automated test, this way we could know what is the percentage of the functions that are automatically testes.
etc.
You can suggest open source or commercial programs.
The Atlassian suite of software would seem to be a good fit and is very cheaply priced for a few users ($10 for up to ten users). I've direct (and good!) experience of using JIRA and find it very simple to use and flexible enough for my needs. Another alternative would be FogBugz, but I've no first-hand experience of using this.
re FogBugz, it is well worth having a look at the processes behind it, having worked on many non software projects I believe it is a universally sound methodology (even if Joel is a little quirky in his thinking.....).
I use SmartSheet because it is simple, but still has heirachial tasking, as you have set out in the question. It is good at dealing with people, unlikely it is good at manageing code, whereas FogBugz presumably does that.
A key feature of SS over Atl and others is additional users cost nothing.
One decision you have to make is do you want the project plan to be output in a simple way which many stakeholders can understand, or detailed so you can track much activity. Obviously the detail will require effort.
You have made a good start by setting out the issues, your culture of management may well be more valuable than the tool you choose.
ciao

How to document applications and how they integrate with other applications? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
As the years go by we get more and more applications. Figuring out if one application is using a feature from another application can be hard. If we change something in application A, will something in application B break?
We have been using MediaWiki for documentation, but it's hard to keep the data up-to-date.
I think what we need is some kind of visual map of everything. And the possibility to create some sort of reference integrity? Any ideas?
I'm in the same boat and still trying to sell my peers on Enterprise Architect, a CASE tool. It's a round trip tool - code to diagrams to code is possible. It's a UML centric too - although it also supports other methods of notation that I'm unfamiliar with...
Here are some things to consider when selecting a tool for documenting designs (be they inter-system communication, or just designing the internals of a single app):
Usability of the tool. That is, how easy is it to not only create, but also maintain the data you're interested in.
Familiarity with the notation.
A. The notation, such as UML, must be one your staff understands. If you try using a UML tool with a few people understanding how to use it properly you will get a big ball of confusion as some people document things incorrectly, and someone who understands what the UML says to implement either spots the error, or goes ahead and implements the erroneously documented item. Conversely more sophisticated notations used by the adept will confound the uninitiated.
B. Documentation isn't/shouldn't be created only for the documenters exclusive use. So those who will be reading the documentation must understand what they're reading. So getting a tool with flexible output options is always a good choice.
Cost. There are far more advanced tools than Enterprise Architect. My reasoning for using this one tool is that due to lack of UML familiarity and high pressure schedules, leaves little room to educate myself or my peers beyond using basic structure diagrams. This tool easily facilitates such a use and is more stable than say StarUML. (I tried both, StarUML died on the reverse engineering of masses of code -- millions of lines) For small projects I found StarUML adequate for home use, up until I got vista installed. Being opensource, it's also free.
With all that said, you will always have to document what uses what, that means maintaining the documentation! That task is one few companies see the value in despite its obvious value to those who get to do it. . .

What are the elements of a team development suite? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
For small-to-large teams developing software together, what tools are used to form a comprehensive team development framework?
Specifically, I'm looking for a comprehensive list of all the individual functions involved (e.g. source control, bug management, testing tools, project management), not specific product recommendations. I'm also not restricting the list to a particular methodology (e.g. Scrum).
Source control (obviously) including branch management
Issue tracking (features and bugs), possibly with task reassignment and forwarding, and often things like screen recording
Individual task management, sometimes integrated with the issue tracking system
Communication software. Some teams use emails and IMs even within the same building or tweets. There are some tools that integrated within the code so you could "chat around a piece of code". Screen and application sharing are also useful.
Good build tool.
Distributed pair programming tools if applicable, shared editors otherwise.
Similar support in case tools.
Less commonly used but promising tools (from academic background), some now have IDE based versions.
Real-time awareness (prevent nerge conflicts by letting you know somebody is working on the same file before you actually write code)
In-code social tagging, useful for bootmarking specific items
In-code contract communication tools (e.g., make a caller aware of special expectations in the invoked method as a way of avoiding errors).
You've hit the major ones in your post:
IDE (Integrated Development Environment)
Coding Guidelines (sometimes looked over, but it still helps tremendously)
Source Control
Testing Suite (Unit Testing, Test Case/Test Script Management and Tracking)
Issue Tracking/Bug Reporting
Build Management
...I'm sure I'm missing something obvious, but somebody around here will correct me.
And the one I missed...
Diagraming software (I.E. Rational Software Modeler, etc.)
A few more:
Requirements management software
Code review software
Continuous integration tool
Documentation repository - e.g. Wiki

Good challenges/tasks/exercises for learning or improving object oriented programming (OOP) skills [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
What is a good challenge to improve your skills in object oriented programming?
The idea behind this poll is to provide an idea of which exercises are useful for learning OOP.
The challenge should be as language agnostic as possible, requiring either little or no use of specific libraries, or only the most common of libraries. Try to include only one challenge per answer, so that a vote will correspond to the merits of that challenge alone. Would also be nice if a level of skill required was indicated, and also the rationale behind why it is a useful exercise.
Solutions to the challenges could then be posted as answers to a "How to..." question and linked to from here.
For example:
Challenge - implement a last-in-first-out stack
Skill level - beginner
Rationale - gives experience of how to reference objects
Building Skills in Object-Oriented Design is a free book that might be of use.
The description is as follows:
"The intent of this book is to help the beginning designer by giving them a sequence of interesting and moderately complex exercises in OO design. This book can also help managers develop a level of comfort with the process of OO software development. The applications we will build are a step above trivial, and will require some careful thought and design. Further, because the applications are largely recreational in nature, they are interesting and engaging. This book allows the reader to explore the processes and artifacts of OO design before project deadlines make good design seem impossible."
Write a challenging program from scratch. Try to get some people (around five, that should be doable) to use it. Respond to their change requests.
Adapt your program's design. Start small, then watch it grow. Manage this growth. This is hard. You will also have to fix bugs and maintain the thing over time, which for me was a very valuable lesson.
Challenge: Write a wrapper for your web site/service API of choice in your language of choice, that doesn't already exist (ex. a ZenDesk API wrapper written in C#). Release the wrapper as open source for others to use.
Skill Level: Beginner to Intermediate
Rationale: To learn how to extrapolate a 3rd party web service API into a meaningful set of objects/classes, making the reuse of that API easier in your chosen language.
After you have learned the basics, study the "Gang of four" design patterns book.
http://www.amazon.com/Design-Patterns-Object-Oriented-Addison-Wesley-Professional/dp/0201633612/ref=pd_bbs_sr_1?ie=UTF8&s=books&qid=1221488916&sr=8-1
This is a classic, and a must read for any coder who wants to understand how to use OO to design elegant solutions to common coding problems.
Take a procedural-style written piece of code and try to transform it into OOP based solution. During the process, consult a book on refactoring and design patterns. A friend of mine was able to make a huge step forward in understanding object oriented concepts exactly this way. As with anything, this might not work for everyone.
I have found CRC cards to be quite effective in learning, teaching and building good OO design.
Certainly a good challenge, although less accessible than a "start from scratch" assignment, is to refactor some existing code that either doesn't use inheritance or doesn't use very much of it to make greater use of inheritance. The process of refactoring will expose a lot of the benefits and gotchas of oop, as it certainly has for me on my most recent project. It also pushed me to understand the concepts better than past projects have where I've created my own object oriented designs.
A given task has very little to do with being "OOP", it's more in how you grade it.
I would look at the Refactoring book, chapter 3, and make sure none of the bad code smells exist in the solution. Or, more importantly, go over ones that do apply.
Most importantly, watch for the existence of setters and getters (indicating that you are operating on values from a class and not asking the class to operate on it's own values)--or using "extends" without applying the Liskov Substitution Principle, stuff like that.