Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I have recently heard about Magic programming language from several sources and didn't recall ever hearing about it before. It was mentioned that it is a programming language from Israel.
I did some googling and couldn't find much information about it. I couldn't find any code examples, and wikipedia didn't have any information on it either.
I think this is the site for it http://www.magicsoftware.com/en/products/?catID=70 though I am not sure, as it mentions uniPaaS instead of magic. However other material on the site indicates that this is the new name for it.
I was interested in learning more about it from it's practitioners, rather than the company. I saw several claims on the internet that it provided really fast application development, similar to claims made by RoR proponents when it came out.
How does it compare to VB?
Is it still a better RAD tool than current .net or mvc frameworks like django, ror ...etc?
How hard is it to learn?
If you can post some sample code it would be most helpful as well.
Could this site be it? Though it links back to the page above.
You're right my friend, Magic is the original name of the "programming language", nowadays is called UniPaaS (Uni Platform as a Service), I use it to develop some business application. Maybe is the fastest way to create an applications(data manipulation), you can create apps in just a few days, but like everything in life has its own drawbacks:
it's very weird so that makes it
difficult to learn.
you do not have all the control of what's happening in the background
and you have to pay a lot for licensing (servers,clients, etc)
If you are interested in learning this, you can download a "free" version of the software that only works with sqlite databases called UniPaaS Jet.
Magic Language is as it’s called today uniPaaS, it used to be Magic than eDeveloper and now uniPaaS as PachinSV menchend before.
uniPaaS is an application platform enabling enterprises, independent software vendors (ISVs) and system integrators (SIs) to more successfully build and deploy business applications.
You can download the free version of uniPaaS Jet here: http://web.magicsoftware.com/unipaas-jet-download,
try it yourself and see how easy it is to use.
Magic technology as you descried is a Magic Software Enterprises tool (uniPaaS), you can find more information on:
official website: www.magicsoftware.com/en/products/?catID=70&pageID=55
uniPaaS Jet developer group on facebook: https://www.facebook.com/groups/unipaasJet/
Magic developer zone: devnet.magicsoftware.com/en/unipaas
Let me know if you find the information helpful
Bob
As PachinSV explained, there is a RAD once called Magic, then eDeveloper, now UniPaaS. This RAD is dedicated for database applications. Programming in this RAD does not look like anything else I know, you mostly don't write code as with usual languages, but it is nearly impossible to explain just with words. The applications are interpreted, not compiled.
As PachinSV said, when developing, you must follow UniPaaS' way of doing things. This is probably why so many people never manage to use Magic properly: if you thought like Magic before learning about it, then you will adapt to it easily; but if you have a long and successful experience using other database development tools, then often the Magic paradigm will never become natural to you. The learning curve is quite steep, you must learn a lot of things before being able to write a little application.
Previous versions stored the "code" inside a database table. The last version, UniPaas stores the code in xml files. I could send you an example, if PachinSV does not answer you before. But the files are pretty big: the smallest xml file I have in a test app is 4000 bytes, and any application is made of at least 11 files, an empty application is 7600 bytes. You must also understand that developers never use those files (they are undocumented AFAIK), they are only the storage format used internally by UniPaaS. The only way to use them is to set them up as a UniPaaS application.
I'm still an active MAGIC Developer... This is the old name used and its a completely different paradigm like some of you mentioned. I've been developing it from Magic version 8.x to eDeveloper 9.x to 10.x then renamed to UniPAAS.
The newer version is much easier to use and it is still very RAD in the sense that there is little or no code you write... a lot of the common programming tasks like IO, SQL command...etc is handled by the tool and is transparent ( so even less code to write since we use it in almost all types of applications)... Its mostly an Enterprise tool... you wouldnt use it for small application...
You can download the free version to learn the paradigm... but the enterprise licenses are expensive.. you need both the development tool and the runtime license if you want to deploy... so it can be costly for small scale projects...
I enjoy it personally, especially when you have to do quick proof of concepts or a quick data migration or porting onto any db platform and bridging any existing system through a wide range of gateways they provide with the licensed version.. It is up to date with the commonly used web technology out there...like SOAP, RIA ...
It's more popular in Europe... The HQ in the States is in Irvine... we used to have 2 branches in Canada but it closed down in 2001 .... Visit the Magic User Group on Yahoo... Its a very active forum with lots of cool people who will help you out in your quest...
http://tech.groups.yahoo.com/group/magicu-l/
I Programmed with Magic for 6 years and found it to be a amazingly fast tool, easy to understand if you are a competent database programmmer because all operations are really about data manipulation. It is certainly a niche area develop in and because of this jobs are few and far between. As it is interpreted there are really no bugs to make. It will work with many databases/connections simultaneously but there is a big memory and processing hit.
Drawbacks :
Little control over communications between machines and devices
No mobile API as yet
Niche area so few skilled practitioners or companies willing to invest.
Good Points :
You can say you are a Magician; you can impress people with uber fast apps development (really)
It is easy to understand if you don't have a PHD in Maths
zero programming "bugs" can creep in. What you do is what you get.
Developed in The original Magic PC referred to by several of the above folks.
It is exactly this: FAST, FAST, but expensive and rigid in what it will allow you to do. It works on a tick tack toe like matrix. Dropping in commands into the various sections determines when they are run. The middle column is run indefinitely until you break the cycle. It is like a do Until loop. If you have to do an item once you put it into this infinite loop and end it after one cycle.
The first column procedures are run first, ONCE, before the infinite middle column is run. The 3rd column of commands is run after the infinite cycle, once. It is very efficient and logical once you get over the idea of an infinite loop.
Types can be specified and an associated program to present the type. Then everywhere the type is used all the settings automatically kick in. I like especially that one can write the program and 5 months later change the name of a variable and it is carried throughout the program. In fact the program does not use your name for anything. The internal name of any and all variables is hidden to the end user, so of course it is not a problem to change a name. It takes a minute to write an input program for any table. It takes a minute to write an export/import program for all the data files in the database.
Attaching to a type of database like Btrieve or SQL independent of the program itself.
I stopped using the language because they demand more for the runtime engine than I could charge for the programs I wished to run with it. Bill Gates went the opposite direction. VB is superior in control and being able to drop `10 datagridviews onto the same screen, but development is 10 times slower.
It's niche then is PROOF of concept for a program in a big company or conversion, importing, exporting for a development company. It is good for $25k programs that are database heavy and not going mobile.
uniPaaS, Magic PC
I did some Magic work around 1993. It was a DOS based 4GL that came from Israel. Haven't seen it since.
How does it compare to VB?
It doesn't.
Is it still a better RAD tool than current .net or mvc frameworks like django, ror ...etc?
If you mean "is it more Rapid", then yes, otherwise no.
How hard is it to learn?
About as hard as learning MS Access.
Coincidentally, if you want to get an idea of what it is and how it works, I've found that comparing it to MS Access is handy. It works in much the same way from a user's or developer's perspective. Obviously what happens in the background is vastly different, but if you've ever developed a form in design view in Access, Magic will seem very familiar.
Google tells me there's also MAGIC/L. All I could find about it was this blurb:
A procedural language written in
Forth. Originally ran on Z80's under
CP/M and later available for IBM-PCs
and Sun 3s.
The only Magic programming language that I know about is one used by a company called Meditech. It's a proprietary language derived from MUMPS.
The language is truly miserable - here's a sample.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
I know that this question is too abstract. But. How much time do i need to learn LabVIEW to become average LabVIEW developer? For example, if I buy good book about LabVIEW and have 8 hours per day (on my work) dedicated to LabVIEW learning how many days i will spend on LabVIEW learning? Could you please provide example from your own experience. More information about me that can be helpful: I'm a developer and know c\c++\python and a little bit of java languages.
Like Swinders said, it might depend a lot on your sensibilities. I have seen people who had a really hard time migrating to the data flow concept. It's a different paradigm from the classic text-based languages and some people can't easily think in these concepts.
If you get past that hurdle, you'll find that the IDE handles a lot of the annoying things you used to take care of for you (things such as syntax and memory allocation). This allows you to become productive very quickly.
It doesn't mean, however, that your level would be high. One potential pit you should try hard to avoid is casting your existing experience onto LV. The most common example is probably local variables. This may be shocking to people coming from a text-based world, but LV does not have variables, per-se. Unfortunately, it does have elements called variables and people migrating from C who find them jump on them and use them as they would use variables in C, leading to LV code which looks like C code and is bad code (at least in LV).
If you do manage to work around this, I would guess you would become better than the global average in less than a month and better than most professional developers after creating three projects you would later look at and say "what the hell was I thinking?".
I never took any of the NI courses (although I understand some of the advanced architecture ones are pretty good), but I would suggest you also spend some time in some of the online communities (such as LAVA or the NI forums) and look at some of the examples and discussions there. There's a lot of material about best practices, design patterns, etc., which would allow you to become a more professional developer.
Above all, do not abandon your current professional conduct. If you have a structured process for designing and developing software, you already have a leg up on the majority of LV programmers. Just make sure you adapt and keep using such a process.
I started with no commercial programming experience (I have always programmed for fun) and followed an on-line tutorial to pick up the basics of LabVIEW. Within a week I was able to understand existing code and could develop a small application.
It is hard to give an estimate on how long it would take to become an 'average' LabVIEW developer as this depends on what you mean by 'average'. One thing to consider is how easy you are able to think in terms of data flow rather than procedural languages. If you can pick up new programming languages quickly then this will help.
Would you be the only person using LabVIEW or are there others at your place of work that could mentor you? You may also find that there are user groups operating near you which I would recommend (check the NI website or contact your local NI office).
There is then the experience that you will need to gain to allow you to produce good LabVIEW code. I was lucky to be able to attend the National Instruments training courses a few years ago which I think helped me but only by using it have I become an 'average' LabVIEW developer.
I'd say a few weeks or most with devoting the majority of your work time to it. I had a similar background to you when I started to develop in LabVIEW. The hardest part was adapting to the lack of variables. There are local variables, but it's not what you're used to at all. Additionally, their functions, called Virtual Instruments (VIs) can have multiple inputs and outputs, similar to how Python can handle n-tuples.
I will warn you, their array handling features are terrible. A lot of general concepts you might be used to are difficult to implement. My mantra when working with the language is it makes hard things easy and easy things hard. There are also a lot of "gotchas" in the language set, especially with their DAQmx function. I'm not sure what you're planning on developing and their Real-Time module has it's own issues as well, different issues from the main language set.
I would definitely spend some time on NI's website and read as many whitepapers as you can, especially about good design practices, here and here. Learn their State Machine (here or here) and Producer/Consumer pattern well, that's the backbone of many applications you'll be writing.
Good luck, it will make your head spin for a while.
There are some excellent resources to help you get started. If your employer can afford training, you can get started pretty quickly by taking a week of training run by National Instruments. The NI website also has an outstanding developer community that is highly responsive to questions even from novice developers. But I would say that the key to being comfortable with the idioms and style of the language is just plain old practice that you get by solving problems using LabVIEW on a regular basis.
You will find eventually that there is the question of hardware and instruments. Labview is really all about data acquisition-- either through NI's DAQ hardware or through traditional GPIB instruments, or through 3rd party api's (activeX, .NET assemblies). If you're using LabVIEW, you're probably interfacing to hardware of some type. This can get really challenging with complex instruments and measurements. If you're getting started, I would recommend making sure that you have unlimited access to at least some of the hardware you'll be working with. In other words, make sure that your manager understands that you need a lot of access to the hardware in order to get good at developing with it.
We are using LabVIEW to create test software for our factory test systems. In the past years I have already trained some beginners to understand LabVIEW. I would say it depends on how good you are at learning new concepts. I have trained some to be able to produce standalone applications using the queued message handler concept, doing dynamic GUIs and using hardware drivers within about 3 weeks. Unfortunately there were others aswell that were only able to learn half of that within half a year.
The most important thing in my opinion is the learning source. Having an experienced LabVIEW user that can guide you is the best option. If there is no one not available I would recommend Youtube Tutorials combined with the shipped LabVIEW examples.
The LabVIEW core tutorials are not very handy in my opinion. Those are quite boring and far from what you really need to get started.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
Definition:
CRUD - Create, Read, Update, Delete; The four basic functions of persistent storage. In the context of this question, specifically related to business applications.
I'll be honest, my goal when I began programming did not include being a lifetime CRUD programmer. Financial data is only so interesting for so long. And to me, that seems like the majority of programming gigs.
I'm still fairly fresh out of school, so any experience is still very beneficial, but eventually I want to move to something "less CRUD like." Currently, I have my eye on some machine control type work. However, I'm just not sure how to go in that direction.
So I want to get a feel for what other developers think about the topic.
Do you enjoy CRUD and why?
What have your experiences in CRUD been like?
How did you move from CRUD to non-CRUD work?
If you've moved, what do you like and hate?
If you've moved, what skills benefited/hastened the transition?
Edit:
I'm approaching CRUD with the attitude that I want to solve problems, not re-create the same form with different fields for a dozen different tables.
I don't think that there's really anyone who enjoys doing CRUD (well at least anyone sane). It's the most tedious part of web programming. My advice is to find or write a framework to automate this for you.
evolutility
django admin panel and django forms
However, if that's the majority of your work, you definitely should consider changing jobs.
Get a different job. Seriously, not all software development is developing business applications. Developing shipping software would make you much happier, I think. Try to find a job at a software company, and write some stuff that's going to ship to customers. Also, if you want to get into some of the low level hardware-style stuff, just start hacking away on some basic microcontrollers so you have at least some background with that.
Develop a framework to make CRUD creation easy in your line of work. Once you have done that, use the free time to improve it in terms of Usability, Security, Performance etc. That should keep your work interesting for a while.
I agree that CRUD's pretty boring. But I don't think it's the fact that it's financial data that makes it so. Perhaps you'd find that financial data a lot more interesting if, for example, it was streaming into a neural net based expert system you'd written to work out how best to invest it?
There's definitely an awful lot more to programming than CRUD. Find an aspect that interests you, and pursue it.
I am curious that no one mention task-based UI and CQRS here.
In fact, to answer your questions :
I don't enjoy CRUD...why ? see the following answers to your question
My experience of CRUD is that's a pain to write CRUD (full stack frameworks are a workaround at best I think), and often a pain for users as well
I move to non-CRUD work when I understand that building software is about giving a powerful tool for users, not a database editor with some business rules
I like to build software less coupled to full stack framework (Symfony2, ASP.NET & cie...), more fully object oriented, but I am more and more annoyed by RDBMS CRUD orientation, and more and more attracted by EventStore (Event Sourcing)
Let's get inspired by task based UI, CQRS and Event Sourcing (search Google, I do not have enough reputation to add more links...) => all together
However, I would like to be less opinionated to finish : there are some points that will not let you get out of CRUD. Some users love CRUD, they feel like in Excel...and also there are probably some applications for which CRUD fulfill all the needs...
CRUD - yes in the end we are storing, reading and updating data. But so what? That is just one part of the equation, at least in my world.
In business, data is essential, but it is the business logic and the decisions made from that data that is important. I have found it very rewarding to take raw data and use it to help business make decisions. We do that with business logic in our code, not to mention the endless ways of presenting that data in the presentation layer.
Yes in the end CRUD is involved, but it is much more than that, no?
Just my opinion.
Having a wide range of experience, my solution is to create my perfect product and start a business around it. I'm facing all sorts of interesting challenges, such as how to stream realtime data from an embedded device to a browser. This stuff gets my programming juices flowing and I have a list of important, fun features to add.
Dream up your perfect product domain, find some people who could benefit and ask them what problems they have. Once you pick up a common theme that interests you (mine was automation and power monitoring) start hacking. Of course for me it helps that my father has run the electronics company Technman for the last 30 years, and wants to create this product with me.
First, have you gone through most of what there is to know about persistent storage? It's worth figuring out how to practically apply database theory, etc. in your current job. Once you've been doing it for a few years and have it all figured out you should definitely think about expanding your horizons. I'd agree with you - unless you're building the DBMS itself - I find that the persistent storage part of the job gets to be fairly boring.
One of the best ways to get a job in a new area is to take a prototype of something relevant to the job to demonstrate at an interview. This is an incredibly powerful statement to make.
Embedded software is really my thing, but the market for this is slowly shrinking in North America and moving to the developing world, and it's a fairly specialized area to get started in.
It seems to me that the application space is still growing. Consider iPhone, J2ME, or Windows Mobile development for example. You can learn to do these on your own with a relatively small investment in equipment.
If you're not already doing this, there's also the web application space. Application server platforms like JBOSS and Glassfish are free and fairly easy to learn. Plus they provide a link back to the CRUD which you already know.
Yes, a lot of business software is CRUD. I used to work on that.
In machine control, part of that can be CRUD too. For example, logging sensor data and reporting it somewhere. Basically CRUD.
But I will admit - in machine control, it's mostly non-CRUD. You would probably enjoy doing something that actually makes an assembly line move, or builds cars, or makes motors spin at a certain speed. I know I do. At a financial institution, it's literally just numbers. Nothing "real" like a motor or a car.
Just about every program is going to have to create, read, update and delete some sort of data. In some systems this presents its' own challenges.
However most of the time reading and writing to databases is fairly easy (which is why they make databases). It is what you do with the data once you have it which is interesting, and generally unique to a business, and keeps you employed.
This article I agree with, basically a lot of programming is boring.
However if you are good and determined enough you will eventually get to do something interesting.
Find or write a way to do the CRUD portions of the applications faster. Do so, tell your manager you are done with your assigned tasks (make sure they ARE done; tested, documented, etc.), and ask what you should do next.
Just take a look to Django and move on to the interesting coding!!!
(Or RoR, or Grails, or whichever suit best to you, but CRUDS shouldn't be still being coded by hand from scratch)
Modern frameworks can do all the crud for you. Checkout the standalone GORM from the GRAILS project.
When I was an undergraduate, I changed my major from Electrical Engineering to Computer Science because I wanted to write video games. Later on, when I started working on business applications for real money, I learned that I simply enjoy solving problems with code.
You may be in the wrong profession.
In this economy, it might be hard for you to just get another job, but that doesn't mean you shouldn't try. Find some type of work you think you would enjoy, go learn it and look for job opportunities. It doesn't hurt to make some phone calls and go on a few interviews even if you think you're unlikely to get the job. Even better, you could figure out a way to start your own company.
Get into web-dev? Seriously the level of basic crud I have to do building web-apps is pretty low, even when there's a DB.
For CRUD of windows FORM based applications developed in c# .net
RocketFramework is the answer
Everyone I work with is obsessed with the data-centric approach to enterprise development and hates the idea of using custom collections/objects. What is the best way to convince them otherwise?
Do it by example and tread lightly. Anything stronger will just alienate you from the rest of the team.
Remember to consider the possibility that they're onto something you've missed. Being part of a team means taking turns learning & teaching.
No single person has all the answers.
If you are working on legacy code (e.g., apps ported from .NET 1.x to 2.0 or 3.5) then it would be a bad idea to depart from datasets. Why change something that already works?
If you are, however, creating a new apps, there a few things that you can cite:
Appeal to experiencing pain in maintaining apps that stick with DataSets
Cite performance benefits for your new approach
Bait them with a good middle-ground. Move to .NET 3.5, and promote LINQ to SQL, for instance: while still sticking to data-driven architecture, is a huge, huge departure to string-indexed data sets, and enforces... voila! Custom collections -- in a manner that is hidden from them.
What is important is that whatever approach you use you remain consistent, and you are completely honest with the pros and cons of your approaches.
If all else fails (e.g., you have a development team that utterly refuses to budge from old practices and is skeptical of learning new things), this is a very, very clear sign that you've outgrown your team it's time to leave your company!
Remember to consider the possibility that they're onto something you've missed. Being part of a team means taking turns learning & teaching.
Seconded. The whole idea that "enterprise development" is somehow distinct from (and usually the implication is 'more important than') normal development really irks me.
If there really is a benefit for using some technology, then you'll need to come up with a considered list of all the pros and cons that would occur if you switched.
Present this list to your co workers along with explanations and examples for each one.
You have to be realistic when creating this list. You can't just say "Saves us lots of time!!! WIN!!" without addressing the fact that sometimes it is going to take MORE time, will require X months to come up to speed on the new tech, etc. You have to show concrete examples where it will save time, and exactly how.
Likewise you can't just skirt over the cons as if they don't matter, your co-workers will call you on it.
If you don't do these things, or come across as just pushing what you personally like, nobody is going to take you seriously, and you'll just get a reputation for being the guy who's full of enthusiasm and energy but has no idea about anything.
BTW. Look out for this particular con. It will trump everything, unless you have a lot of strong cases for all your other stuff:
Requires 12+ months work porting our existing code. You lose.
Of course, "it depends" on the situation. Sometimes DataSets or DataTables are more suited, like if it really is pretty light business logic, flat hierarchy of entities/records, or featuring some versioning capabilities.
Custom object collections shine when you want to implement a deep hierarchy/graph of objects that cannot be efficiently represented in flat 2D tables. What you can demonstrate is a large graph of objects and getting certain events to propagate down the correct branches without invoking inappropriate objects in other branches. That way it is not necessary to loop or Select through each and every DataTable just to get the child records.
For example, in a project I got involved in two and half years ago, there was a UI module that is supposed to display questions and answer controls in a single WinForms DataGrid (to be more specific, it was Infragistics' UltraGrid). Some more tricky requirements
The answer control for a question can be anything - text box, check box options, radio button options, drop-down lists, or even to pop up a custom dialog box that may pull more data from a web service.
Depending on what the user answered, it can trigger more sub-questions to appear directly under the parent question. If a different answer is given later, it should expose another set of sub-questions (if any) related to that answer.
The original implementation was written entirely in DataSets, DataTables, and arrays. The amount of looping through the hundreds of rows for multiple tables was purely mind-bending. It did not help the programmer came from a C++ background attempting to ref everything (hello, objects living in the heap use reference variables, like pointers!). Nobody, not even the originally programmer, could explain why the code is doing what it does. I came into the scene more than six months after this, and it was stil flooded with bugs. No wonder the 2nd-generation developer I took over from decided to quit.
Two months of tying to fix the chaotic mess, I took it upon myself to redesign the entire module into an object-oriented graph to solve this problem. yeap, complete with abstract classes (to render different answer control on a grid cell depending on question type), delegates and eventing. The end result was a 2D dataGrid binded to a deep hierarchy of questions, naturally sorted according to the parent-child arrangement. When a parent question's answer changed, it would raise an event to the children questions and they would automatically show/hide their rows in the grid according to the parent's answer. Only question objects down that path were affected. The UI responsiveness of this solution compared to the old method was by orders of magnitude.
Ironically, I wanted to post a question that was the exact opposite of this. Most of the programmers I've worked with have gone with the custom data objects/collections approach. It breaks my heart to watch someone with their SQL Server table definition open on one monitor, slowly typing up a matching row-wrapper class in Visual Studio in another monitor (complete with private properties and getters-setters for each column). It's especially painful if they're also prone to creating 60-column tables. I know there are ORM systems that can build these classes automagically, but I've seen the manual approach used much more frequently.
Engineering choices always involve trade-offs between the pros and cons of the available options. The DataSet-centric approach has its advantages (db-table-like in-memory representation of actual db data, classes written by people who know what they're doing, familiar to large pool of developers etc.), as do custom data objects (compile-type checking, users don't need to learn SQL etc.). If everyone else at your company is going the DataSet route, it's at least technically possible that DataSets are the best choice for what they're doing.
Datasets/tables aren't so bad are they?
Best advise I can give is to use it as much as you can in your own code, and hopefully through peer reviews and bugfixes, the other developers will see how code becomes more readable. (make sure to push the point when these occurrences happen).
Ultimately if the code works, then the rest is semantics is my view.
I guess you can trying selling the idea of O/R mapping and mapper tools. The benefit of treating rows as objects is pretty powerful.
I think you should focus on the performance. If you can create an application that shows the performance difference when using DataSets vs Custom Entities. Also, try to show them Domain Driven Design principles and how it fits with entity frameworks.
Don't make it a religion or faith discussion. Those are hard to win (and is not what you want anyway)
Don't frame it the way you just did in your question. The issue is not getting anyone to agree that this way or that way is the general way they should work. You should talk about how each one needs to think in order to make the right choice at any given time. give an example for when to use dataSet, and when not to.
I had developers using dataTables to store data they fetched from the database and then have business logic code using that dataTable... And I showed them how I reduced the time to load a page from taking 7 seconds of 100% CPU (on the web server) to not being able to see the CPU line move at all.. by changing the memory object from dataTable to Hash table.
So take an example or case that you thing is better implemented differently, and win that battle. Don't fight the a high level war...
If Interoperability is/will be a concern down the line, DataSet is definitely not the right direction to go in. You CAN expose DataSets/DataTables over a service but whether you SHOULD or is debatable. If you are talking .NET->.NET you're probably Ok, otherwise you are going to have a very unhappy client developer from the other side of the fence consuming your service
You can't convince them otherwise. Pick a smaller challenge or move to a different organization. If your manager respects you see if you can do a project in the domain-driven style as a sort of technology trial.
If you can profile, just Do it and profile. Datasets are heavier then a simple Collection<T>
DataReaders are faster then using Adapters...
Changing behavior in an objects is much easier than massaging a dataset
Anyway: Just Do It, ask for forgiveness not permission.
Most programmers don't like to stray out of their comfort zones (note that the intersection of the 'most programmers' set and the 'Stack Overflow' set is the probably the empty set). "If it worked before (or even just worked) then keep on doing it". The project I'm currently on required a lot of argument to get the older programmers to use XML/schemas/data sets instead of just CSV files (the previous version of the software used CSV's). It's not perfect, the schemas aren't robust enough at validating the data. But it's a step in the right direction. The code I develop uses OO abstractions on the data sets rather than passing data set objects around. Generally, it's best to teach by example, one small step at a time.
There is already some very good advice here but you'll still have a job to convince your colleagues if all you have to back you up is a few supportive comments on stackoverflow.
And, if they are as sceptical as they sound, you are going to need more ammo.
First, get a copy of Martin Fowler's "Patterns of Enterprise Architecture" which contains a detailed analysis of a variety of data access techniques.
Read it.
Then force them all to read it.
Job done.
data-centric means less code-complexity.
custom objects means potentially hundreds of additional objects to organize, maintain, and generally live with. It's also going to be a bit faster.
I think it's really a code-complexity vs performance question, which can be answered by the needs of your app.
Start small. Is there a utility app you can use to illustrate your point?
For instance, at a place where I worked, the main application had a complicated build process, involving changing config files, installing a service, etc.
So I wrote an app to automate the build process. It had a rudimentary WinForms UI. But since we were moving towards WPF, I changed it to a WPF UI, while keeping the WinForms UI as well, thanks to Model-View-Presenter. For those who weren't familiar with Model-View-Presenter, it was an easily-comprehensible example they could refer to.
Similarly, find something small where you can show them what a non-DataSet app would look like without having to make a major development investment.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
I have being toying with the idea of creating software “Robots” to help on different areas of the development process, repetitive task, automatable task, etc.
I have quite a few ideas where to begin.
My problem is that I work mostly alone, as a freelancer, and work tends to pill up, and I don’t like to extend or “blow” deadline dates.
I have investigated and use quite a few productivity tools. I have investigated CodeGeneration and I am projecting a tool to generate portions of code. I use codeReuse techniques. Etc.
Any one as toughs about this ? as there any good articles.
I wouldn't like to use code generation, but I have developed many tools to help me do many of the repetitive tasks.
Some of these could do nice things:
Email Robots
These receive emails and do a lot of stuff with them, they need to have some king of authentication to protect you from the bad stuff :
Automatically logs whatever was entered in a database or excel spreadsheet.
Updates something in a database.
Saves all the attachments in a specific shared folder.
Reboot a server.
Productivity
These will do repetitious tasks:
Print out all the invoices for the month.
Automatically merge data from several sources.
Send reminders of GTD items.
Send reminders of late TODO items.
Automated builds
Automated testing
Administration
These automate some repetitive server administration tasks:
Summarize server logs, remove regular items and send the rest by email
Rebuild indexes in a database
Take automatic backups
Meta-programming is a great thing. If you easily get access to the data about the class structure then you can automate a few things. In the high level language I use, I define a class like 'Property' for example. Add an integer for street number, a string for street name and a reference to the owning debtor. I then auto generate a form that has a text box for street number and street name, a lookup box for the debtor reference and the code to save and load is all auto-generated. It knows that street number is an integer so its text box can only accept integers. If I declare a read only property it will also make sure the text box is read only.
There are software robots, but often you really don't see them. For example consider a robot that is used to package stuff. There is a person who monitors the robot in case of a failure. When the robot fails, the person shuts the robot down and fixes things. That person is like a programmer who operates IDE to compile, refactor etc. When errors occur, the programmer fixes the code and runs the compiler again.
Well compiling is not very robot like, but then there are software that compile your project automatically. Now that is more like a kind of a robot. That software robot also checks things in the code like is there enough comments and so on.
Then we have software that generates code according to our input. For example we can create forms in MS Access easily with Wizards. The wizards are not automatically producing new forms form after form after form, because we need every form to be different. But the form generator is a kind of robot-like tool that is operated.
Of course you could input the details of every form first and then run generate, but people like to see soon every form. Also the input mechanism is the form pretty much already, so you get what you create on the fly. Though with data transformation tools you can create descriptions of forms from a list of field names, generate the forms, and call that as using robots.
There are even whole books about automated software production, but the biggest problem is, that the automation of the process lasts longer then the process itself.
Mostly programmers give up on this, since they try to achive everything on one step, from manual programming, to automation.
Common automation in software production is done through IDEs, CodeGenerators and such, until now nearly no logic is automated.
I would appreciate any advance in this topic. Try to automate little tasks from the process, and connect those tasks afterwards. Going step by step.
I'm guessing that, just like just about every software developer on planet Earth, you want to write software that writes software by itself. Unfortunately, it's an idea that only works on paper. I mean, we have things like code generators, DSLs, transformation pipelines, Visual Studio add-ins that statically analyse code and generate derivative code, and so on. But it's nowhere near anything one would call a 'robot'.
Personally, I think more needs to be done in this area. For example, the IDE should be able to infer things and make suggestions based on what I'm actually doing. For example, if I'm adding a property, the IDE infers what attributes other properties in the file has, and how the property itself is structured, and adjusts the property accordingly.
Any sort of AI is hard work and, regrettably, does not have such a great ROI. But it sure if fun.
Scripting away the repetitive tasks - that's what you refer? I guess you're a Windows developer where scripting is not as nearly common as in *nix world. Hence your question.
You might want to have a look at the *nix side of software development arena where the workflow is more or less similar to what you describe (at least more than Windows). Plowing your way via bash, perl, python, etc.. will get you what you want.
ps. Also look at nsr81's post in comments for similar scripting tools on Windows.
Code generation is certainly a viable tool for some tasks. If done poorly it can create maintenance problems, but it doesn't have to be done poorly. See Code Generation Network for a fairly active community, with conference, papers, etc.
Code Generation in Action is one book that comes to mind.
You can try Robot framework
http://robotframework.org/
Robot Framework is a generic automation framework,It has easy-to-use tabular test data syntax and it utilizes the keyword-driven approach.
Even you can used this tools as software bot (RPA).
Robotic Process Automation
First, a little back-story... In 2011, I was the Operations Manager for Contracting Center of Excellence at Bristol-Myers Squibb. We were in the early stages of rolling out a brand new Global Contracting System. This new system was replacing a great deal of manual effort across the globe with the intention of one system to create, store and retrieve Contracting information for all of the organization. No small task to be sure, and one we certainly underestimated the scope and eventual impact of. Like most organizations getting a handle on this contract management process, we found it to be from 4 to 10 times larger than originally expected.
We did a lot of things very right, including the building of a support organization from the ground up, who specialized on this specific application and becoming true subject matter experts to the organization in (7) languages and most time zones.
The application, on the other hand, brought it's own challenges which included missing features, less than stellar performance and a lot of back-end work needing done by the Operations team. This is where the Robotics Process Automation comes into the picture.
Many of the 'features' of this software were simply too complicated for end users to use, but were required to create contracts. The first example was adding a "Contact" to whom the Contract would be made with. The "Third Party", if you will. This is a seemingly simple thing, which took (7) screens of data entry, a cryptic point of access, twenty two minutes and a masters degree to figure out, on your own for each one. We quickly made the business decision to have the Operations team create these 'Contacts' on behalf of our end users. We anticipated the need to be a few thousand a year. We very quickly passed 800 requests per week. With three FTE's working on it, we had a backlog ever growing and a turn-around time of more than two weeks per request. Obviously, this would NOT due in any business environment.
The manual process was so complicated, even my staff had a large number of errors in creating them, even as subject matter experts. The resulting re-work further complicated the issue and added costs. I had some previous Automation experience and products that I worked with, but this need was even more intense and complicated than I had encountered before. I needed something great, fast, easy to implement and that would NOT require IT assistance (as that had it's own pitfalls.) I investigated a number of products, all professing to do similar things. One of course, stood out to me. It seemed to be the most capable, affordable and had good support options. The product I selected was Automation Anywhere at the bargain price of about $4000.00 USD.
I am not here to pitch for Automation Anywhere, or any specific product, for that matter. But, my experiences with this tool, forever changed my expectations and understanding of what Robotic Process Automation really means.
Now, don't get me wrong, I am not here to pitch for Automation Anywhere, or any specific product, for that matter. But, my experiences with this tool, forever changed my expectations and understanding of what Robotic Process Automation really means. (see below, if you are unsure)
After my first week, buying the tool and learning some of the features, I was able to implement a replacement of the manual process of creating a "Contact" in the contracting system from a two week turn around, to a (1) hour turn-around. It took the FTE effort of 22 minutes for each entry, to zero. I was able to run this Automated process from a desktop PC and handle every request, fully automated, including the validation and confirmation steps into other external systems to ensure better data quality than was ever possible, previously. In the first week, my costs for the software were recovered by over 200% in saved labor, allowing those resources to focus on other higher value tasks. I don't care where you are from, that is an amazing ROI!
That was just the beginning, now that we had this tool, and in fact it could do much more than this initial task I needed, it became one of the most valued resources for developing functional Proof of Concept/prototypes of more complex processes we needed to bridge the gaps in the contracting system. I was able to add on to the original purchase with an Enterprise License and secure a more robust infrastructure partnering with our IT department at a an insanely low cost for total implementation. I now had (5) dedicated Corporate servers operating 24/7 and (2) development licenses for building and supporting automation tasks and we were able to continue to support the Contracting initiative, even with the volume so much greater than anticipated with the same number of FTEs as we started with. It became the platform for reporting, end user notification, system alerts, updating data, work-flow, job scheduling, monitoring, ETL and even data entry and migration from other systems. The cost avoidance because of implementing this Robotic Process Automation tool can not be over stated. The soft-dollar savings from delivering timely solutions to the business community and the continued professional integrity we were able to demonstrate and promote is evident in the successful implementation to more than 48 countries in under (1) year and the entry of over 120,000 Contracts entered each year since.
It became the platform for reporting, end user notification, system alerts, updating data, work-flow, job scheduling, monitoring, ETL and even data entry and migration from other systems.
While the term, Robotic Process Automation is currently all the buzz, the concepts have been around for some time. Please, please however, don't make the assumption that this means it is a build and forget situation. As it grows, and it will grow, you need a strong plan to manage tasks, resources and infrastructure to keep things running. These tools basically mimic anything a human can do, and much more than a human as well. However, a human can rather quickly change their steps in a process if one of the 'source' systems she/he is using has a change in the user interface. Your Automation Tasks will need 'tweaked' to make that change in most cases. Some business processes can be easier than others to Automate and might be two complex for a casual "Automation task creator" to build and or maintain. Be very sure you have solid resources to build and maintain the tasks. If you plan to do more than one thing with your RPA tool, make sure to have solid oversight, governance, resources and a corporate 'champion' or I assure you, your efforts will not be successful.
Robotic Process Automation Defined:
(IRPA) Institute for Robotic Process Automation: “Robotic process automation (RPA) is the application of technology that allows employees in a company to configure computer software or a “robot” to capture and interpret existing applications for processing a transaction, manipulating data, triggering responses and communicating with other digital systems.”
Wikipedia: “Examples of robotic automation include the use of industrial robots in manufacturing and the use of software robots in automating clerical processes in services industries. In the latter case, the use of the term robot is metaphorical, conveying the similarity of those software products – which are produced to provide a generic automation capability and then configured within the end user environment to execute manual and repetitive tasks – to their industrial robot counterparts. The metaphor is apt in the sense that the software “robot” is now mimicking or replacing a function classically associated with a person.”