Here is the scenario.
We are developing a product where we have a base product and regional variations for the product. We have all the common code checked into the main trunk while we have created 2 branches (branch_us, branch_uk) for the variations off of the main trunk. There is common code that is constantly being checked into the main trunk and the code that is being checked into branch_uk,branch_us is dependent on the code that is checked into the main trunk. This is being done because we expect more regions to added in future releases and as a result we want to have max reuse as well as thin regional variations layer.
Based on the current strategy, the developer will have to develop locally and then manually check-in the common files into main_trunk and regional variations into branch_uk & branch_us. Then everytime code is checked into the main_trunk, we will have to perform a merge from main_trunk->branch_uk & main_trunk->branch_us before we can perform a build for branch_uk & branch_uk (two separate deployments) because of dependency of new code in branch_uk/us branch to the new common code in main_trunk. This model seems extremely painful to think about and unproductive.
I'm by no means an expert on TFS. Here is what I am seeking opinion on:
Is there a way TFS can dynamically pull changes into branch_uk/branch_us from the main_trunk without doing a manual merge after every check-in (in the main_trunk)?
Do you guys have any other recommendations on the code management process that might be more effective/productive than the current one?
Any thoughts and feedback will be much appreciated!
This seems like a weird architecture to me, but of course I'm coming at it from a position of almost total ignorance, so there might be a compelling reason to approach it that way.
That being said: It sounds to me like you don't have a single application with two regional variations, you have two separate applications that share a common ancestor. The short answer to your question is "No". A slightly longer answer is "No, but you could write code to automate it."
A more thoughtful question-answer is "Are you sure centralized version control is the right tool for the job?" It might be more intuitive to use Git for this. What you have are, in effect, a base repository and two forks of that repository. Developers can work against whatever fork makes sense, and if something represents a change that should apply to all localizations, open a pull request to have the change merged into the base repository. This would require more discipline on the part of the developers, since they would have to ensure that their commits are isolated such that they can open a pull request that contains just commits that apply to the core platform. Git has powerful but difficult history-rewriting tools that can assist. Or, of course, they could just switch back and forth between working on the core platform, then pulling changes from the core platform back up to the separate repositories. This puts you back to where you started, but Git merges are very fast and shouldn't be a big issue.
Either way, thinking of the localizations are a single application is your mistake.
A non-source control answer might involve changing the application's architecture so that all localizations run off of the same codebase, but with locale-specific functionality expressed in a combination of configuration flags and runtime-discoverable MEF plugins, or making a "core" application platform that runs as an isolated service, and separately developed locale-specific services that express only deviations from the core application platform.
I am following verification and validation threads and I think an example might be helpful. I am not an experienced developer so I would like to know whether this would be correct:
User equirement: I want to be save my friend's name, address and phone number to the system
Software Requirement specification: User wants to be able to enter and save a name, address, phone number.
Technical analysis: web UI for data entering. Data will be saved into the SQL DB.
Detailed design: UI elements: 3 fields of a string type, 1 button, object XYZ, dbConnection....
Code: (actual code of UI, db scripts)
Is it like that? Could anyone correct or add what I am missing here?
As for verification, each phase can be verified against requirement (traceability). As for validation, the functional code should work as expected (save three attributes).
While this is some what theoretically true (I have to say this), it is completely wrong in all practical and real world scenarios.
Capture user needs and Why he wants to do a certain thing. This allows you to build just the software that user wants, eliminate waste that come as part of made-up requirements, technical requirements, nice to haves etc.
So instead of,
I want to be save my friend's name, address and phone number to the system...
I'd rather like to have the below which emphasizes Why? the real need of the user
I want to send a greeting card to my friend on his birthday.
Now, I know I just need his name and address. Since this is for future I also want to store this information. So what I write next is a set of acceptance criteria to meet the above customer needs. If I can capture these as a set of executable specifications then it is even better as those are verifiable programmatically.
Ignore everything else. Traceability is unnecessary overhead. We need it if we are building software based on fabricated requirements.
Read the below
Agile Manifesto
ATDD and BDD
Impact Mapping
I've never seen a good way to trace code to requirements outside a single sprint/time box. And also, you're missing testers from your list! Unless your testers are also your business analysts (I my experience professional testers find a lot of the requirements inconsistencies - aka bugs).
I think the best approach is to have everyone as involved as possible, so you can cross reference each person expectations often. If everyone works together, you don't need to implement a cargo-cult process where batches of information are transferred down stream in one way.
The simplest tool have traceability is your VCS, where each commit includes the ID of the user story/use case that the commit is related to.
I manage a small team of developers who up till recently have all been working on independent projects.
We have now all come together to work on one holistic project and its really tough. People are changing things, without consulting other programmers and its very difficult to manage.
We are also working in a pure production environment (no test/production)
I am a bit lost on where to start, I am looking at products like Team Foundation Server or their is a product by Red gate called "Source control"
Does anyone know any good books on the above subject, I cannot be the only person in this situation? :)
Source Code:
If you're looking for how to use source control (versus how to get your team to a controlled source environment) you should check out Source Control HOWTO by Eric Sink. The disclaimer here is that he has written a source control product but the HOWTO isn't specifically tailored to it.
Database Schema:
If you're looking for how to get your database (structure not data) in source control (since it was mentioned in the title) then you should check out this Coding Horror post: Get your database under version control. It covers some basic concepts and points to a few articles on how to actually implement getting your database in source control.
Source Control Software:
If you are looking for actual source control software, a list of applications can be found here: List of Source Control Systems with Visual Studio Plugins.
Don't make the mistake I did when putting the database definition under source code control by allowing branches and merges. This led developers (and managers) to assume they could have several versions of a table in the same database, leading to programmers overwriting each others changes, and having incompatible versions of source code and table definition all over the place.
As you might well imagine this lead to a great deal of wasted time and many frayed tempers.
Depending if time or money is more important, do you have the option to being someone in for a short time with the specific task of improving your processes? They would be able to set you up a test environment, install a version-control server and get your guys (and you) up to speed.
The first thing that you need to do is to give every developer their own copy of the database.
Secondly, you need each dev to be able re-create a baseline database from the master.
Once each dev is happy with their changes to the database, you will then need them to create a series of patches for their changes.
Lastly, each one of these patches needs to be able to checked in to source-control, and re-distributed to the other devs.
If you are looking for a good source control server, consider Subversion, with Tortoise SVN as your client.
Take a look at DBSourceTools. (http://dbsourcetools.codeplex.com).
It's specifically designed to help developers use source control for database changes.
This tool will allow you to baseline your database at a specific point, and create a named version (v1).
All files are scripted to disk, and very easy to add to a Subversion repository.
Have a look at the patching process for creating and using patches.
Have fun.
I have a Mailman managed list with years of history that I want to migrate into a web-based forum. Things it would be nice to do:
Keep the mailing list going for those who are used to and prefer it to a web interface, but have it integrate with the web-forum activity.
Take the web-based forum posts, and send them out to the mailing list.
I have run sites based on phpBB, Drupal, Simple Machines, etc., and am able to do a little bit of coding if that was what was required to integrate some package into Mailman, or replace it entirely. But I'm unaware of what is available, commercial, or open-source, that could accomplish this. I am also open to replacing Mailman, if the candidate package can perform its functions reasonably well and integrate well into its own forum.
Even if the phpBB/mail2forum option is using the older version, it is a nice option. OpenSceneGraph just implemented this for their long running mailing list, and it seems to work flawlessly. It's the only option I've seen that allows for forum posts and mailing list messages to work very seamlessly, and has the means to allow for tags to separate forum subjects from mailing list posts, etc.
Their v2 dev works with phpBB3, too - so if you're willing to live on the bleeding edge, that might be an option.
So far, here's what I've looked into:
Drupal, with its Mailmanager and Listmanager modules, though I've been having trouble turning on the IMAP feature so it can talk to my mail box (clients can access it fine), and the Drupal's forum module isn't really up to the feature level I'd like.
PhpBB, mail2forum, though it looks like currently it only works with the older phpBB version, so not really an alternative for me.
The only one that seems to really "work" is FUDforum and its maillist.php module, which integrates directly at the procmail level, nice. I can take all my Mailman archives and "formmail -ds" them directly into a chosen forum.
I've started a bounty for this issue, I'm looking for something, even commercial, that really integrates the email interface into the forum experience for the end users, particularly the handling of accounts. With FUDforum I'm still going to have to resolve the separation of the Mailman accounts from the FUDforum accounts.
I ran across this on the web: http://mail2forum.com/
I haven't used it but it looks promising and has both a 1.2 stable version and a 2.0 development version, so it's not a derelict SourceForge project with 1 developer and no commits, or anything :)
Theoretically my org may use it at some point in the next year or two, but we have to finish our own Listserv to Mailman transition first.
Well, based on what you've said (first, that you're able to do a bit of coding and second that FUDForum would work for you except for the issue of maintaining list membership, have you considered that:
Mailman stores its user information in a plain text file and
FUDForums stores its user information in a single sql table
The obvious solution would be to declare one of these master (probably FUDForum, since it looks as if it holds a proper superset of the Mailman info) and have a little script/cron job that copied changes from the master to the slave.
A passing note -- neither of these systems appears very secure (actually, they both look leaky as all get out) and combining them may well reduce the collective security even further. If you are doing anything even remotely confidential you should rethink your goals, and in any case you should take appropriate steps to protect your system from attack.
I know an SME which switched from the mailing list to phpBB!
If you want to keep your mailing list, you have also to maintrain it.
Finally the people I know diabled the mailing list service.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
I have being toying with the idea of creating software “Robots” to help on different areas of the development process, repetitive task, automatable task, etc.
I have quite a few ideas where to begin.
My problem is that I work mostly alone, as a freelancer, and work tends to pill up, and I don’t like to extend or “blow” deadline dates.
I have investigated and use quite a few productivity tools. I have investigated CodeGeneration and I am projecting a tool to generate portions of code. I use codeReuse techniques. Etc.
Any one as toughs about this ? as there any good articles.
I wouldn't like to use code generation, but I have developed many tools to help me do many of the repetitive tasks.
Some of these could do nice things:
Email Robots
These receive emails and do a lot of stuff with them, they need to have some king of authentication to protect you from the bad stuff :
Automatically logs whatever was entered in a database or excel spreadsheet.
Updates something in a database.
Saves all the attachments in a specific shared folder.
Reboot a server.
Productivity
These will do repetitious tasks:
Print out all the invoices for the month.
Automatically merge data from several sources.
Send reminders of GTD items.
Send reminders of late TODO items.
Automated builds
Automated testing
Administration
These automate some repetitive server administration tasks:
Summarize server logs, remove regular items and send the rest by email
Rebuild indexes in a database
Take automatic backups
Meta-programming is a great thing. If you easily get access to the data about the class structure then you can automate a few things. In the high level language I use, I define a class like 'Property' for example. Add an integer for street number, a string for street name and a reference to the owning debtor. I then auto generate a form that has a text box for street number and street name, a lookup box for the debtor reference and the code to save and load is all auto-generated. It knows that street number is an integer so its text box can only accept integers. If I declare a read only property it will also make sure the text box is read only.
There are software robots, but often you really don't see them. For example consider a robot that is used to package stuff. There is a person who monitors the robot in case of a failure. When the robot fails, the person shuts the robot down and fixes things. That person is like a programmer who operates IDE to compile, refactor etc. When errors occur, the programmer fixes the code and runs the compiler again.
Well compiling is not very robot like, but then there are software that compile your project automatically. Now that is more like a kind of a robot. That software robot also checks things in the code like is there enough comments and so on.
Then we have software that generates code according to our input. For example we can create forms in MS Access easily with Wizards. The wizards are not automatically producing new forms form after form after form, because we need every form to be different. But the form generator is a kind of robot-like tool that is operated.
Of course you could input the details of every form first and then run generate, but people like to see soon every form. Also the input mechanism is the form pretty much already, so you get what you create on the fly. Though with data transformation tools you can create descriptions of forms from a list of field names, generate the forms, and call that as using robots.
There are even whole books about automated software production, but the biggest problem is, that the automation of the process lasts longer then the process itself.
Mostly programmers give up on this, since they try to achive everything on one step, from manual programming, to automation.
Common automation in software production is done through IDEs, CodeGenerators and such, until now nearly no logic is automated.
I would appreciate any advance in this topic. Try to automate little tasks from the process, and connect those tasks afterwards. Going step by step.
I'm guessing that, just like just about every software developer on planet Earth, you want to write software that writes software by itself. Unfortunately, it's an idea that only works on paper. I mean, we have things like code generators, DSLs, transformation pipelines, Visual Studio add-ins that statically analyse code and generate derivative code, and so on. But it's nowhere near anything one would call a 'robot'.
Personally, I think more needs to be done in this area. For example, the IDE should be able to infer things and make suggestions based on what I'm actually doing. For example, if I'm adding a property, the IDE infers what attributes other properties in the file has, and how the property itself is structured, and adjusts the property accordingly.
Any sort of AI is hard work and, regrettably, does not have such a great ROI. But it sure if fun.
Scripting away the repetitive tasks - that's what you refer? I guess you're a Windows developer where scripting is not as nearly common as in *nix world. Hence your question.
You might want to have a look at the *nix side of software development arena where the workflow is more or less similar to what you describe (at least more than Windows). Plowing your way via bash, perl, python, etc.. will get you what you want.
ps. Also look at nsr81's post in comments for similar scripting tools on Windows.
Code generation is certainly a viable tool for some tasks. If done poorly it can create maintenance problems, but it doesn't have to be done poorly. See Code Generation Network for a fairly active community, with conference, papers, etc.
Code Generation in Action is one book that comes to mind.
You can try Robot framework
http://robotframework.org/
Robot Framework is a generic automation framework,It has easy-to-use tabular test data syntax and it utilizes the keyword-driven approach.
Even you can used this tools as software bot (RPA).
Robotic Process Automation
First, a little back-story... In 2011, I was the Operations Manager for Contracting Center of Excellence at Bristol-Myers Squibb. We were in the early stages of rolling out a brand new Global Contracting System. This new system was replacing a great deal of manual effort across the globe with the intention of one system to create, store and retrieve Contracting information for all of the organization. No small task to be sure, and one we certainly underestimated the scope and eventual impact of. Like most organizations getting a handle on this contract management process, we found it to be from 4 to 10 times larger than originally expected.
We did a lot of things very right, including the building of a support organization from the ground up, who specialized on this specific application and becoming true subject matter experts to the organization in (7) languages and most time zones.
The application, on the other hand, brought it's own challenges which included missing features, less than stellar performance and a lot of back-end work needing done by the Operations team. This is where the Robotics Process Automation comes into the picture.
Many of the 'features' of this software were simply too complicated for end users to use, but were required to create contracts. The first example was adding a "Contact" to whom the Contract would be made with. The "Third Party", if you will. This is a seemingly simple thing, which took (7) screens of data entry, a cryptic point of access, twenty two minutes and a masters degree to figure out, on your own for each one. We quickly made the business decision to have the Operations team create these 'Contacts' on behalf of our end users. We anticipated the need to be a few thousand a year. We very quickly passed 800 requests per week. With three FTE's working on it, we had a backlog ever growing and a turn-around time of more than two weeks per request. Obviously, this would NOT due in any business environment.
The manual process was so complicated, even my staff had a large number of errors in creating them, even as subject matter experts. The resulting re-work further complicated the issue and added costs. I had some previous Automation experience and products that I worked with, but this need was even more intense and complicated than I had encountered before. I needed something great, fast, easy to implement and that would NOT require IT assistance (as that had it's own pitfalls.) I investigated a number of products, all professing to do similar things. One of course, stood out to me. It seemed to be the most capable, affordable and had good support options. The product I selected was Automation Anywhere at the bargain price of about $4000.00 USD.
I am not here to pitch for Automation Anywhere, or any specific product, for that matter. But, my experiences with this tool, forever changed my expectations and understanding of what Robotic Process Automation really means.
Now, don't get me wrong, I am not here to pitch for Automation Anywhere, or any specific product, for that matter. But, my experiences with this tool, forever changed my expectations and understanding of what Robotic Process Automation really means. (see below, if you are unsure)
After my first week, buying the tool and learning some of the features, I was able to implement a replacement of the manual process of creating a "Contact" in the contracting system from a two week turn around, to a (1) hour turn-around. It took the FTE effort of 22 minutes for each entry, to zero. I was able to run this Automated process from a desktop PC and handle every request, fully automated, including the validation and confirmation steps into other external systems to ensure better data quality than was ever possible, previously. In the first week, my costs for the software were recovered by over 200% in saved labor, allowing those resources to focus on other higher value tasks. I don't care where you are from, that is an amazing ROI!
That was just the beginning, now that we had this tool, and in fact it could do much more than this initial task I needed, it became one of the most valued resources for developing functional Proof of Concept/prototypes of more complex processes we needed to bridge the gaps in the contracting system. I was able to add on to the original purchase with an Enterprise License and secure a more robust infrastructure partnering with our IT department at a an insanely low cost for total implementation. I now had (5) dedicated Corporate servers operating 24/7 and (2) development licenses for building and supporting automation tasks and we were able to continue to support the Contracting initiative, even with the volume so much greater than anticipated with the same number of FTEs as we started with. It became the platform for reporting, end user notification, system alerts, updating data, work-flow, job scheduling, monitoring, ETL and even data entry and migration from other systems. The cost avoidance because of implementing this Robotic Process Automation tool can not be over stated. The soft-dollar savings from delivering timely solutions to the business community and the continued professional integrity we were able to demonstrate and promote is evident in the successful implementation to more than 48 countries in under (1) year and the entry of over 120,000 Contracts entered each year since.
It became the platform for reporting, end user notification, system alerts, updating data, work-flow, job scheduling, monitoring, ETL and even data entry and migration from other systems.
While the term, Robotic Process Automation is currently all the buzz, the concepts have been around for some time. Please, please however, don't make the assumption that this means it is a build and forget situation. As it grows, and it will grow, you need a strong plan to manage tasks, resources and infrastructure to keep things running. These tools basically mimic anything a human can do, and much more than a human as well. However, a human can rather quickly change their steps in a process if one of the 'source' systems she/he is using has a change in the user interface. Your Automation Tasks will need 'tweaked' to make that change in most cases. Some business processes can be easier than others to Automate and might be two complex for a casual "Automation task creator" to build and or maintain. Be very sure you have solid resources to build and maintain the tasks. If you plan to do more than one thing with your RPA tool, make sure to have solid oversight, governance, resources and a corporate 'champion' or I assure you, your efforts will not be successful.
Robotic Process Automation Defined:
(IRPA) Institute for Robotic Process Automation: “Robotic process automation (RPA) is the application of technology that allows employees in a company to configure computer software or a “robot” to capture and interpret existing applications for processing a transaction, manipulating data, triggering responses and communicating with other digital systems.”
Wikipedia: “Examples of robotic automation include the use of industrial robots in manufacturing and the use of software robots in automating clerical processes in services industries. In the latter case, the use of the term robot is metaphorical, conveying the similarity of those software products – which are produced to provide a generic automation capability and then configured within the end user environment to execute manual and repetitive tasks – to their industrial robot counterparts. The metaphor is apt in the sense that the software “robot” is now mimicking or replacing a function classically associated with a person.”