Why Consumer-driven Contract Test is not working? - testing

Why CDCT is not working for most cases in real life? The concept and tools have been selled by architects for quite a few years especially in micro-service architect, or in multi-modules complex system, there are a lot of pains for integration testings, but why CDCC is not implemented everywhere?

I heard the concept and tools about CDCT (consumer-driven contract test) about three years ago, I used to do some research in our enterprise software (one of the most complex SaaS softwares in the world, 15 years-old, developed by more than a thousand engineers) and discuss it with our chief arch about two years ago. It looks promising that we are supposed to be able to find a real case to implement it via a proper tool like pact, between two proper teams who have pain point so does the motivation, why not? The concept absolutely makes a lot of sense, the problem it aimed to solve is a very common one (who doesn't have an integration broken by another team?), everything looks perfect and I even added into my yearly goal.
I failed, I was young and simple, it didn't work out, hopeless.
Today I heard a same failure from another team, and no surprise they have same reason that’s why I think it might be write it down to as a reminder and useful (probably) knowledge to share.
The reason is high adoption cost including mindset change. CDCT is not a tool (you can use a tool like pact to better implement it), it's not even a methodology only, it's a new mindset to tell people how to work together.
Yes it’s aimed to solve the problem between multiple systems/modules, but it is more to create a new mindset which needs the two groups of people to accept: firstly a contract is needed (vesus no contract is needed), secondly consumer is the driver of the contract (vesus provider is the driver of integration).
Here is the tricky part, from consumer perspective, what needs to be done for integration point(s):
Before CDCT: 1. find an API and use it. 2. when it breaks, blame provider
After CDCT: 1. find an API 2. drive: find provider, meet with provider, negotiate with provider, come up a contract, repeat this if there is gap, signoff the contract and save it. 3. Write testing, ask provider to review the testing, ask provider to put your testing into their pipeline. Figure out how to make sure provider always make your testing pass rather than comment them out before they release a new version of service.
I can understand why consumer may not really want this, or why they want the result but hesitate to pay the cost first.
So when CDCT implementation will be successful? I think there might be two conditions:
The consumer's business is too important to be broken (say accounting), they have no choice but do everything can safeguard the dependency. However, in such a case the better idea is to remove the dependency, or adding duplication and fail-over mechinsim, testing is still the last choice.
The provider and consumer are working very closely so the mindset and setup cost will be mininum, unfortunately contract testing might not be needed in this case, because the teams are working very closely.
Regards,
Emil

Related

Is Contract testing necessary when both consumer and provider are developed by the same company in different scrum teams?

Is Contract testing necessary when both consumer and provider are developed by the same company in different scrum teams ?
Yes, definitely. Contract testing is particularly useful when you rely on an 'external' service, where by external I mean any service that is not under your direct control, including the case you mentioned. Here is an interesting article from Martin Fowler.
Short answer: no, contract testing isn't necessary in any situation, the same as unit testing.
Long answer: Not having testing greatly reduces your confidence as a developer to deploy without breaking anything. Unit testing is good for testing an individual function, while contract testing is good at figuring if your changes will affect any consumers of the data you provide. The consumers of your data could be anyone, it could be someone across the room from you, a client external of the company or even yourself. The whole point is to try to segment and simplify the development process so that problems are caught earlier on. It also has the added benefit that you don't need to run the data producer locally just to have the consumer working while developing, which is definitely a great bonus when the consumer doesn't (or can't) have access provider code, like an external client.
These tools are meant to make your life as a developer simpler and easier to manage, Pact strives to accomplish this in your workflow and to prevent issues from happening in production and giving the developer a quicker feedback loop of potential issues.
The team that wrote Pact in the first place was responsible for both ends of the integration, and they still found contract testing valuable. Just because you're developing both sides now, doesn't mean that you will continue to be responsible for both sides in the future. Contract tests will ensure that changes made by future developers will not break anything.

Integrating my RESTful web app with clients' SAP installations

My company runs a couple of B2B apps (written in Rails) dealing with parts and inventory and we've been trying to figure out the best way to integrate with some of our bigger users. We already offer the REST-style API that comes with Rails, but that, of course requires an IT Department on their end to decide to integrate it, so we'd like to lower that barrier if possible.
From what we've found, most of them are on SAP systems. Now, pretty much all I know about SAP is it's 1) expensive, 2) huge, 3) and does everything and anything you could ever need for your gigantic business to run. Naturally, this is all a bit imposing, and the resources on the site are a cross between impenetrable buzz-word laden sales material, and impenetrable jargon laden advanced technical material with little for the new, but technically competent user to be able to sink his teeth into.
So what I'm wondering is: as a 3rd party, that's not running a SAP installation, is there a way for us to offer access to our site's data through a web service or other API? Is it just a matter of providing or implementing a certain WSDL (and what would that be)? Is this feasible for someone without in-depth experience with SAP? Or is this a complete non-starter?
I'd say it's not possible without someone who knows the SAP system. You probably won't need to hire someone with in-depth SAP knowledge, but at least for the initial implementation, you'll need both the knowledge and a working system you can develop against. Technically speaking, it's not really that hard, but considering the fact that SAP systems are designed to handle multiple organizations, countries, legal systems, localizations and several thousands of users simultaneously, things are bound to be a bit more complex than almost any other software around - and most of the time not even bloated, it's just easy to get lost in that kind of flexibility.
My recommendation would be to find a customer (or a prospective customer) who has someone in their IT department with the necessary technical and processual knowledge and who is interested in conducting a development project. This way, you'd get access to a real system (testing of course) and someone who can explain to you the basics of the system. But, as I said, be prepared for complexity.
vwegert makes some excellent points.
As to this part of your question:
So what I'm wondering is: as a 3rd
party, that's not running a SAP
installation, is there a way for us to
offer access to our site's data
through a web service or other API? Is
it just a matter of providing or
implementing a certain WSDL (and what
would that be)?
Technically it is possible to expose any of your system's services as web-services to a client's SAP system. In order to do this you do not need any prior knowledge of SAP. (SAP should be able to import a WSDL, although there may be some limitations in the earlier pre-ECC5 systems).
For example a service that provides meter reads, airport departure schedules, industry trends etc is not dependend of what is in the user's system or how they set it up. However as soon as there is a need to initiate updates to the client system's data is when you need access to more specialised SAP knowledge.
Also note that many SAP functions can also be exposed as web services, but generally you do need someone with SAP (ABAP) knowledge to do this.
The ABAP language is actually fairly simple, but there is a huge learning curve to understand the data model and the myriad of configurable options in SAP.

Is This Normal Development Procedure? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 months ago.
Improve this question
First a little about myself. I am not an experienced software engineer, architect or developer. I have done mostly small ASP and ASP.NET projects in C# for the last 5 years. I am pretty good with HTML and JavaScript. These projects were done when I had free time from my other duties which were not related to software development. I have now been moved into a software developer position. The company I work for is not a software development firm.
I am now working on a Silverlight LOB application with WCF and Entity Framework. I have been given little specifications for this project, just the 'make an application like X, only simpler so we don't have to pay for it', my boss doesn't check on my progress as often as I think he should, the project manager(a co-worker) will stop by now and then but we never discuss the specs, architecture, UI or business rules. I am mostly just asked when I think it will be done. I have had to learn Silverlight, WCF and Entity Framework to work on this project which is not a problem as I really enjoy working with these technologies. The problem is I am the only one in the company that knows anything about these and have no mentor/boss to discuss the problems and how they could be solved. I have been able to seek out one interested party in the company that has at least given me a list of some of the requirements.
I can't believe this is how software development should be done. I think the project managers should offer guidance and keep a closer eye on what is being done to prevent going in the wrong direction(but how can they in my situation since the don't know the technologies!).
Should I feel this way or am I way off base?
Thanks for listening.
What you describe is certainty not optimal, but it's extremely common, particularly in smaller shops. Some people find it rewarding to work in that kind of environment. It's not what the software engineering books teach, but that's why there are so many software engineering books.
If you want to continue working in this environment, you're going to have to supply all the discipline you rightly recognize as missing yourself. Write up a spec. Build a schedule. Share these with your management. Hold yourself to deadlines.
Share your concerns with your management; don't be shy about that. Chances are, they recognize the situation. Your boss doesn't check your progress? Publish your progress to him. Show him where you need to get to, how far along you are, and what's blocking you.
It'll be chaotic, no doubt, but you'll learn a lot.
Every organization is different. If they are operating in this capacity then you should adapt and make the best of the situation. It's either happening because that's how things are done and they are aware of it, or they don't know the wiser or don't want to invest to improve the process of delivering strategic/tactical projects.
In a perfect world everyone would have a robust Quality Methodology in place which would provide a framework for Project delivery and systems implementation. It's just not a reality.
Here are some tips to help you operate more effectively:
Identify your sponsors (the people who own the product) and determine the high level benefits and driving objectives of the business problem they seek to solve
Identify your stakeholders (who has influence and who has interest) and get them to communicate their needs as much as possible
Involve both sponsors and stakeholders in the process as much as possible or as much as they want
Capture what requirements you can from them through written form (email)
Provide opportunities for them to gain visibility into the delivery and to provide feedback
Your project will likely fail from your boss point of view. Because i'm sure you developing program not suitable for him. But you don't feel guilty. It's your boss' pain.('because you are good programmer). Sorry for so dark post :-).
The role of the project manager is not to know the technology, but they definitely should have a finger on the pulse of the project, so to speak. The real project management job is not to control the project, but rather to enable it. Either way, from your description, looks like yours isn't doing such a great job at it.
The other extreme is a process-heavy organization where meetings and committees decide everything, and all the real communication, if it exists at all, happens through side channels.
The ideal world lies somewhere in between.
Your project manager should not be too concerned with how you're doing things. Since they have no qualifications, the best they can do is connect you with someone who does. When they can't verify that you're building the thing right, they should at the very least ensure you're building the right thing. Even if it's for internal use, you still have a customer, and no communication with the customer spells bad news to me. :)
If your PM is not concerned about the issue, you could try to do something yourself. For example, ask the PM to connect you with a would-be end user of the application. Extract bits of your application and give them to the user to play with -- just make sure the bits you give them don't look or feel too finished.
If you can't change things, take this as a learning experience. Make sure next time you're up for a project, you know the things that went wrong last time, and try to mitigate them from the start.
And finally, if your bosses tell you this is a "more agile way" of working, punch them in the face. Agile is, or should be, synonymous with discipline, not complete lack thereof.
Good luck!
It is a hard situation. Only you can really determine the best way to proceed. However, I do think that the concern with the schedule and concurrent lack of documentation (requirements, expectations, use-case scenario documentation, etc) is a train-wreck waiting to happen. Even the sharpest and most experienced dev-teams suffer from the same problems.
The "when will it be done?" questions are best mitigated by regularly providing small partially functional builds that you can use to get useful information out of the moving target that is your customer. It is amazing how much communication can occur when somebody (your boss/customer/end-user) can actually "play with" something in front of them and reconsider what they really want.
I believe this situation is quite common. I had this, too, at my previous job. Here the bet is on the fact that you are already independent and well-versed in your business. I think you should tell your manager how you feel about this.
They should change something after hearing your opinion about this situation. Because if you do something wrong and the manager does not notice it, the company can lose a lot of money and time.
But it’s also not worth constantly waiting for someone to guide you and check your work. In any case, your workflow should have self-management.

How to get your clients to test

I build web apps for a living.
An important but often painful process is client/user acceptance testing.
How do you manage this process?
i.e. How do you get them to test? Do you give them test scripts?
Do you give them a system to log bugs and change requests/feedback. How do you get the client to understand the difference between a bug and a feature change?
How go get clients to give you repeatable steps to create a bug/issue?
Any good web apps for managing this process (thinking a Basecamp like app would be very uesful for this)
Thanks,
Ed
Don't give them test scripts.
To me that invalidates the testing process to a large degree because if you're thinking up test cases your software probably handles them because you've thought of them.
The idea of good testing is that there is a level of independence in testing so you can't cater for known test cases and also the client is likely to think of scenarios that you won't, which is the whole idea.
But how do you motivate them? Well, honestly I'd be surprised if they weren't motivated. I've generally found that motivating them to comment on func specs, requirements and other preliminary documentation is a far tougher battle. By the time you get to testing, you've eliminated an important psychological hurdle in that the software is now "real".
How you handle this depends to a large extent on the nature of your relationship to the client. If you have a formal process with an agreed upon spec, you should really be saying that the client has a certain period to sign off and accept the software and inaction is implied acceptance.
If it's an internal client well then that's harder. It probably all comes down to who's driving the project? Who are the stakeholders? These are the people you need to motivate such activity.
Usually the best method that I've come across for client testing is having them send screenshots of the problem and some of the things they did to create it. By this point, most of the testing should have been done in house and the egregious bugs should be weeded out. Having a system that automatically emails that an error occurs lets me know they are testing and I get most of the gory details from the stacktrace in the email.

As a programmer how much are you expected to know outside of programming? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
I'm wondering what you do as a programmer that's not programming but necessary for your task (eg: local setup, server setup, deployment, etc). I'm curious to know how many non-programming related tasks people are performing.
For example, when on web development projects I often:
Install servers
Manage user right/access to servers
Perform backups
Configure IIS/Apache
Setup FTP sites
On non-web projects I often:
Write build scripts
Setup source code management tools/procedures
Probably more stuff I'm not thinking of
Some tasks are more related to programming than others (such as writing build scripts) but others fall outside of my area of expertise (domain setup comes to mind). Just interested to know how many people perform tasks in their jobs that are not programming related.
The sad reality is that non-technical people look at technical people and expect them to know everything that is technology related, not understanding that there are specializations within technology which we might know nothing about.
I often think it is very much like a doctor that specializes in a particular discipline. All doctors have a baseline of knowledge in the medical field, but will not know the specifics of other specializations (a cardiologist will not know as much about anesthesiology and vice versa).
So while I think it is unreasonable for people to expect technologists to know everything, I do think that it is reasonable for them to expect that we know something when it comes to technology.
I think a more important facet of this question is how much one is expected to know about the specific domain where they apply their skills (finance, manufacturing, etc, etc). I think that is incredibly important, as having that domain knowledge makes them much more valuable as a programmer, as they can understand the problems on a deep level, and as a result, provide more comprehensive solutions for them.
Expected? Almost nothing, but everyone's always really happy when you know more.
The more you know outside the narrow confines of programming, the more valuable you are to your employer.
Things that have come up for me:
requirements gathering
writing use cases
evaluating test plans
negotiating with vendors
tax law
revenue recognition rules
ideas about how users behave
basic economic theory
usability guidelines
differences in consumer behavior in different countries
system administration (being a full on sysadmin)
database configuration, optimization, setup (basically being a DBA)
monitoring systems
networking principles and techniques (you'd be amazed how handy a packet trace can be when debugging something...)
being able to evaluate a business plan written by someone else
image manipulation
how to diffuse a situation and avoid arguments
how to corner someone and make them to commit to something when they don't want to
how to choose battles
I think the non-programming skill I use the most in my programming job is writing. It's really crucial to be able to explain ideas, designs, algorithms, and so on, and you can never count on being around to do it in person (or having the time). I spend a good amount of time at work writing up design documents and other documentation so other engineers can get their heads around my code and algorithms. So I'm really thankful that I had good writing classes in school and can put a sentence together. :-)
Probably depends on the size of the company you work for. As someone who has worked mainly at small to medium sized businesses, I've also been responsible for:
database creation, management, and tuning
supporting the internal applications I launch
managing website certificates
setting up external hosting
and I'm sure there's more as well
Well, since a programmer's primary tool is his computer, I think it's fair to assume some expertise with it. Most of those sorts of things you've described are difficult for someone unfamiliar with computers, but pretty easy (even with little prior experience) for someone who understands the domain and knows how to find and read documentation.
In a big, well-organized business or project, I'd expect someone who was more specifically familiar with those sort of administrative things to take care of them. However, if there's not enough of them to warrant a full-time job, then I don't think it's unreasonable to have anyone competent work on it; and programmers are probably at the head of the queue in that regard.
I find the vast majority of "bugs" discovered by users are configuration problems with the systems on which the application is installed. Having developers that understand the common machine and network setup errors is very desirable.
For example if an application sends email as part of its operation its useful to have developers knowledgable in DNS and SMTP configuration.
Of course it depends on your size of business, large organisations can probably shield developers from this by using other specialists.
I realized I'm never hired for the actual job, but as a problem solver. Whether I figure out what's going on, and fix it through code, or software, or something on the network, this seems to be the main perception of what clients want.
This will vary greatly depending on where you are. I've worked with people who know none of this stuff, and people who are experts.
Knowing this will help you greatly. In general it's always better to understand the environment your code is running in. Not understanding the context leaves you somewhat helpless.
Additionally there are often bugs that are not code related but configuration related, for example a page not showing up because of the apache configuration. You're very handicapped in debugging if you don't understand the environment.
People around a work place probably expect a programmer to be their IT HelpDesk guy... it happens around here to me. argh.
Where I work, all developers are expected to be able to use Subversion and have to be able to setup and configure Apache and Tomcat on their PC.
The biggest challenge is not the technical issues associated with getting the environment up and running but the domain knowledge required to effectively develop software in a small shop. For me, I work on a lot of different projects from a variety of sources in a mostly isolated development environment. This means that I need to come up to speed on the domain of the project pretty quickly in order to be effective in developing a solution. In the past I've worked on print accounting solutions, active directory management, research survey databases, and currently a quasi-CRM solution for a charitable organization. I wish I only had to know the nuts and bolts of setting up my development and build environment.
It often depends on the size of the company. In a little company, you have to know how to do everything, including systems admin, and network admin, even if your job is focused on programming.
In a big company, you get to see a little slice of the universe, and they often don't like you peeking outside of your box. Not only do you not need to learn everything, they're often unhappy with you if you try.
However, the more you understanding about the machines, how they work, and how they function in an operational environment, the easier it is to diagnose problems and write better software. The more you understanding about the domain you're writing applications for, the better you are able to differentiate between the users needs and their desires.
One of the coolest things about being a software developer is you have a life long excuse for sticking your nose into both the technologies and the various business domains. If you've shifted around to a few different industries, you tend to become loaded down with all sorts of interesting tidbits. There is always more to learn ...
Paul.
It's good to expose yourself to other technologies, but I really think it's a bad idea for you to not fully disclose the fact that you aren't experts in those areas (esp. domain setup). I've worked with people who thought they could do it all but ended up doing those tasks so poorly that with all the time (and money) they've spent trying to get it right, a consultant would have been paid for several times over.
I've worked at a company where I was responsible for everything "related to a computer" including the domain, PCs, database, custom software, builds, MS Office, PowerPoint, Quickbooks...; a mid-size company where it was development and builds; and a large company where I focus solely on the .Net code for my project (someone else handles the database and another handles reporting).
The mid-size company has been the best experience so far (pretty new at the large company) where I was given enough responsibility to feel useful and had easy access to everyone else to ask questions about those other tasks.
You are not alone out there. The position I signed up for was "ASP.NET Web Developer"... However, my job consists of:
Windows Server Administration
Limited Linux Administration (running
top to monitor CPU utilization and changing apache configs)
LDAP Administration / Tuning
MS SQL Server 2005 Administration /
Tuning
Database Development
Crystal Reports Developer
Perl Scripts
C# Win32 Developement
C# / ASP.NET Web Developement
Managing User Access Rights for
Windows Servers
Limited Network Troubleshooting
Being in a company that is constantly striving for supreme "Operation Effectiveness" my task list only grows by the day. I did not make up that list either. All of the items mentioned above, I have either touched or supported in the past 3 years I have worked in this company.
That being said, in a good development shop, you should have one specific task. As the saying goes, Jack of all trades ... master of none.
This depends greatly on what you're programming. If you're doing low level device drivers, it's vital that you understand the underlying hardware. If you're doing a standalone Java app, the better you understand the JVM and libraries you're using, the better - but it isn't strictly necessary to know a lot.
In general, the more you understand about your system environment, the better. How much your peers and management expect you to know depends on them.
Ignorance will, eventually, be punished. If not by your peers and management, the world will do it. Check any week's headlines or RISKS digest for examples where ignorance of the system environment cause software failure.
[rant mode on]
Ha, the curse of Excel and Word.
Outside work - particularly friends and family but sometimes when consulting or delivering software too, any and all non-technical people expect you to understand these. There's that internal groan when someone asks you across to have a look at a small problem they're having with some facet of Office. And because it's a client and you want to appear helpful you agree.
There's just this blanket expectation that because you're a developer you have an innate knowledge of configuring spreadsheets, fixing Word templates, and any and all other office techie tasks, and furthermore you can cast your eye over some badly configured Office mess and instantly diagnose what the problem is.
I can only just about manage to put together a spreadsheet to schedule my reoccuring invoices and set up a Word template to write them. I regularly tell people that too - but no-one ever listens.
It depends a lot on the type of software you're currently developing
For example, when I was working on software for a local government, I had to learn things like
What are the rules for registering animals (pets). What are the types of registrations, what discounts apply, what are penalties for not registering on time
How are council rates calculated. How are rates raised yearly (actually, the algorithm for raising yearly rates and its implementation was the most complex task I met so far).
How are building permits issued. What types of inspections can be performed. Who is involved in the process of issuing a building permit (owner, builder, architect, officers etc.)
How often are water meters read. How are water meters assigned to properties, how many dials are on a water meter, how to detach a water meter from one property and to attach to a different one
What are different pension types. What are different discounts that are granted depending on a pension type.
What are different types of receipts. What different types of terminal printers (those that are used to print small receipts) exist and how to print to them.
What are properties, strata children, what are rules for dividing properties into 'parcels' ...
Well, that's just part of non-programming stuff that I learned during the 2 years on the project. The most unfortunate thing here is that now that I moved to a different company, there is very little chance that any of this knowledge I will ever use.
My job title is "Senior Software Engineer". In point of fact, for most of the past several years, I did fairly little software development, but did do a lot of:
Systems & web administration
Static web page development with HTML (I don't consider that programming, although I have done PHP, CGI, and JavaScript).
As others have said, help desk sorts of stuff, although not as much as in the past.
As a "task leader", I'm expected to have some people/management skills, although that usually devolves to writing monthly reports. I also get sucked into CMMi stuff from time to time, which in an ideal world might be somewhat relevant, but is usually just record keeping so the employer can bid on new contracts which require it.
Working in science lab, there's a need to know some of the science, especially if you want/need to work on the code doing the scientific calculations.
Working in a (U.S.) government facility, there's lots of paperwork and a need to know lots of government regulation (e.g. Freedom of Information Act)
Fortunately, I've recently made an internal transfer where I'm doing more development work and less of this other stuff!
Personally, I find that knowing more is always good, it paves the way to the next level. The hardest things in life is at the integration point. Literally. People focus a lot on specializing, but don't forget that you need people who can straddle both realms.