Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 5 years ago.
Improve this question
I've just finished two days of agile/scrum coaching at work which was great. I'm a a novice to professional programming so I needed it. However, I'm struggling very much with the notion of vertical slicing. In particular I cannot see how a database schema can emerge from several developers working separately across all development tiers (front middle and back)? Designing the database schema all in one go is called horizontal slicing and it's considered a no-no. I'm familiar with object relational mapping - slightly - from having worked with frameworks like Grails. But again, I still designed the schema first and worked from there.
To be clear, I am not arguing against the case for agile/scrum/vertical slicing, rather I am asking how it is possible? - in particular when it comes the mapping of objects the the database schema, how can this be done by several developers in tandem and not from the beginning?
This article which i really like seems to - if i've read it correctly - suggest that the database schema should be designed in the spring planning stage before any user stories are implemented.http://www.vertabelo.com/blog/notes-from-the-lab/data-modeling-in-agile-development-one-data-modelers-experience
The idea of a vertical slice is that you do the minimum amount needed to deliver some functionality.
Now it could be that the minimum amount of database design you need to do for the first story is the complete schema. But I think that is unlikely.
For example, to get the first bit of functionality working, do you need the tables related to authentication? What about the tables that do auditing? Maybe some parts of the schema are not related to certain functionality. Say you have a shopping basket application and one aspect of the schema relates to customer's saving their favourite look-and-feel. Could that part of the schema be left until the corresponding story is started?
If you have several developers following this approach then it is going to require a lot of conversation and coordination. Continuous integration of the database schema can really help. So as a developer makes changes to the schema regression tests are run to ensure it hasn't interfered with the other developers work.
This post explains it pretty well: http://blogs.adobe.com/agile/2013/09/27/splitting-stories-into-small-vertical-slices/
You do not necessarily have to have different people working on each piece of the vertical slice. It is more about breaking each part into smaller pieces to achieve incremental results.
From the above article:
Each slice is comprised of any work needed to be done in an architectural layer as well as any testing and integration that may need to be done to make it ready to release.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I know this question has been asked in various forms over and over again but I cannot find a single complete answer and I believe it is a general problem in the RDBMS/data area and industry. To explain the problem I will tell you a short (and maybe boring) story!
The Story
"I have a friend" who works in A company that uses 100+ systems. The scale and size of these systems vary from full-blown ITIL to custom/in-house, single purpose, LAMP/SQLite/CSV-based solutions. The vast majority of these systems, at one point or another use a database/data-store... Big-data has now become a trend, and A company though it is a very good idea to keep (or log) historical data from all systems forever! For that reason they have built a "warehouse". My friend is responsible for writing software that will do the analysis on some of this data ... however, he is kinda confused. There are thousands of tables in that warehouse containing data from the beginning of time (1970s I think :)).
The Problem
[Since I started telling you about this guy, I should probably continue]
My friend is very upset because of the lack of documentation in that warehouse. It seems that no-one knows what is what?! Few of the problems he faces (and I quote):
Man, some fields are constants... they have a special meaning to the application but I have no way of knowing? But that is OK... cause some other fields are bit-masks! Different bit values in the field have different meaning!
and he continues...
That's not all... these are the easy cases you know... Since we have data from multiple systems, we end up with a situation where different systems refer to the same thing in a different way... how can I explain it to you... for example, a network device has an FQDN, however some systems treat it as the primary key, some others don't and instead they allocate an auto-increment integer value, which in turn they use for foreign keys (you know... referencing this device).
and he can go on forever!
The Question
[Yes, it is one question]
He says:
We have come a long way regarding documentation in the software world... we have started with documents, moved to wikis and concluded to inline docblocks serving both as parameter/signature documentation and as wiki! We can auto-generate documentation, clear enough that a person in the other hemisphere and side of this world can easily follow!
and he continues:
... in the data side of things, we also had major achievements! Storage methods, serialization, transmission and data analysis techniques have evolved tremendously... We have also managed to map database tables into objects and in some cases we can even represent relationships!
So why the frell don't we have a standard method/technique of documenting our data structures in an RDBMS?
... he concluded :)
Enough with my friend, so my comments:
I know about comments on fields in various systems, but that is usually enough for a "deprecated" and not for an explanation
Updating a wiki or even worse a document, every time you release a database patch is not a solution... that patch should contain the relevant documentation!
ER diagrams can be easily generated based on the schema information, however this is not the easiest form of documentation to read... for anything more than 10 tables!
There is the saying (please comment if you know who said this! - respect)
Documentation is like sex: when it is good, it is very, very good; and when it is bad, it is better than nothing
Why SQL doesn't provide the means to any?
Any kind of documentation would get stalled if not maintained.
Also, the SQL world provides all kind possibilities to document things:
comments in SQL files
comments in columns/tables metadata
as you said - E/R diagrams
the classic way of documenting stuff - docs and wikis
good discipline in adhering to an intuitive naming scheme for the things in DB - I think this should be the standard
We have all the tools we need, we just have to convince our managers to let us write the docs (lol)
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
I have more of a "what-would-you-do" question than an actual coding question.
It relates to a project I am currently working on. In this project,
we are tasked with combining several marketplace APIs into one interface. Each
API has its own unique way of categorizing products. The top-level
parent categories for all the APIs we are looking at are more-or-less
the same with some variations. But, the subcategories are wildly
different.
For example, one API requires long bread-crumb trails
to select a category, such as: Sports > Ball Sports > New England >
Football > Active Teams > Patriots > Memorabilia. While another API has two-level
categorization: Sports > Patriots Memorabilia. In many cases, there are sub-categories
that don't relate whatsoever to the subcategories of other APIs.
So, the question is - what is the best approach to take when designing
the interface? We are currently wrestling between two possibilities:
1) Design a custom category UI on the client and then build logic into
the server that is able to sort through the needs of the various APIs
based on user-selected choices.
2) Create the UI in such a way that the user has to walk through the
necessary steps for each individual API. Depending on user settings, this means that
he may need to fill out API - specific information 5,6,10, or more
times.
While I am told that option number one is a real programming nightmare (the
example I am given is changing API data fields) I feel strongly that option number two will piss off customers.
Any ideas out there??
This a very hard problem. If you search for "ontology product classification" you'll find many research papers and discussions on the topic. If one was simply a more detailed version of the other it would be quite feasible, but your description implies that isn't the case, and thus you'll need to construct your own classification scheme and map the others onto it.
Do you have a common key (UPC code? or other) that will allow you to verify your mapping between the different product categories? If so, you might be able to construct your own categorization scheme and then map the others onto it with some degree of success.
Clearly the first option is the best for the consumer but it could be very hard and very time consuming to construct such a mapping and it will need constant updates.
One approach would be to construct a simpler hierarchy than any of the ones provided. A simpler hierarchy will make the [mostly manual] effort of mapping categories into your hierarchy much easier as most will simply be inclusions. This might make the user experience worse but if you add great search capabilities and great "related products" / "people who bought this also bought this" tools around the product browsing experience you can probably make up for the lack of hierarchy.
Number One isn't that bad of a nightmare. Your users' experience is the number one priority; never forget that. If the user has an easier time navigating a shorter route, then give them that opportunity. Also, I would wrap the API with some abstraction so my code doesn't know about the API at all and only knows about the abstraction layer. This way I can change APIs and leave most of my code alone, only changing the abstraction layer.
Use a session to pass data from page to page and a factory to create the page's state on entry based on the session data; this will strengthen the context between page, state, and user data.
Keep first level objects (the ones the page directly talks to) in context to the page; this will help when diagnosing problems.
Most importantly, create a series of tests for your abstraction layer that test every object, function, and input output pair to make sure your application is rock solid.
You have to provide to your clients a consistent not changing interface.
I would like to see an example of the two different approaches you have in mind.
API.find( PRODUCT, CATEGORIES_LIST )
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I have been doing a coding in for last 7 months. Most part of it has been the updations to the very poorly coded software in asp. Now I am going to redevelop entire software in asp.net. I want it to be very efficient. I need some tips and guidance about how to plan project and design a good database. Thanks in advance for help.
As far as planning for the project is concerned you first need to create your application's pages on paper. Write all your ideas on the paper.
Draw each and every page on paper and complete it with your new ideas.
Then taking every page's feature into consideration draw the database tables. This is a very careful phase. So include every aspect of your application's functionality.
For designing the database follow these steps :
List all entities
Find Relationships [E-R matrix]
Draw Cardinality
Define Primary keys
Draw ERD
Eliminate many-to-many relationships
Identify attributes [attribute entity matrix]
Map attributes - only 1:1 with each entity
Draw final ERD
More help here
Structured process you must know to develop a web application
Here's my super-quick spiel about database design.
A database is two things. It is a 'logical engine', and it is a 'physical engine'. It is a logical engine in that it allows you to create meaningful models of entities. It is a physical engine in that it processes bits of data to update or return results.
I suggest paying about equal attention to each of these aspects when designing a new system. To adress the first, work out the relational model and normalize as far as possible. To address the second, think about where you might want to denormalize for speed, what indexes you'll need to speed the processing, etc.
It's good that you ask, this is one of the most anti intuitive things in software development. My approach would be: pick one feature, implement it in ASP.NET, and convince your users to move over. You'll be surprised how many things you missed, or how actual performance compares to developer testing.
The quicker you iterate, the better your replacement project will be, and the less time you spend on things that don't really matter. There's a excellent blog post at Coding Horror about this; it's called Boyd's Law of Iteration.
Closed. This question is opinion-based. It is not currently accepting answers.
Closed 9 months ago.
Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
I read this term a lot. What exactly is a legacy database? I ask because I had thought it meant an old database like dbase or rdb, but I don't think I'm right.
When looking at RoR or Django and "legacy database" integration, what does legacy database really mean? Is it different than a generic term "legacy database"?
In the general context, it can refer to any of the older database technologies.
In a more specific context, it can refer to a database system that was inherited by a team from previous project owners.
legacy: anything from the past that keeps coming around to haunt you.
A legacy database is generally something that you will have to inherit and base some of your design decisions around. Most companies that put out work may already have some other (usually horrible) solution and you need to give them a bigger and better product...
BUT
It has to work with all of their old legacy data. The company is not going to want to manage two different applications just so they can keep all their old records. You will need to develop your solution to be able to migrate the data from the legacy system over into your system. This can have a massive impact on the overall design of the new database, because it cannot stray too far from the previous without introducing a lot of problems in terms of data integrity.
It's usually derogatory in my experience:
Something no-one wants to touch in case it breaks
Databases that can't be maintained (say that SQL 6.5 box lying around)
Someone else's badly designed and implemented database
Something that someone is trying to replace
Supported by the 93 year old wierdo
If it's in-use but still has maintenance or development activities, it can't be legacy...
Edit:
Given the age of the SQL language and the RDBMS, everything is legacy (including my new system due next year) compared to the software listed. At what point does Ruby turn legacy from the database perspective..?
We mostly use the term 'legacy database' as a db schema we can not 'easily' modify without breaking other software/systems using this schema.
this sums it up pretty well.
[edit] Broken link. Here's the quote from FOLDOC:
Legacy System -- A computer system or application program which continues to be used because of the cost of replacing or redesigning it and often despite its poor competitiveness and compatibility with modern equivalents. The implication is that the system is large, monolithic and difficult to modify.
If legacy software only runs on antiquated hardware the cost of maintaining this may eventually outweigh the cost of replacing both the software and hardware unless some form of emulation or backward compatibility allows the software to run on new hardware.
Flat file, hierarchy, and network databases are usually referred as legacy databases. They represent the ways people used to organize information in prehistoric times — about 30 years ago.
Legacy is used to denote the old thing. legacy database is something which continues to be used because of it cost of replacing and redesigning it.
In general context refers to old code inherited. Tipycally cobol code.
It is used for code which it is still used for historcal reasons.
It applies also for DB schemas
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
What would you suggest for the following scenario:
A dozen of developers need to build and design a complex system. This design needs to be documented for future developers and the design decisions must be noted. These reports need to be made about every two months. My question is how this project should be documented.
I see two possibilities. Each developer writes about the things they helped design and integrate and then one person combines each of these documents together. The final document will probably be incoherent or redundant at times since the person tasked of assembling everything won't have much time to adjust every part.
Assume that the documentation parts from each developer arrive just a few days before deadline. A collaborative system (i.e. wiki) wouldn’t work properly since there wouldn’t be anything to read until a few days before deadline.
Or should a few people (2-3) be tasked with writing the documentation while the rest of the team works on actually developing the system? The developers would need a way to transfer their design choices and conclusions to the technical writers. How could this be done efficiently?
We approach this from 2 sides, using a RUP style approach. In the first case, you'll have a domain expert who is responsible for roughing out the design of what you're going to deliver - with developers chipping in as necessary. In the second case, we use a technical author - they document the application, so they should have a good idea of how it hangs together, and you involve them right through the design and development process. In this case, they can help to polish the design, and to make sure that it matches what they thought was being developed.
We use confluence (atlassian's wiki-like-thing) and document all kinds of different "things". The developers do it continiously, and we push each other for docs - we let peer pressure decide what is necessary. Whenever someone new comes along he/she is tasked with reading through everything and to find out what still is correct. The incorrect stuff is either deleted or updated as a consequence of this. We're happy when we can delete stuff ;)
The nice thing about this process is that the relevant stuff stays and the irrelevant stuff is deleted. We always "got away" from the more formalized demands by claiming that we could always construct the word documents they wanted if "they" needed them. "They" never needed them.
I think alternative 2 is the less agile, because it means a new stage to the project (although it may be in parallel with tests).
If you are in an agile model, then just add documentation (following a guideline) as a story.
If you are in a staged approach, then I would nevertheless ask developers to work on documentation, following some guidelines, and review that documentation along the design and the code. Eventually, you may have a technical writer reviewing everything for proper English, but that would be a kind of "release" activity.
I think you can use Sand Castle to document your project.
Check it out
Sand Castle from Microsoft
It's not a complete documentation, but making sure that interfaces etc. are commented using Doxygen-style comments means writing code and documenting it are closer together.
That way, developers should document what they do. I still think a review by the architect(s) is needed to ensure consistent quality, but ensuring people document what they do is the best way to ensure they follow the architecture.