As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
Background:
I am taking a class at my university called "Software Constraints". In the first lectures we were learning how to build good APIs.
A good example we got of a really bad API function is the socket public static void Select(IList checkRead, IList checkWrite, IList checkError, int microseconds); in C#. The function receives 3 lists of sockets, and destroys them making the user have to clone all the sockets before feeding them into the Select(). It also has a timeout (in microseconds) which is an int, that sets the maximum time the server can wait for a socket. The limits of this is +/-35 minutes (because it is an int).
Questions:
How do you define an API as
'bad'?
How do you define an
API as 'good'?
Points to consider:
Function names that are hard to remember.
Function parameters that are hard to understand.
Bad documentation.
Everything being so interconnected that if you need to change 1 line of code you will actually need to change hundreds of lines in other places.
Functions that destroy their arguments.
Bad scalability due to "hidden" complexity.
It's required from the user/dev to build wrappers around the API so that it can be used.
In API design I've always found this keynote very helpful:
How to Design a Good API and Why it Matters - by Joshua Bloch
Here's an excerpt, I'd recommend reading the whole thing / watching the video.
II. General Principles
API Should Do One Thing and Do it Well
API Should Be As Small As Possible But No Smaller
Implementation Should Not Impact API
Minimize Accessibility of Everything
Names Matter–API is a Little Language
Documentation Matters
Document Religiously
Consider Performance Consequences of API Design Decisions
Effects of API Design Decisions on Performance are Real and Permanent
API Must Coexist Peacefully with Platform
III. Class Design
Minimize Mutability
Subclass Only Where it Makes Sense
Design and Document for Inheritance or Else Prohibit it
IV. Method Design
Don't Make the Client Do Anything the Module Could Do
Don't Violate the Principle of Least Astonishment
Fail Fast - Report Errors as Soon as Possible After They Occur
Provide Programmatic Access to All Data Available in String Form
Overload With Care
Use Appropriate Parameter and Return Types
Use Consistent Parameter Ordering Across Methods
Avoid Long Parameter Lists
Avoid Return Values that Demand Exceptional Processing
You don't have to read the documentation to use it correctly.
The sign of an awesome API.
Many coding standards and longer documents and even books (Framework Design Guidelines) have been written on this topic, but much of this only helps at a fairly low level.
There is also a matter of taste. APIs may obey every rule in whatever rulebook, and still suck, due to slavish adherence to various in-vogue ideologies. A recent culprit is pattern-orientation, wherein Singleton Patterns (little more than initialized global variables) and Factory Patterns (a way of parameterizing construction, but often implemented when not needed) are overused. Lately, it's more likely that Inversion of Control (IoC) and associated explosion in the number of tiny interface types that adds redundant conceptual complexity to designs.
The best tutors for taste are imitation (reading lots of code and APIs, finding out what works and doesn't work), experience (making mistakes and learning from it) and thinking (don't just do what's fashionable for its own sake, think before you act).
Useful - it addresses a need that is not already met (or improves on existing ones)
Easy to explain - the basic understanding of what it does should be simple to grasp
Follows some object model of some problem domain or real-world. It uses constructs that make sense
Correct use of synchronous and asynchronous calls. (don't block for things that take time)
Good default behavior - where possible allow extensibility and tweaking, but provide defaults for all that is necessary for simple cases
Sample uses and working sample applications. This is probably most important of all.
Excellent documentation
Eat your own dog food (if applicable)
Keep it small or segment it so that it is not one huge polluted space. Keep functionality sets distinct and isolated with few if any dependencies.
There are more, but that is a good start
A good API has a semantic model close to the thing it describes.
For example, an API for creating and manipulating Excel spreadsheets would have classes like Workbook, Sheet, and Cell, with methods like Cell.SetValue(text) and Workbook.listSheets().
A good API allows the client to do pretty much everything they need to do, but doesn't require them to do a lot of mindless busy-work. Examples of "mindless busy work" would be initializing data structure fields, calling several routines in a sequence that never varies with no real custom code in between, etc.
The surest sign of a bad API is if your clients all want to wrap it with their own helper code. At the absolute least, your API should have provided that helper code. Most likely, it should have been designed to provide the higher level of abstraction the clients are rolling themselves on their own every time.
A bad API is one that is not used by its intended audience.
A good API is one that is used by its intended audience for the purpose for which it was designed.
A great API is one that is used by both its intended audience, for the its intended purpose, and an unintended audience for reasons unanticipated by its designers.
If Amazon publishes its API as both SOAP and REST, and the REST version wins out, that doesn't mean the underlying SOAP API was bad.
I'd imagine that the same will be true for you. You can read all you want about design and try your best, but the acid test will be use. Spend some time building in ways to get feedback on what works and what doesn't and be prepared to refactor as needed to make it better.
A good API is one that makes simple things simple (minimum boilerplate and learning curve to do the most common things) and complicated things possible (maximum flexibility, as few assumptions as possible). A mediocre API is one that does one of these well (either insanely simple but only if you're trying to do really basic stuff, or insanely powerful, but with a really steep learning curve, etc). A horrible API is one that does neither of these well.
There are several other good answers on this already, so I thought I'd just throw in some links I didn't see mentioned.
Articles
"A Little Manual Of API Design" by Jasmin Blanchette of Trolltech
"Defining QT-Style C++ APIs" also Trolltech
Books:
"Effective Java" by Joshua Bloch
"The Practice Of Programming" by Kernighan and Pike
I think a good API should allow custom IO and memory management hooks if it's applicable.
A typical example is you have your custom compressed archive format for data on disk and a third party library with a poor api wants to access data on disk and expects a path to a file where it can load its data.
This link has some good points:
http://gamearchitect.net/2008/09/19/good-middleware/
If the API produces an error message, ensure that the message and diagnostics help a developer work out what the problem is.
My expectation is that the caller of an API passes in input that is correct. A developer is the consumer of any error messages produced by the API (not the end user), and messages aimed at the developer help the developer debug their calling program.
An API is bad when it is badly documented.
An API is good when it is well documented and follows a coding standard.
Now these are two, very simple and also very hard points to follow, this brings one into the area of software architecture. You need a good architect that structures the system and helps the framework follow its own guidlines.
Commenting code, writing an well explained manual for the API is Mandatory.
An API can be good if it has a good documentation which explains how to use it. But if the code is clean, good and follows a standard inside itself, it doesnt matter if it doenst have a decent documentation.
I've written a little about coding structure here
I think what is paramount is readability, by which I mean the quality that makes the greatest number of programmers to make sense of what the code is doing in the shortest possible amount of time. But judging which piece of software is readable and which is not has that indescribable human quality: fuzziness. The points you mention do partly succeed in crystallizing it. However, in its entirety it has to remain a case-by-case affair and it would be really hard to come up with universal rules.
Related
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I'm trying to learn enough about Windows Runtime to make a recommendation about what it would entail for my employer to port our existing applications to it. I'm having trouble finding documentation that provides a technical overview of how the API works.
All my web searches seem to lead me to API reference on MSDN, which is terse to the point of unreadability. It documents the formal signatures of API classes and methods, but seems to assume that the reader already knows how things fit together. The purpose of each method is usually just described as a terse sentence fragment that restates its name with spaces instead of CamelCase, and further explanations about restrictions, expectations and invariants beyond what is evident in the type declarations are almost completely absent. (This contrasts with the fairly informative "Remarks" sections in the reference documentation of the ordinary Win32 API).
Clearly, I'm not supposed to be using this documentation to develop an initial overview of how the API works. What am I supposed to be using?
Moving one level up in MSDN there is a section with the promising name Concepts and architecture, and some even more promising-sounding Programming concepts and Fundamentals -- but what they actually describe is a seemingly random selection of fairly specialized topics, certainly not what I need to make sense of the API reference.
Is there official documentation in book form that I need to buy and read? Something outside of MSDN? A secret MSDN link that I haven't been able to find?
I've seen this previous question which didn't get any real answers, perhaps because the it was phrased rather opaquely with $5 words like "ontology". In an attempt to explain better what I'm looking for, here are some examples of questions I hope the documentation I seek would tell me the answers to:
(Note that these are examples only. My primary goal is to find a specification that answers these and similar questions, rather than get answers to these specific examples.)
Windows.Networking.Sockets.StreamSocket has an InputStream property of type Windows.Storage.Streams.IInputStream, which I'm clearly supposed to use to read from the socket. But the only method of IInputStream is ReadAsync which reads into an IBuffer, and IBuffer is an interface that declares nothing but capacity and size properties. How do I get at the actual bytes being read? If I implement IBuffer myself, how will the system deliver them to me?
After hours of frustrated clicking and googling, I have tentatively concluded that the interface is a lie -- IBuffer is not something anyone can implement, but ReadAsync wants specifically a Windows.Storage.Streams.Buffer (without the I), no matter what its type declaration says. Then it seems I can use DataReader to read the actual bytes from the Buffer. Is that really how it's supposed to go?
or
Hmm, it looks like DataReader has a constructor that takes an IInputStream, so perhaps I can cut out the Buffer middleman after all. However, this seems to be wrong, because DataReader's methods such as ReadBytes are synchronous and supposedly all I/O in WinRT is asynchronous; certainly the one declared method of IInputStream is. So how does that work?
After more frustrated googling and clicking: Oh, there's a LoadAsync method in DataReader that does ... something. According to MSDN, it "loads data from the input stream", but what are the conventions for using it? Am I supposed to call it just once immediately after constructing the DataReader, or can I call it multiple times to reuse the same DataReader for the next read operation? Does DataReader contain a circular buffer internally? What happens if I try to read more bytes than have been read asynchronously already? The super-terse documentation of the ReadFoo methods mention no exceptions or error conditions; neither do the class documentation for DataReader or IDataReader.
or
Apparently apps can be multi-threaded, since the supported Win32 APIs include things like InterlockedCompareExchange, EnterCriticalSection and so forth. But neither CreateThread nor the RTL's _beginthreadex seem to be supported, and there doesn't appear to be any Java-ish Thread class anywhere in the WinRT class hierarchy. How does one start a new thread?
or
Speaking about asynchronous I/O ... I'm quite comfortable with the general idea of asynchronous I/O and completion continuations, but what are the precise rules in WinRT for, say, which thread the completion routine is called in? If it's always the same thread I started the I/O operation from (which I hope!), do I need to make sure it enters some kind of alertable wait from time to time, so the system has a chance to call my code there?
or
Wikipedia claims that "WinRT is essentially a COM-based API, although relying on an enhanced COM." What exactly is this "enhancement"? If I follow COM rules and conventions, do I risk being bitten by things that work differently due to "enhancements"? Or, conversely, is there things I can do easier because of the enhancement?
or
The only description of how asynchronous callbacks work make it look like they are quite specific to the implementation language -- it looks fairly different between C#/CLR, JavaScript and C++/CX. What's actually happening at the COM/ABI level here? In particular, since the API documentation appears to assume that "C++" means "C++/CX", how does asynchronous I/O work if I use WRL instead? Or is it just the case that the await and then business is just language-provided sugar and the real ABI is always in terms of AsyncOperationWithProgressCompletedHandler and so forth, as described in the API reference? But that's a delegate type; does that even have a well-defined meaning in terms of COM?
I've just noticed that there seem to be two parallel page hierarchies on MSDN describing the WinRT API:
Windows Store app development | API Reference | Windows API reference for Windows Store apps
This is the almost vacuous documentation I rant about in the question. However, some of the API elements are also described in
MSDN Library | Additional Resources | Windows Runtime C++ reference
which is slightly closer to the COM metal, and occasionally contains useful Remarks sections. For example its page for IBuffer reveals that implementations of IBuffer must also implement IBufferByteAccess which provides access to the actual bytes.
It is not ideal (and still seems to leave a lot of information implicit), but it is at least something.
I think this post could be a https://chat.stackoverflow.com/ debate, but not a question.
WinRT Api and projections (c#/xaml or html/js etc.) references are in their first release and from my point of view at this time they are just a basic reference, not a extense documentation source.
This happpen usually with all recently created technologies, I think you just need to wait a couple of months and documentation will start to improve gradually.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I've read a blog post called Blogging with Noir, and I was honestly surprised that the author uses java.jdbc instead of libraries like Korma which I found surprising. What are the advantages of writing SQL queries in your code instead of letting tools do it for you?
I guess it is for the usual reasons that you might choose to use an API directly in Clojure rather than use a wrapper:
Existing knowledge: you already know the JDBC well and know that it will get the job done, why spend time learning a new abstraction unless there is a clear advantage?
Uncertainty - does the library have all the features you need? Will it continue to be maintained and implement new features in the future?
Stability - the wrapper may not yet be mature, so you run the risk of your code having to change if breaking changes occur / bugs are discovered.
Completeness - the wrapper may not (yet) encapsulate all of the functionality of the original API that you need
Overhead - sometimes extra layers of abstraction add a performance overhead that you don't need/want
Extra dependency - adds complexity to your build, and conceptual overhead in terms of the number of abstractions you need to keep in your head.
Ultimately it's a trade-off - the above are reasons that you might want to use the underlying API, but there are equally good reasons that you may choose to use the wrapper:
More idiomatic - a wrapper library is likely to give you much cleaner, more elegant code than a Java-based API (particularly if the Java API is imperative/stateful). You have to admit that Korma is pretty elegant!
More composable - Clojure wrappers tend to adopt a functional style, which leads to easy composability with other clojure code / libraries.
New features - often Clojure wrappers add extra functionality that the original API does not posess (for example, look at the data binding functionality added on top of Swing by Seesaw)
Korma IMO isn't nearly ready to be used as a full replacement for SQL. It's definitely handy, but right now a lot of my queries have (raw "...") snippets in them, and for more complicated stuff all the main querying is done inside SQL views which are then selected on via korma.
The main alternative, ClojureQL, doesn't even work with Clojure 1.3+
In short, it's hard to abstract SQL, and Korma - even though it tries to be minimal, meaning you still have to understand SQL pretty well to use it - isn't finished.
I can think about two reasons:
Almost everybody knows SQL, almost nobody knows Korma
This is a guess, because I do not know Korma myself, but raw SQL is sometimes suitable or even necessary if you want to do something specific like features that are only present in a particular database
Mine is not really a question, it's more of a call for opinions - and perhaps this isn't even the right place to post it. Nevertheless, the community here is very informed, and there's no harm in trying...
I was thinking about ways to create a highly scalable and, above all, highly modular back-end architecture. For example, an entire back-end ecosystem for a large site that had the potential for future-proof evolution into a massive site.
This would entail a very high degree of separation of concerns, to the extent that not only could (say) the underling DB be replaced (ie from Oracle to MySQL) but the actual type of database could be replaced (ed SQL to KV, or vice versa).
I envision a situation where each sub-system exposes its own API within the back-end ecosystem. In this way, the API could remain constant, whilst the implementation could change (even radically) over time.
The system must be heterogeneous in that it's not tied to a specific language. It must be able to accommodate modules or entire sub-systems using different languages.
It then occurred to me that what I was imagining was simply the architecture of the web itself.
So here is my discussion point: apart from the overhead of using (mainly) text-based protocols is there any overriding reason why a complex back-end architecture should not be implemented in the manner I describe, or is there some strong rationale I'm missing for using communication protocols such as Twisted, AMQP, Thrift, etc?
UPDATE: Following a comment from #meagar, I should perhaps reformulate the question: are the clear advantages of using a very simple, flexible and well-understood architecture (ie all functionality exposed as a series RESTful APIs) enough to compensate the obvious performance hit incurred when using this architecture in a back-end context?
[code]the actual type of database could be replaced (ed SQL to KV, or vice versa).[/code]
And anyone who wrote a join between two tables will be sad. If you want the "ability" to switch to KV, then you should not expose an API richer than what KV can support.
The answer to your question depends on what it is you're trying to accomplish. You want to keep each module within reasonable reins. Use proper physical layering of code, use defined interfaces with side-effect contracts, use test cases for each success and failure case of each interface. That way, you can depend on things like "when user enters blah page, a user-blah fact is generated so that all registered fact listeners will be invoked." This allows you to extend the system without having direct calls from point A to point B, while still having some kind of control over widely disparate dependencies. (I hate code bases where you can't find-all to find all possible references to a symbol!)
However, the fact that we put lots of code and classes into a single system is because calling between systems is often very, very expensive. You want to think in terms of code modules making requests of each other where you can. The difference in timing between a function call and a REST call is something like one to a million (maybe you can get it as low as one to ten thousand, if you only count cycles, not wallclock time -- but I'm not so sure). Also, anything that goes on a wire in a datacenter may potentially suffer from packet loss, because there is no such thing as a 100% loss-free data center, no matter how hard you try. Packet loss means random latency spikes in the response time for your application.
I asked this question about Microsoft .NET Libraries and the complexity of its source code. From what I'm reading, writing general purpose libraries and writing applications can be two different things. When writing libraries, you have to think about the client who could literally be everyone (supposing I release the library for use in the general public).
What kind of practices or theories or techniques are useful when learning to write libraries? Where do you learn to write code like the one in the .NET library? This looks like a "black art" which I don't know too much about.
That's a pretty subjective question, but here's on objective answer. The Framework Design Guidelines book (be sure to get the 2nd edition) is a very good book about how to write effective class libraries. The content is very good and the often dissenting annotations are thought-provoking. Every shop should have a copy of this book available.
You definitely need to watch Josh Bloch in his presentation How to Design a Good API & Why it Matters (1h 9m long). He is a Java guru but library design and object orientation are universal.
One piece of advice often ignored by library authors is to internalize costs. If something is hard to do, the library should do it. Too often I've seen the authors of a library push something hard onto the consumers of the API rather than solving it themselves. Instead, look for the hardest things and make sure the library does them or at least makes them very easy.
I will be paraphrasing from Effective C++ by Scott Meyers, which I have found to be the best advice I got:
Adhere to the principle of least astonishment: strive to provide classes whose operators and functions have a natural syntax and an intuitive semantics. Preserve consistency with the behavior of the built-in types: when in doubt, do as the ints do.
Recognize that anything somebody can do, they will do. They'll throw exceptions, they'll assign objects to themselves, they'll use objects before giving them values, they'll give objects values and never use them, they'll give them huge values, they'll give them tiny values, they'll give them null values. In general, if it will compile, somebody will do it. As a result, make your classes easy to use correctly and hard to use incorrectly. Accept that clients will make mistakes, and design your classes so you can prevent, detect, or correct such errors.
Strive for portable code. It's not much harder to write portable programs than to write unportable ones, and only rarely will the difference in performance be significant enough to justify unportable constructs.
Even programs designed for custom hardware often end up being ported, because stock hardware generally achieves an equivalent level of performance within a few years. Writing portable code allows you to switch platforms easily, to enlarge your client base, and to brag about supporting open systems. It also makes it easier to recover if you bet wrong in the operating system sweepstakes.
Design your code so that when changes are necessary, the impact is localized. Encapsulate as much as you can; make implementation details private.
Edit: I just noticed I very nearly duplicated what cherouvim had posted; sorry about that! But turns out we're linking to different speeches by Bloch, even if the subject is exactly the same. (cherouvim linked to a December 2005 talk, I to January 2007 one.) Well, I'll leave this answer here — you're probably best off by watching both and seeing how his message and way of presenting it has evolved :)
FWIW, I'd like to point to this Google Tech Talk by Joshua Bloch, who is a greatly respected guy in the Java world, and someone who has given speeches and written extensively on API design. (Oh, and designed some exceptionally good general purpose libraries, like the Java Collections Framework!)
Joshua Bloch, Google Tech Talks, January 24, 2007:
"How To Design A Good API and Why it
Matters" (the video is about 1 hour long)
You can also read many of the same ideas in his article Bumper-Sticker API Design (but I still recommend watching the presentation!)
(Seeing you come from the .NET side, I hope you don't let his Java background get in the way too much :-) This really is not Java-specific for the most part.)
Edit: Here's another 1½ minute bit of wisdom by Josh Bloch on why writing libraries is hard, and why it's still worth putting effort in it (economies of scale) — in a response to a question wondering, basically, "how hard can it be". (Part of a presentation about the Google Collections library, which is also totally worth watching, but more Java-centric.)
Krzysztof Cwalina's blog is a good starting place. His book, Framework Design Guidelines: Conventions, Idioms, and Patterns for Reusable .NET Libraries, is probably the definitive work for .NET library design best practices.
http://blogs.msdn.com/kcwalina/
The number one rule is to treat API design just like UI design: gather information about how your users really use your UI/API, what they find helpful and what gets in their way. Use that information to improve the design. Start with users who can put up with API churn and gradually stabilize the API as it matures.
I wrote a few notes about what I've learned about API design here: http://www.natpryce.com/articles/000732.html
I'd start looking more into design patterns. You'll probably not going to find much use for some of them, but as you get deeper into your library design the patterns will become more applicable. I'd also pick up a copy of NDepend - a great code measuring utility which may help you decouple things better. You can use .NET libraries as an example, but, personally, i don't find them to be great design examples mostly due to their complexities. Also, start looking at some open source projects to see how they're layered and structured.
A couple of separate points:
The .NET Framework isn't a class library. It's a Framework. It's a set of types meant to not only provide functionality, but to be extended by your own code. For instance, it does provide you with the Stream abstract class, and with concrete implementations like the NetworkStream class, but it also provides you the WebRequest class and the means to extend it, so that WebRequest.Create("myschema://host/more") can produce an instance of your own class deriving from WebRequest, which can have its own GetResponse method returning its own class derived from WebResponse, such that calling GetResponseStream will return your own class derived from Stream!
And your callers will not need to know this is going on behind the scenes!
A separate point is that for most developers, creating a reusable library is not, and should not be the goal. The goal should be to write the code necessary to meet requirements. In the process, reusable code may be found. In that case, it should be refactored out into a separate library, where it can be reused in the future.
I go further than that (when permitted). I will usually wait until I find two pieces of code that actually do the same thing, or which overlap. Presumably both pieces of code have passed all their unit tests. I will then factor out the common code into a separate class library and run all the unit tests again. Assuming that they still pass, I've begun the creation of some reusable code that works (since the unit tests still pass).
This is in contrast to a lesson I learned in school, when the result of an entire project was a beautiful reusable library - with no code to reuse it.
(Of course, I'm sure it would have worked if any code had used it...)
When designing the public API of a generic library, how much of the low-level stuff that's used internally should be exposed? On the one hand, users should not depend too heavily on implementation details, and too many low-level functions/classes might clutter the API. Therefore, the knee-jerk response might be "none". On the other hand, some of the low-level functionality might be useful to people, and exposing more of it can prevent abstraction inversion (the re-implementing of low-level constructs on top of high-level constructs).
Furthermore, exposing more low-level details could provide performance shortcuts. For example, let's say you have a function to find the median of an array. The principle of least surprise says that you should duplicate the array so that users of your API don't have to care that its implementation involves the side effect of reordering elements. Should you, in this case, note that median() costs a memory allocation and provide another function that bypasses the allocation, but will arbitrarily reorder the user's input?
What are some general guidelines for how much of this kind of detail to expose?
As little as is possible.
The more details you expose, the more likely a change will break a consumer.
Your API shouldn't allow callers to "break" anything by mucking up the state of the internals (e.g. reordering collections, etc.). To solve that problem, your exposed interfaces should be read only when necessary.
With respect to complexity, I lean far in the direction of simple, basic methods. I try very hard not to over-engineer anything with what I think will be needed down the road.
Write to today's requirements (maybe tomorrow's), but not beyond. You can always extend in the future. It's much harder to just drop things that you can't maintain anymore.
The unix way of doing it is to provide mechanisms, not policies. Just provide the right tools to do things (say, a knife), but try not to anticipate how they are going to be used (to peel apples or to sharpen pencils).
One way I've heard it expressed: Expose the what, but not the how.
The goal is to provide a useful and rich library for clients to use, without making them dependent of the library's internals. You want to be able to change the internals, without breaking the callers (as someone else already noted).
Writing a good API involves a certain amount of artful brinkmanship.