I have a very CPU intensive F# program that depends on persistent data-structures - about 40% of the total CPU time is spent in the Map module. So I thought I'd try out the PersistentHashMap in FSharpX collections. (BTW, this is already a big improvement over the previous version of F# in VS2013 where the same program spent 70% of its time in Map. I also notice that running programs with the debugger attached doesn't have the huge penalty it did before - good work guys...) There is also a hot-spot where I'm re-sorting all the time, where instead I should be adding to a Heap, so I thought I'd give that a go as well.
Two issue became immediately apparent:
(1) Swapping out one for the other from an interface perspective proved harder than it seems it should - I.e., making a shim that let me switch from a Map to a PersistentMap, preserving both the needed module-based let-bound functions and Types necessary to use the each map. I know that having full HM type-inference (and no type-classes) is orthogonal to LSP-style referential transparency for the most part - but maybe I was missing some way to do this better with a minimal amount of code.
(2) The biggest problem (which I'd like to focus on here) is the reliance of the F# functional data-structs on oo-style dispatched equality and comparison via the IComparison (when 't : comparison), etc., family of interfaces.
Even for OO programs ISTM that the idea of dispatching equality and comparison is a bad idea -- an object "knows" how to perform its own domain-specific tasks, but it doesn't "know" for the most part what notion of equality is going to be necessary at various points in the program for various purposes -- so equality/comparison should not be part of the object's interface, but when these concepts are needed, they should always be mentioned explicitly. For example, there should never be a .Sort(), only a .SortWith(...). One could argue that even something as basic as structural equality in F# could be explicit a.StructEq(b) or a ~= b - otherwise you always get object.Equals -- but even stipulating that doing things this way is the best for a multi-paradigm language that's a first-class .Net citizen, it seems like there should at least be the option of using passed-in comparison and equality functions, but this is not the case.
This means that: (a) type constraints are enforced even if you don't want them, causing ripples of broken inferred typing (and hundreds of wavy red lines with it being unclear where the actual "problem" is) and (b), that by implementing a notion of equality or comparison that makes one container type happy in one part of your program (and in my case I want to use the same container and item type with two different notions of ordering in two different places), it is likely to silently break (or cause inefficiency, if one subsumes the other) in other parts of the code that depended on the default/previous implementation.
The only way around this that I could think of is wrapping each item a adapter object using new...with object expression - but I really don't want to create so much garbage just to get the code to work.
So, ISTM that we could have a "pure" version of each persistent data struct that could be loaded if desired (even basics like List, etc.) that do not depend on dispatched equality/comparison/hashing and do not impose type constraints - all such needs should be via a passed in fn's at the time of the call. (Dispatched eq/cmp would be only for used for interop with BCL collections that don't accept delegates.) Then we could have a [EqCmpHashThrowNotImplemented] attribute, and I could be sure that there were no default operations happening at all, and I would feel better about the efficiency and predictability of my code. (And this also let's one change from a Record to a Class or visa-versa w/o worrying about any changes in behavior due to default implementations.) Again, this would be optional, but done by with a simple import. (Which does mean that each base core collection type would have to be broken out into its own module, which isn't really a bad idea anyway.)
If I've overlooked a better way to do things or there are some patterns people are using here, I'd be interested.
I'm just learning to program in scala.
I have some experience in functional programming, as I have in object oriented programming.
My question is kind of simple, yet tricky:
Which structures should be used in Scala? Should we only stick to immutables, eg. modifing lists by iterating through it and stick a new one together, or go for mutables? What is your opinion on that, what are the performance aspects, memory related aspects, ...
I'm likely to program in a functional style, but it often expands to an insane amount of effort to do things which are easily done by using mutables. Is it situation dependent, what to use?
Prefer immutable to mutable state. Use mutable state only where it is absolutely necessary. Some notable reasons include:
Performance. The standard libraries make wide use of vars and while loops, even though this is not idiomatic Scala. This should not be emulated, however, except for cases where you have profiled to determine that modifying the code to be more imperative will bring a significant performance gain.
I/O. I/O, or interacting with the outside world is inherently state dependent, and thus must be dealt with in a mutable manner.
This is no different than the recommended coding style found in all major languages, imperative or functional. For example, in Java it is preferable to use data objects with only private final fields. Code written in an immutable (and functional) way is inherently easier to understand because when one sees a val, they know it will never change, reducing the possible number of states any particular object or function can be in.
In many cases, it also allows automatic parallel execution, for example, collection classes in Scala all have a par function, which will return a parallel collection that automatically run the calls to functions like map or reduce in parallel.
(I thought this must be a duplicate but couldn't easily find an earlier similar one, so I venture to answer...)
There is no general answer to this question. The rule of thumb suggested by the creators of Scala is to start with immutable vals and structures and stick to them as long as it makes sense. You can almost always create a workable solution to your problem this way. But if not, of course be pragmatic and use mutability.
Once you have a solution, you can tweak it, test it, measure its performance etc. If you find that e.g. it is too slow or overly complex, identify the critical part of it, understand what makes it problematic and - if needed - reimplement it using mutable variables, ideally keeping it isolated from the rest of the program. Note though that in many cases, a better solution can be found from within the immutable realm as well, so try looking there first. Especially for a beginner like myself, it still happens regularly that the best solution I could come up with looked contorted and complex with no apparent way to improve it - until seeing a simple and elegant solution to the same problem in a few lines of code, created by an experienced Scala developer who controls more of the power of the language and its libraries.
I usually obey the following rules:
Never use static mutable vars
Keep all user defined data types (typically case classes) immutable unless they are very expensive to copy. This will simplify a lot of the application logic.
If a data structure/collection is inherently mutable (i.e. it's designed to change over time), using a mutable data structure/collection might be appropriate. An example might be a large game world that is updated when players move. Remember to (almost) never share these data structures between threads though.
It's fine to use mutable local vars in methods
Use immutable collections for function results. These can be strictly or lazily evaluated depending on what gives best performance in the used context. Be careful if you use a lazily evaluated result which depends on a mutable collection though.
I was writing a class file and I included a CGPoint as an Ivar. This got me wondering about the overhead associated with smaller objective-c data structures. Is the memory footprint of something like a CGPoint significant enough to justify making a pointer to it, or would I just be making a pointer to 2 CGfloat values? For that matter, if all I need are x/y coordinates why not just stick 2 ints in as ivars?
On a related note, is there a nomenclature for describing tiny data structures, like "petty data structures", or "trivial data structures"; a word that describes a a struct made of a few primitives.
There are certainly issues with small objects, but there are supposedly several small-object optimizations in the system already.
In general, if you need an object, use an object. If not, then don't.
However, the bigger issue is to write your code so that it is easy to read by humans, and easy to maintain by humans. Use performance tools (like Instruments) to isolate places where system resource utilization needs to be addressed, and only then address those issues.
Of course, there are obvious stupid things to avoid, but in general, focus on a clean design, and easy to read/change implementation. Running the performance tools on your test suite should easily spot anything too wrong.
First don't worry about the overhead until you have a performance issue.
Most (but not all) object-oriented languages make a distinct between small, non-object, types and larger, object-based, ones; but the boundary is fuzzy. A useful distinction to decide how to represent a type is whether you think of it as a simple value, which you probably think as operated on or calculated with; or something more involved, which may be something which has inherent behaviour, operates on etc.
The built-in primitive types are mostly values: integers, characters, etc. In that vein complex numbers, fractions, coordinates etc. are simple values - use structures; trees, stacks, hierarchies are not - use objects.
On your related note consider many use "value", you can also consider "composite". "Petty" isn't a good choice, wrong connotations ;-) Others might say "basic", "trivial", "structure" or "record" - while the latter two can also be used to refer to very large types they often are used for small ones.
The creator of the Clojure language claims that "open, and large, set of functions operate upon an open, and small, set of extensible abstractions is the key to algorithmic reuse and library interoperability". Obviously it contradicts the typical OOP approach where you create a lot of abstractions (classes) and a relatively small set of functions operating on them. Please suggest a book, a chapter in a book, an article, or your personal experience that elaborate on the topics:
motivating examples of problems that appear in OOP and how using "many functions upon few abstractions" would address them
how to effectively do MFUFA* design
how to refactor OOP code towards MFUFA
how OOP languages' syntax gets in the way of MFUFA
*MFUFA: "many functions upon few abstractions"
There are two main notions of "abstraction" in programming:
parameterisation ("polymorphism", genericity).
encapsulation (data hiding),
[Edit: These two are duals. The first is client-side abstraction, the second implementer-side abstraction (and in case you care about these things: in terms of formal logic or type theory, they correspond to universal and existential quantification, respectively).]
In OO, the class is the kitchen sink feature for achieving both kinds of abstraction.
Ad (1), for almost every "pattern" you need to define a custom class (or several). In functional programming on the other hand, you often have more lightweight and direct methods to achieve the same goals, in particular, functions and tuples. It is often pointed out that most of the "design patterns" from the GoF are redundant in FP, for example.
Ad (2), encapsulation is needed a little bit less often if you don't have mutable state lingering around everywhere that you need to keep in check. You still build ADTs in FP, but they tend to be simpler and more generic, and hence you need fewer of them.
When you write program in object-oriented style, you make emphasis on expressing domain area in terms of data types. And at first glance this looks like a good idea - if we work with users, why not to have a class User? And if users sell and buy cars, why not to have class Car? This way we can easily maintain data and control flow - it just reflects order of events in the real world. While this is quite convenient for domain objects, for many internal objects (i.e. objects that do not reflect anything from real world, but occur only in program logic) it is not so good. Maybe the best example is a number of collection types in Java. In Java (and many other OOP languages) there are both arrays, Lists. In JDBC there's ResultSet which is also kind of collection, but doesn't implement Collection interface. For input you will often use InputStream that provides interface for sequential access to the data - just like linked list! However it doesn't implement any kind of collection interface as well. Thus, if your code works with database and uses ResultSet it will be harder to refactor it for text files and InputStream.
MFUFA principle teaches us to pay less attention to type definition and more to common abstractions. For this reason Clojure introduces single abstraction for all mentioned types - sequence. Any iterable is automatically coerced to sequence, streams are just lazy lists and result set may be transformed to one of previous types easily.
Another example is using PersistentMap interface for structs and records. With such common interfaces it becomes very easy to create resusable subroutines and do not spend lots of time to refactoring.
To summarize and answer your questions:
One simple example of an issue that appears in OOP frequently: reading data from many different sources (e.g. DB, file, network, etc.) and processing it in the same way.
To make good MFUFA design try to make abstractions as common as possible and avoid ad-hoc implementations. E.g. avoid types a-la UserList - List<User> is good enough in most cases.
Follow suggestions from point 2. In addition, try to add as much interfaces to your data types (classes) as it possible. For example, if you really need to have UserList (e.g. when it should have a lot of additional functionality), add both List and Iterable interfaces to its definition.
OOP (at least in Java and C#) is not very well suited for this principle, because they try to encapsulate the whole object's behavior during initial design, so it becomes hard add more functions to them. In most cases you can extend class in question and put methods you need into new object, but 1) if somebody else implements their own derived class, it will not be compatible with yours; 2) sometimes classes are final or all fields are made private, so derived classes don't have access to them (e.g. to add new functions to class String one should implement additional classStringUtils). Nevertheless, rules I described above make it much easier to use MFUFA in OOP-code. And best example here is Clojure itself, which is gracefully implemented in OO-style but still follows MFUFA principle.
UPD. I remember another description of difference between object oriented and functional styles, that maybe summarizes better all I said above: designing program in OO style is thinking in terms of data types (nouns), while designing in functional style is thinking in terms of operations (verbs). You may forget that some nouns are similar (e.g. forget about inheritance), but you should always remember that many verbs in practice do the same thing (e.g. have same or similar interfaces).
A much earlier version of the quote:
"The simple structure and natural applicability of lists are reflected in functions that are amazingly nonidiosyncratic. In Pascal the plethora of declarable data structures induces a specialization within functions that inhibits and penalizes casual cooperation. It is better to have 100 functions operate on one data structure than to have 10 functions operate on 10 data structures."
...comes from the foreword to the famous SICP book. I believe this book has a lot of applicable material on this topic.
I think you're not getting that there's a difference between libraries and programmes.
OO libraries which work well usually generate a small number of abstractions, which programmes use to build the abstractions for their domain. Larger OO libraries (and programmes) use inheritance to create different versions of methods and introduce new methods.
So, yes, the same principle applies to OO libraries.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
Our company keeps a MiscUtilities class that consists solely of public static methods that do often unrelated tasks like converting dates from String to Calendar and writing ArrayLists to files. We refer to it in other classes and find it pretty convenient. However, I've seen that sort of Utilities class derided on TheDailyWTF. I'm just wondering if there's any actual downside to this sort of class, and what the alternatives are.
Rather than giving personal opinion, I will quote from an authoritative source in the Java community, and examples from 2 very reputable third party libraries.
A quote from Effective Java 2nd Edition, Item 4: Enforce noninstantiability with a private constructor:
Occasionally you'll want to write a class that is just a grouping of static methods and static fields. Such classes have acquired a bad reputation because some people abuse them to avoid thinking in terms of objects, but they do have valid uses. They can be used to group related methods on primitive values or arrays, in the manner of java.lang.Math or java.util.Arrays. They can also be used to group static methods, including factory methods, for objects that implements a particular interface, in the manner of java.util.Collections. Lastly, they can be used to group methods on a final class, instead of extending the class.
Java libraries has many examples of such utility classes.
Apache Commons Lang follows the TypeUtils naming convention
ArrayUtils, StringUtils, ObjectUtils, BooleanUtils, etc
Guava follows the Types naming convention
Objects, Strings, Throwables, Collections2, Iterators, Iterables, Lists, Maps, etc.
The package summary actually has a specific section on classes of static utility methods
Another entire package consists of nothing but utility classes for working with Java primitives, Ints, Floats, Booleans, etc.
Short summary
Prefer good OOP design, always
static utility classes have valid uses to group related methods on:
Primitives (since they're not objects)
Interfaces (since they can't have anything concrete of their own)
final classes (since they're not extensible)
Prefer good organization, always
Group utility methods for SomeType to SomeTypeUtils or SomeTypes
Avoid a single big utility class that contains various unrelated tasks on different types/concepts
Convenient, most likely.
Possible to grow into a scary, hard to maintain swiss-army-rocket-chainsaw-and-floor-polisher, also most likely.
I'd recommend separating the various tasks into separate classes, with some logical grouping besides "won't fit anywhere else".
The risk here is that the class becomes a tangled mess nobody fully comprehends and noone dares to touch - or replace. If you feel that is an acceptable risk and/or avoidable under your circumstances, nothing really prevents you from using it.
I've never been a fan of the MiscUtilities class. My biggest issue is that I never know what is in it. Anything filed under miscellaneous is not discoverable. Instead I prefer to use a common dll that I can import into my projects that contains well named, separated classes for different purposes. The difference is subtle, but I find that it makes my life a little easier.
For languages that support functions, this sort of class is undeniably bad form.
For languages that don't, this sort of class isn't, and is probably superior to extending other classes with random utility methods. The static utility methods, because they are in some other class, can only use the public interface of the objects they handle, which decreases the likelihood of certain kinds of bug. And this approach also avoids polluting public interfaces with a random grab bag of whatever people happened to find useful at the time.
There's a certain amount of personal style involved of course. I'm not a big believer in classes that provide everything under the sun (even C++'s std::string is a tad over-featured for my taste) and tend to prefer to have helper functionality as separate functions. Makes maintenance of the class easier, forces the public interface to be useful and efficient, and with duck-typing style mechanisms the external functions can be used across a wide range of types without having to duplicate source text or share base classes and so on. (The oft-derided algorithms in the C++ Standard Library are a good demonstration of this, imperfect and verbose as they are.)
That said, I've worked with many who complain about strings that don't know how to interpret themselves as filenames, or split themselves into words, or what have you, and so on. (I pick on strings because they seem to be the prime target for utility functions...) I happen to think there are unseen maintenance and reliability costs associated with having large classes like that, quite apart from the ugliness of having a nominally simple class that's actually a vast illogical mishmash of unrelated concerns whose grubby fingers end up poking themselves into every last corner -- because your self-tokenizing string needs some kind of container to put the tokens in, right?! -- but it's a balancing act, even if my wording suggests it's more clean-cut than that.
I'm not a big believer in the notion of "OO dogma", but perhaps the paranoid might see it at work here. There's no good reason that all functionality should be attached to a particular class, and many good reasons why it should not. But some languages still don't allow the creation of functions, which does nothing to remove the need for them and forces people to work around the restriction by creating classes that consist of nothing but static methods. This rather overloads the meaning of the class concept, to my mind, and not in any good way.
So that IS a good reason to rail against this practice, but it's pretty futile unless the language changes to accommodate what people need to do. And languages don't come without functions unless their designers have an axe to grind, or there are technical reasons for it, so I should think that change in either case is unlikely.
I suppose the executive summary is: no.
Well, bad utility classes are derided on TheDailyWTF :)
There's really nothing wrong with having a generic utilities class for miscellaneous static business functions. I mean, you could try to put it all into a more object oriented approach, but at what cost in time and effort to the business and for what trade-off of maintainability? If the latter outweighs the former, go for it.
One approach you may be able to take, depending on the language, etc., is to perhaps move some of the logic into extensions on existing objects. For example, extending the String class (thinking in C# here) with a method that tries to parse the string into a DateTime. An in-house library of extensions just enhances the language with your business' own little DSL(s).
The company I work for has a class like that in its repository. Personally I find it annoying because you have to be really intimate with the class in order to know what it's useful for. Consequently, I've found myself re-writing methods that this class already covers! Double annoying because I've now wasted my time.
I would prefer a more object oriented approach that would lead to expandability. Have a Utilities class for sure, but inside it put other classes that expand toward specific functionality. Like Utilities.XML, Utilities.DataFunctions, Utilities.WhateverYouWant. That way you can expand and eventually take your 20 function MiscUtilities class and turn it into a 500 function class library.
A Class Library like this could then be used by anyone, and added to by anyone (with privileges) in a logically organized way.
I think the wrong defect of such a class is that it break Separation of concerns principle. I usually create multiple "Helpers" class to contains widely used, public static methods, for example ArrayHelpers to writing ArrayLists to files, and DatesHelper to converting dates from String to Calendar.
Moreover, if the class does contain complicated methods, it's better to try to refactor them using more object-oriented tecnique.
You can always switch from your "Helpers" class to the use of various OO pattern, leaving your old static methods to function as a Facade.
Yuo'll find great benefits everytime you'll be able to do so.
I keep a separate misc class for each project, and copy/paste code from other projects as needed. Perhaps not the best approach, but I prefer to avoid cross-project dependencies.
Examples of things in my helper class:
hex2, hex4, and hex8 (accept integer parameters, except hex8 which has integer and uinteger variations; all versions ignore higher-order bits)
byt (convert 8 lsb's of argument into a byte)
getSI, getUI, getSL, getUL (each takes a byte array and an offset, and returns the little-endian signed word, unsigned word, signed 32-bit word, or unsigned 32-bit word at that offset
putSI, putUI, putSL, putUL (takes a byte array, offset, and a value to put there in little-endian format)
hexArr (converts a byte array or portion thereof into a hex string)
hexToArr (converts a hex string to a byte array)
Zap(of T as iDisposable) (takes a byref iDisposable; if not Nothing, disposes it and sets it to Nothing)
Many of those are only useful when fiddling with binary data, but none of them is really domain-specific. Maybe the first six could go in a BinaryHelpers module, but I'm not sure where Zap should go other than in a misc utilities class.
Utility classes aren't bad, in and of themselves. They can be (mis|ab|over)used at times, but they do have their place. If you have utility methods for types you own, consider moving the static methods to the appropriate types. Or creating extension methods.
Do try to avoid a monolithic utilities class - they may be static methods, but they will have poor cohesion. Break up a large set of unrelated functions into smaller groupings of related functionality, much like you would your "normal" classes. Name them *Helper or *Utils, or whatever your preference is. But be consistent, and group them together, perhaps in a folder within a project.
When utility classes are broken up as described, you can create methods for working with specific types - primitives or classes, such as arrays, strings, dates and times, and so on. Admittedly, these wouldn't belong anywhere else, so a utility class is the place to go.
Personally, I often find such a class handy - even if only in the short term. That said, I try not to share them between projects. I would not keep a global version, but write one specific to each project - otherwise you're incorporating dead-weight which may cause issues for security or architecture.
What I do for my personal projects is keep a misc library but rather than adding a reference in my projects, I paste the relevant bits of code in to the relevant places. It's technically duplicaintg it, but not within a single solution and thats the important thing. However I don't think this would work on a larger scale, too messy.
I generally don't have a problem with them, although, like all things, they can be abused:
They grow wildly large, so that most problems that use the class don't use 99% of the functions.
They grow wildly large, so that 90% of the functions aren't used by any program still in use.
Often they are a dumping ground for functions which are specific to one domain. They should be pared off to a similar class use just by program in that domain. Often, these function would be better off incorporated into proper classes.
I used to have, in every project, a module called MiscStuffAndJunk. It was a place to hold everything that didn't have a clear place to go, either because the functionality was a one-off, or because I didn't want to change my focus, so as to do a proper design for a function that was needed by but extraneous away from what I was currently concentrating on.
Still, it these modules are clearly in violation of OO design principles.
So nowadays, I name the module StuffIHaventRefactoredYet, and all is right with the world.
Depending on what your static utility functions actually do and return, it may be cause problems unit testing. I have come across a method in a class that calls a static function on a static class that return things I do not want in my unit test, rendering the whole method untestable...