Is there an article or resource on getting started with BookSleeve Redis? Thanks
These are some of the articles/links I read when getting upto speed:
http://marcgravell.blogspot.com/2011/04/async-redis-await-booksleeve.html
http://marcgravell.blogspot.com/2011/07/booksleeve-transactions-hashes-and.html
https://code.google.com/p/booksleeve/
http://redis.io/
besides that, search stackoverflow for related questions:
https://stackoverflow.com/questions/tagged/booksleeve
There is not a ton out there, you need to roll up your sleeves and dig in.
Book sleeve is just a wrapper around redis, so any data decisions really relate equally to any redis guide. http://redis.io would be a starting place. In terms of the API, the only things that should need highlighting are:
most operations are async and are exposed via the Task api; this allows a range of usage patterns including (but not limited to) the new "async"/"await" keywords
the operations are split by data type, so .Strings, .Hashes, etc
a connection is a thread-safe multiplexer and is designed to be used by multiple callers concurrently - you do not (and should not) need to use a connection scoped per caller
I am interested in what things you would want me to include in more detailed documentation. What would be useful? What is unclear? What do you need?
If you can describe what you need, I can write it :p
Related
The last 6 months learning curve have been challenging with CQRS and DDD the main culprits.
It has been fun and we are 1/2 way through our project and the area I have not had time to delve into is a messaging framework.
Currently I don't use DTC so there is a very good likely hood that if my read model is not updated then I will have inconsistency between the read and write databases. Also my read and write database will be on the same machine. I doubt we will ever put them on separate machines.
I don't have a large volume of messages in my system so my concern is more to do with consistency and reliability of the system.
So, do I have to put in a messaging framework like NServiceBus (even though both read and write databases are on the same machine) or do I have other options? Yes there is learning curve but I suppose there would be a hell of a lot to learn if I don't use it.
Also, I don't want to put in a layer if it is not necessary
Thoughts?
Currently I don't use DTC so there is a very good likely hood that if
my read model is not updated then I will have inconsistency between
the read and write databases.
Personally, I dislike the DTC and try to avoid it. Instead, it is often possible to implement a compensation mechanism, especially for something like a read model where eventual consistency is already acceptable and updates are idempotent. For example, you could implement a version on entities and have a background task which ensures versions are in-sync. Having a DTC will provide transactional retry functionality, but it still won't solve cases where failure occurs after retries - you still have to watch the error log and have procedures in place to deal with errors.
So, do I have to put in a messaging framework like NServiceBus (even
though both read and write databases are on the same machine) or do I
have other options?
It depends on a few things. What you often encounter in a CQRS system is need for pub/sub where several sub-systems publish events to which the query/caching system subscribes to. If you see a need for pub/sub beyond basic point-to-point messaging, then go with something like NServiceBus. Also, I wouldn't immediately shy away from using NServiceBus even if you don't need it for scalability purposes because I think the logical partitioning is beneficial on its own. On the other hand, as you point out, adding layers of complexity is costly, therefore first try to see if the simplest possible thing will work.
Another question to ask is whether you need a separate query store at all. If all you have is a single machine, why bother? You could use something simpler like the read-model pattern and still reap a lot of the benefits of CQRS.
Does a CQRS project need a messaging framework like NServiceBus?
The short answer: no.
It is the first time I hear about the 'read-model pattern' mentioned by eulerfx. It is a nice enough name but there is a bit more to it:
The general idea behind the 'query' part is to query a denormalized view of your data. In the 'read-model pattern' link you will notice that the query used to populate the read-model is doing some lifting. In the mentioned example the required data manipulation is not that complex but what if it does become more complex? This is where the denomalization comes in. When you perform your 'command' part the next action is to denormalize the data and store the results for easy reading. All the heavy lifting should be done by your domain.
This is why you are asking about the messaging. There are several techniques here:
denormalized data in same database, same table, different columns
denormalized data in same database, different table
denormalized data in different database
That's the storage. How about the consistency?:
immediately consistent
eventually consistent
The simplest solution (quick win) is to denormalize your data in your domain and then after saving your domain objects through the repository you immediately save the denomarlized data to the same data store, same table(s), different columns. 100% consistent and you can start reading the denormalized data immediately.
If you really want to you can create a separate bunch of objects to transport that data but it is simpler to just write a simple query layer that returns some data carrying object provided by your data-access framework (in the case of .Net that would be a DataRow/DataTable). Absolutely no reason to get fancy. There will always be exceptions but then you can go ahead and write a data container.
For eventual consistency you will need some form of queuing and related processing. You can roll your own solution or your may opt for a service bus. That is up to you and your time / technical constraints :)
BTW: I have a free open-source service bus here:
Shuttle.Esb
documentation
Any feedback would be welcomed. But any old service bus will do (MassTransit / NServiceBus / etc.).
Hope that helps.
Not a specific question as such, I'm more trying to test the waters. I like distributed objects, and I like grand central dispatch; How about I try to combine the two?
Does that even make sense? Has anyone played around in these waters? Would I be able to use GCD to help synchronize object access across machines? Or would it be better to stick to synchronizing local objects only? What should I look out for? What design patterns are helpful and what should I avoid?
as an example, I use GCD queues to synchronize accesses to a shared resource of some kind. What can I expect to happen if I make this resource public via distributed objects? Questions like: How nicely to blocks play with distributed objects? Can I expect to use the everything as normal across machines? If not, can I wrangle it to do so? What difficulties can I expect?
I very much doubt this will work well. GCD objects are not Cocoa objects, so you can't reference them remotely. GCD synchronization primitives don't work across process boundaries.
While blocks are objects, they do not support NSCoding, so they can't be transmitted across process boundaries. (If you think about it, they are not much more than function pointers. The pointed-to function must have been compiled into the executable. So, it doesn't make sense that two different programs would share a block.)
Also, Distributed Objects depends on the connection being scheduled in a given run loop. Since you don't manage the threads used by GCD, you are not entitled to add run loop sources except temporarily.
Frankly, I'm not even sure how you envision it even theoretically working. What do you hope to do? How do you anticipate it working?
Running across machines -- as in a LAN, MAN, or WAN?
In a LAN, distributed objects will probably work okay as long as the server you are connecting to is operational. However, most programmers you meet will probably raise an eyebrow and just ask you, "Why didn't you just use a web server on the LAN and just build your own wrapper class that makes it 'feel' like Distributed Objects?" I mean, for one thing, there are well-established tools for troubleshooting web servers, and it's easier and often cheaper to hire someone to build a web service for you rather than a distributed object server.
On a MAN or WAN, however, this would be slow and a very bad idea for most uses. For that type of communication, you're better off using what everyone else uses -- REST-like APIs with HTTPS/HTTP, sending either XML, JSON, or key/value data back and forth. So, you could make a class wrapper that makes this "feel" sort of like distributed objects. And my gut feeling tells me that you'll need to use tricks to speed this up, such as caching chunks of data locally on the client so that you don't have to keep fetching from the server, or even caching on the server so that it doesn't have to interact with a database as often.
GCD, Distributed Objects, Mach Ports, XPC, POSIX Message Queues, Named Pipes, Shared Memory, and many other IPC mechanisms really only make the most sense on local, application to application communication on the same computer. And they have the added advantage of privilege elevation if you want to utilize that. (Note, I said POSIX Message Queues, which are workstation-specific. You can still use a 'message queue service' on a LAN, MAN, or WAN -- there are many products available for that.)
I'm looking for articles and references that give an overview of 'queueing' (I'm probably not even using the right term here). I'm hoping for an introductory styled guide through a world of Redis, RabbitMQ, Celery, Kombu, and whatever other components exist that I haven't read about yet, and how they fit together.
My problem is I need to queue up background tasks for issued by my Django website, and every blog and article I read recommend different solutions.
Lots of options available to you, and your choice will likely come down to personal preference and what dependencies you feel comfortable installing.
I'll give a vote for Redis. I evaluated RabbitMQ, ActiveMQ, HornetQ, and Redis and found Redis to offer the best mix of ease of installation, simplicity, and performance.
It's technically not a message queue, but the push/pop primitives for the list types provide atomic queue-like operations, so it can effectively be used as a queue. It has worked well for us.
One python specific project on top of Redis you might look at:
http://richardhenry.github.com/hotqueue/tutorial.html
Very simple. But again, all the other options, like Celery, are viable too.
RabbitMQ has a good introduction here: http://www.rabbitmq.com/getstarted.html There's examples in Python, even.
HornetQ has a very good documentation, and it's simple to install.
You can find the documentation at www.hornetq.org, and you would have several examples available with the distribution.
I am writing my first objective-c daemon type process that works in the background. Everything it does needs to be logged properly.
I am fairly new to Apple stuff so I am not sure, what is the most common and/or best way to log activity? Does everyone simply log to a text file in their own special format, or use some sort of system call?
You should look at the Apple System Logger. ASL writes to the system log database (making it easy to query the log from Console.app or from within your own app) and additionally to one or more flat files (if you choose). Peter Hosey's introduction to the ASL is the best I'm aware of. ASL is a C-level API, but it's relatively easy to wrap in Objective-C if you'd like. I would recommend also taking a look at Google's Toolbox for Mac. Among many other goodies, it contains a GTMLogger facility that includes ASL support. I've ditched my home-grown ASL wrapper in favor of the GTMLogger.
Another alternative you might want to try is https://github.com/CocoaLumberjack. Lumberjack is quite flexible and will allow you to log to various destinations, configure log levels, etc. It's very log4j / log4net like, if you are familiar with those.
It's also reports that it is faster than ASL... I don't know how it compares to GTMLogger with respect to functionality or speed, but the documentation seems to be a bit more approachable.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
Background:
I am taking a class at my university called "Software Constraints". In the first lectures we were learning how to build good APIs.
A good example we got of a really bad API function is the socket public static void Select(IList checkRead, IList checkWrite, IList checkError, int microseconds); in C#. The function receives 3 lists of sockets, and destroys them making the user have to clone all the sockets before feeding them into the Select(). It also has a timeout (in microseconds) which is an int, that sets the maximum time the server can wait for a socket. The limits of this is +/-35 minutes (because it is an int).
Questions:
How do you define an API as
'bad'?
How do you define an
API as 'good'?
Points to consider:
Function names that are hard to remember.
Function parameters that are hard to understand.
Bad documentation.
Everything being so interconnected that if you need to change 1 line of code you will actually need to change hundreds of lines in other places.
Functions that destroy their arguments.
Bad scalability due to "hidden" complexity.
It's required from the user/dev to build wrappers around the API so that it can be used.
In API design I've always found this keynote very helpful:
How to Design a Good API and Why it Matters - by Joshua Bloch
Here's an excerpt, I'd recommend reading the whole thing / watching the video.
II. General Principles
API Should Do One Thing and Do it Well
API Should Be As Small As Possible But No Smaller
Implementation Should Not Impact API
Minimize Accessibility of Everything
Names Matter–API is a Little Language
Documentation Matters
Document Religiously
Consider Performance Consequences of API Design Decisions
Effects of API Design Decisions on Performance are Real and Permanent
API Must Coexist Peacefully with Platform
III. Class Design
Minimize Mutability
Subclass Only Where it Makes Sense
Design and Document for Inheritance or Else Prohibit it
IV. Method Design
Don't Make the Client Do Anything the Module Could Do
Don't Violate the Principle of Least Astonishment
Fail Fast - Report Errors as Soon as Possible After They Occur
Provide Programmatic Access to All Data Available in String Form
Overload With Care
Use Appropriate Parameter and Return Types
Use Consistent Parameter Ordering Across Methods
Avoid Long Parameter Lists
Avoid Return Values that Demand Exceptional Processing
You don't have to read the documentation to use it correctly.
The sign of an awesome API.
Many coding standards and longer documents and even books (Framework Design Guidelines) have been written on this topic, but much of this only helps at a fairly low level.
There is also a matter of taste. APIs may obey every rule in whatever rulebook, and still suck, due to slavish adherence to various in-vogue ideologies. A recent culprit is pattern-orientation, wherein Singleton Patterns (little more than initialized global variables) and Factory Patterns (a way of parameterizing construction, but often implemented when not needed) are overused. Lately, it's more likely that Inversion of Control (IoC) and associated explosion in the number of tiny interface types that adds redundant conceptual complexity to designs.
The best tutors for taste are imitation (reading lots of code and APIs, finding out what works and doesn't work), experience (making mistakes and learning from it) and thinking (don't just do what's fashionable for its own sake, think before you act).
Useful - it addresses a need that is not already met (or improves on existing ones)
Easy to explain - the basic understanding of what it does should be simple to grasp
Follows some object model of some problem domain or real-world. It uses constructs that make sense
Correct use of synchronous and asynchronous calls. (don't block for things that take time)
Good default behavior - where possible allow extensibility and tweaking, but provide defaults for all that is necessary for simple cases
Sample uses and working sample applications. This is probably most important of all.
Excellent documentation
Eat your own dog food (if applicable)
Keep it small or segment it so that it is not one huge polluted space. Keep functionality sets distinct and isolated with few if any dependencies.
There are more, but that is a good start
A good API has a semantic model close to the thing it describes.
For example, an API for creating and manipulating Excel spreadsheets would have classes like Workbook, Sheet, and Cell, with methods like Cell.SetValue(text) and Workbook.listSheets().
A good API allows the client to do pretty much everything they need to do, but doesn't require them to do a lot of mindless busy-work. Examples of "mindless busy work" would be initializing data structure fields, calling several routines in a sequence that never varies with no real custom code in between, etc.
The surest sign of a bad API is if your clients all want to wrap it with their own helper code. At the absolute least, your API should have provided that helper code. Most likely, it should have been designed to provide the higher level of abstraction the clients are rolling themselves on their own every time.
A bad API is one that is not used by its intended audience.
A good API is one that is used by its intended audience for the purpose for which it was designed.
A great API is one that is used by both its intended audience, for the its intended purpose, and an unintended audience for reasons unanticipated by its designers.
If Amazon publishes its API as both SOAP and REST, and the REST version wins out, that doesn't mean the underlying SOAP API was bad.
I'd imagine that the same will be true for you. You can read all you want about design and try your best, but the acid test will be use. Spend some time building in ways to get feedback on what works and what doesn't and be prepared to refactor as needed to make it better.
A good API is one that makes simple things simple (minimum boilerplate and learning curve to do the most common things) and complicated things possible (maximum flexibility, as few assumptions as possible). A mediocre API is one that does one of these well (either insanely simple but only if you're trying to do really basic stuff, or insanely powerful, but with a really steep learning curve, etc). A horrible API is one that does neither of these well.
There are several other good answers on this already, so I thought I'd just throw in some links I didn't see mentioned.
Articles
"A Little Manual Of API Design" by Jasmin Blanchette of Trolltech
"Defining QT-Style C++ APIs" also Trolltech
Books:
"Effective Java" by Joshua Bloch
"The Practice Of Programming" by Kernighan and Pike
I think a good API should allow custom IO and memory management hooks if it's applicable.
A typical example is you have your custom compressed archive format for data on disk and a third party library with a poor api wants to access data on disk and expects a path to a file where it can load its data.
This link has some good points:
http://gamearchitect.net/2008/09/19/good-middleware/
If the API produces an error message, ensure that the message and diagnostics help a developer work out what the problem is.
My expectation is that the caller of an API passes in input that is correct. A developer is the consumer of any error messages produced by the API (not the end user), and messages aimed at the developer help the developer debug their calling program.
An API is bad when it is badly documented.
An API is good when it is well documented and follows a coding standard.
Now these are two, very simple and also very hard points to follow, this brings one into the area of software architecture. You need a good architect that structures the system and helps the framework follow its own guidlines.
Commenting code, writing an well explained manual for the API is Mandatory.
An API can be good if it has a good documentation which explains how to use it. But if the code is clean, good and follows a standard inside itself, it doesnt matter if it doenst have a decent documentation.
I've written a little about coding structure here
I think what is paramount is readability, by which I mean the quality that makes the greatest number of programmers to make sense of what the code is doing in the shortest possible amount of time. But judging which piece of software is readable and which is not has that indescribable human quality: fuzziness. The points you mention do partly succeed in crystallizing it. However, in its entirety it has to remain a case-by-case affair and it would be really hard to come up with universal rules.