I have a dll that hides differences of different ADO.NET providers and has lots of code like:
private static void AppendProviderSpecificParameterCmdStr(StringBuilder sb, DbCommand cmd, string fieldNameToUse, ComparisonOperator oprtr, string parameterName)
{
if (cmd is System.Data.OracleClient.OracleCommand || cmd is Oracle.DataAccess.Client.OracleCommand)
{
sb.AppendFormat("{0}{1}:{2}", fieldNameToUse, GetComparisonOperatorStr(oprtr, cmd), parameterName);
}
else if (cmd is SqlCommand)
{
sb.AppendFormat("{0}{1}#{2}", fieldNameToUse, GetComparisonOperatorStr(oprtr, cmd), parameterName);
}
else if (cmd is OleDbCommand)
{
sb.AppendFormat("{0}{1}?", fieldNameToUse, GetComparisonOperatorStr(oprtr, cmd));
}
else
{
throw new Exception(string.Format("Wrong database command type: {0},", cmd.GetType()));
}
}
where Comparison operator is my own enum.
Oracle.DataAccess is present on all machines that have oracle client and this code have been ok for my needs. However now I've faced a situation where there is only SqlClient and they have no need to have oracle at all. So my code works only if I copy Oracle.DataAccess.dll which is naturally a horrible solution. How this should be done the correct way?
Thanks -matti
I wouldn't call a dependency on dll a horrible solution. Your solution supports Oracle and consequently you have an oracle dll in your solution - it is what it is.
That said, there are things you could do abstract away the command type.
One - create complete data access methods that implement an interface. Your current solution I'd classify as more of a helper or utility method for generic data access. You could instead declare an interface specific instead to a domain - customer for example - like ICustomerDA. In your case you'd have 3 implementations of ICustomerDA.Insert, with the database specifics buried inside. Your main code would only need to know about ICustomerDA. This is probably what I would do in a larger solution as differences and features between RDBMSs go well beyond parameter declaration.
Two - If you wanted to stick with more of the helper/utility idea, you could create an interface for a wrapper for db objects, say IDBCommand. Implementations IDBCommand would hide the underlying command object, and then have specific implementations of an .AppendProviderSpecificParameterCmdStr method which would allow you to do something like:
OracleDbHelper : IDbCommand...
public void AppendProviderSpecificParameterCmdStr(...){
sb.AppendFormat("{0}{1}:{2}", fieldNameToUse, GetComparisonOperatorStr(oprtr, cmd), parameterName);
}
IDBCommand cmd = DAFactory.GetCommand();
cmd.AppendProviderSpecificParameterCmdStr(...
The key to both of these solutions is referencing by a common interface from your main project rather than individual types. Once you did this, you could use reflection in your factory or better yet, something like MEF to create the actual types.
So my code works only if I copy Oracle.DataAccess.dll
Not unless you also have the native OCI DLLs, for example because you have already installed the Oracle Client.
To avoid forcing your users to install the full Oracle Client, you can distribute the DLLs from the Oracle Instant Client together with the application. If user never chooses to connect to Oracle, these DLLs are never called and just sit there quietly without causing any trouble.
For some hints on what to distribute and how to cover both 32-bit and 64-bit, take a look at this post.
We have a home-grown abstraction layer that currently works with Oracle and MS SQL Server (and is portable to any DBMS with a decent ADO.NET provider), and this system has worked quite well so far.
Related
A common way to interact with a SQL database in Go is to use the built in database/sql interface. Many different third-party packages implement this interface in a way that is specific to some particular database without exposing that work to you as a consumer, e.g. Postgres driver, MySQL driver, etc.
However, database/sql doesn't provide any specific error types, leaving it up to the driver instead. This presents a problem: any error handling you do for these specific errors beyond nil checks now works off of the assumption of a particular driver. If you decide to change drivers later, all of the error handling code must be modified. If you want to support multiple drivers, you need to write additional checks for that driver too.
This seemingly undermines the primary benefit of using interfaces: portability with an agreed-upon contract.
Here's an example to illustrate this problem using the jackc/pgx/v4/stdlib driver and suite of helper packages:
import (
"database/sql"
"errors"
"github.com/jackc/pgconn"
"github.com/jackc/pgerrcode"
)
// Omitted code for the sake of simplification, err comes from database/sql
if err != nil {
var pgerr *pgconn.PgError
if errors.As(err, &pgerr) {
if pgerrcode.IsIntegrityConstraintViolation(pgerr.SQLState()) {
return nil, errors.New("related entity does not exist")
}
}
// If we wanted to support another database driver, we'd have to include that here
return nil, errors.New("failed to insert the thing")
}
If I already have to put driver-specific code into my package, why bother accepting the database/sql interface at all? I could instead require the specific driver, which is arguably safer since it prevents the consumer from trying to use some other unsupported driver that we don't have error handling for.
Is there better way to handle specific database/sql errors?
You don't need driver specific code to get SQLState. Example:
func getSQLState(err error) {
type checker interface {
SQLState() string
}
pe := err.(checker)
log.Println("SQLState:", pe.SQLState())
}
But SQLState is a database specific anyway. If you switch to another database/driver in the future then you need to change all error codes manually. Compiler would not help to detect it.
Package sql provides a generic interface around SQL (or SQL-like) databases.
There is a compromise between providing the minimal common set of features, and providing features that would not be available for all implementations. The sql package has prioritized the former, while maybe you prefer more of the latter.
You could argue that every possible implementation should be able to provide a specific error for your example. Maybe that's the case. Maybe not. I don't know.
Either way it is possible for you to wrap pgerrcode.IsIntegrityConstraintViolation inside a function that does this check for every driver that you support. Then it is up to you to decide how to deal with drivers that lacks support.
Background
I'm in the process of reworking and refactoring a huge codebase which was written with neither testability nor maintainability in mind. There is a lot of global/static state going on. A function needs a database connection, so it just conjures one up using a global static method: $conn = DatabaseManager::getConnection($connName);. Or it wants to load a file, so it does it using $fileContents = file_get_contents($hardCodedFilename);.
Much of this code does not have proper tests and has only ever been tested directly in production. So the first thing I am intending on doing is write unit tests, to ensure the functionality is correct after refactoring. Now sadly code like the examples above is barely unit testable, because none of the external dependencies (database connections, file handles, ...) can be properly mocked.
Abstraction
To work around this I have created very thin wrappers around for example the system functions, that can be used in places where non-mockable function calls were used before. (I'm giving these examples in PHP, but I assume they are applicable for any other OOP language as well. Also this is a highly shortened example, in reality I am dealing with much larger classes.)
interface Time {
/**
* Returns the current time in seconds since the epoch.
* #return int for example: 1380872620
*/
public function current();
}
class SystemTime implements Time {
public function current() {
return time();
}
}
These can be used in the code like so:
class TimeUser {
/**
* #var Time
*/
private $time;
/**
* Prints out the current time.
*/
public function tellsTime() {
// before:
echo time();
// now:
echo $this->time->current();
}
}
Since the application only depends on the interface, I can replace it in a test with a mocked Time instance, which for example allows to predefine the value to return for the next call to current().
Injection
So far so basic. My actual question is how to get the proper instances into the classes that depend upon them. From my Understanding of Dependency injection, services are meant to be passed down by the application into the components that need them. Usually these services would be created in a {{main()}} method or at some other starting point and then strung along until they reach the components where they are needed.
This model likely works well when creating a new application from scratch, but for my situation it's less than ideal, since I want to move gradually to a better design. So I've come up with the following pattern, which automatically provides the old functionality while leaving me with the flexibility of substituting services.
class TimeUser {
/**
* #var Time
*/
private $time;
public function __construct(Time $time = null) {
if ($time === null) {
$time = new SystemTime();
}
$this->time = $time;
}
}
A service can be passed into the constructor, allowing for mocking of the service in a test, yet during "regular" operation, the class knows how to create its own collaborators, providing a default functionality, identical to what was needed before.
Problem
I've been told that this approach is unclean and subverts the idea of dependency injection. I do understand that the true way would be to pass down dependencies, like outlined above, but I don't see anything wrong with this simpler approach. Keep in mind also that this is a huge system, where potentially hundreds of services would need to be created up front (Service locator would be an alternative, but for now I am trying to go this other direction).
Can someone shed some light onto this issue and provide some insight into what would be a better way to achieve a refactoring in my case?
I think You've made first good step.
Last year I was on DutchPHP and there was a lecture about refactoring, lecturer described 3 major steps of extracting responsibilyty froma god class:
Extract code to private method (it should be simple copy paste since
$this is the same)
Extract code to separate class and pull
dependency
Push dependency
I think you are somewhere between 1st and 2nd step. You have a backdoor for unit tests.
Next thing according to above algorithm is to create some static factory (lecturer named it ApplicationFactory) which will be used instead of creation of instance in TimeUser.
ApplicationFactory is some kind of ServiceLocator pattern. This way you will inverse dependency (according to SOLID principle).
If you are happy with that you should remove passing Time instance into constructor and use ServiceLocator only (without backdoor for unit tests, You should stub service locator)
If you are not, then You have to find all places where TimeUser is being instantiated and inject Time implemenation:
new TimeUser(ApplicationFactory::getTime());
After some time yours ApplicationFactory will become very big. Then You have to made a decision:
Split it into smaller factories
Use some dependency injection container (Symfony DI, AurynDI or
something like that)
Currently my team is doing something similar. We are extracting responsibilities to seperate classes and inject them. We have an ApplicationFactory but we use it as service locator at as hight level as possible so classes bellow gets all dependencies injected and don't know anything about ApplicationFactory. Our application factory is big and now we are preparing to replace it with SymfonyDI.
You asked for a good mechanism to do this.
You've described some stages you might force the program to go through to accomplish this, but you are still apparantly planning to do this by hand at apparantly a very high cost.
If you really want to get this done on a huge code base, you might consider automating the steps using a program transformation engine: http://en.wikipedia.org/wiki/Program_transformation
Such a tool can let you write explicit rules for modifying code. Done right, this can make code changes reliably. That doesn't minimize your need for testing, but can let you spend more time writing tests and less time hand-changing the code (erroneously).
I'm not so sure the title is a good match for this question I want to put on the table.
I'm planning to create a web MVC framework as my graduation dissertation and in a previous conversation with my advisor trying to define some achivements, he convinced me that I should choose a modular design in this project.
I already had some things developed by then and stopped for a while to analyze how much modular it would be and I couldn't really do it because I don't know the real meaning of "modular".
Some things are not very cleary for me, like for example, just referencing another module blows up the modularity of my system?
Let's say I have a Database Access module and it OPTIONALY can use a Cache module for storing results of complex queries. As anyone can see, I at least will have a naming dependency for the cache module.
In my conception of "modular design", I can distribute each component separately and make it interact with others developed by other people. In this case I showed, if someone wants to use my Database Access module, they will have to take the Cache as well, even if he will not use it, just for referencing/naming purposes.
And so, I was wondering if this is really a modular design yet.
I came up with an alternative that is something like creating each component singly, without don't even knowing about the existance of other components that are not absolutely required for its functioning. To extend functionalities, I could create some structure based on Decorators and Adapters.
To clarify things a little bit, here is an example (in PHP):
Before
interface Cache {
public function isValid();
public function setValue();
public function getValue();
}
interface CacheManager {
public function get($name);
public function put($name, $value);
}
// Some concrete implementations...
interface DbAccessInterface {
public doComplexOperation();
}
class DbAccess implements DbAccessInterface {
private $cacheManager;
public function __construct(..., CacheManager $cacheManager = null) {
// ...
$this->cacheManager = $cacheManager;
}
public function doComplexOperation() {
if ($this->cacheManager !== null) {
// return from cache if valid
}
// complex operation
}
}
After
interface Cache {
public function isValid();
public function setValue();
public function getValue();
}
interface CacheManager {
public function get($name);
public function put($name, $value);
}
// Some concrete implementations...
interface DbAccessInterface {
public function doComplexOperation();
}
class DbAccess implements DbAccessInterface {
public function __construct(...) {
// ...
}
public function doComplexQuery() {
// complex operation
}
}
// And now the integration module
class CachedDbAcess implements DbAccessInterface {
private $dbAccess;
private $cacheManager;
public function __construct(DbAccessInterface $dbAccess, CacheManager $cacheManager) {
$this->dbAccess = $dbAccess;
$this->cacheManager = $cacheManager;
}
public function doComplexOperation() {
$cache = $this->cacheManager->get("Foo")
if($cache->isValid()) {
return $cache->getValue();
}
// Do complex operation...
}
}
Now my question is:
Is this the best solution? I should do this for all the modules that do not have as a requirement work together, but can be more efficient doing so?
Anyone would do it in a different way?
I have some more further questions involving this, but I don't know if this is an acceptable question for stackoverflow.
P.S.: English is not my first language, maybe some parts can get a little bit confuse
Some resources (not theoretical):
Nuclex Plugin Architecture
Python Plugin Application
C++ Plugin Architecture (Use NoScript on that side, they have some weird login policies)
Other SO threads (design pattern for plugins in php)
Django Middleware concept
Just referencing another module blows up the modularity of my system?
Not necessarily. It's a dependency. Having a dependencies is perfectly normal. Without dependencies modules can't interact with each other (unless you're doing such interaction indirectly which in general is a bad practice since it hides dependencies and complicates the code). Modular desing implies managing of dependencies, not removing them.
One tool - is using interfaces. Referencing module via interface makes a so called soft dependency. Such module can accept any implementation of an interface as a dependency so it is more independant and as a result - more maintainable.
The other tool - designing modules (and their interfaces) that have only single responcibility. This also makes them more granular, independant and maintainable.
But there is a line which you should not cross - blindly applying these tools may leed to a too modular and too generic desing. Making things too granular makes the whole system more complex. You should not solve universe problems, making generic modules, that all developers can use (unless it is your goal). First of all your system should solve your domain tasks and make things generic enough, but not more than that.
I came up with an alternative that is something like creating each component singly, without don't even knowing about the existance of other components that are not absolutely required for its functioning
It is great if you came up with this idea by yourself. The statement itself, is a key to modular programming.
Plugin architecture is the best in terms of extensibility, but imho it is hard to maintenance especially in intra application. And depending the complexity of plugin architecture, it can make your code more complex by adding plugin logics, etc.
Thus, for intra modular design, I choose the N-Tier, interface based architecture. Basically, the architecture relays on those tiers:
Domain / Entity
Interface [Depend on 1]
Services [Depend on 1 and 2]
Repository / DAL [Depend on 1 and 2]
Presentation Layer [Depend on 1,2,3,4]
Unfortunately, I don't think this is achieveable neatly in php projects as it need separated project / dll references in each tier. However, following the architecture can help to modularize the application.
For each modules, we need to do interface-based design. It can help to enhance the modularity of your code, because you can change the implementation later, but still keep the consumer the same.
I have provided an answer similiar to this interface-based design, at this stackoverflow question.
Lastly but not least, if you want to make your application modular to the UI, you can do Service Oriented Architecture. This is simply make your application as bunch of services, and then make the UI to consume the service. This design can help to separate your UI with your logic. You can later use different UI such as desktop app, but still use the same logic. Unfortunately, I don't have any reliable source for SOA.
EDIT:
I misunderstood the question. This is my point of view about modular framework. Unfortunately, I don't know much about Zend so I will give examples in C#:
It consist of modules, from the smallest to larger modules. Example in C# is you can using the Windows Form (larger) at your application, and also the Graphic (smaller) class to draw custom shapes in the screen.
It is extensible, or replaceable without making change to base class. In C# you can assign FormLoad event (extensible) to the Form class, inherit the Form or List class (extensible) or overridding form draw method to create a custom window graphic (replaceable).
(optional) it is easy to use. In normal DI interface design, we usually inject smaller modules into a larger (high level) module. This will require an IOC container. Refer to my question for detail.
Easy to configure, and does not involve any magical logic such as Service Locator Pattern. Search Service Locator is an Anti Pattern in google.
I don't know much about Zend, however I guess that the modularity in Zend can means that it can be extended without changing the core (replacing the code) inside framework.
If you said that:
if someone wants to use my Database Access module, they will have to take the Cache as well, even if he will not use it, just for referencing/naming purposes.
Then it is not modular. It is integrated, means that your Database Access module will not work without Cache. In reference of C# components, it choose to provide List<T> and BindingList<T> to provide different functionality. In your case, imho it is better to provide CachedDataAccess and DataAccess.
At the moment, we've got an unmaintanable ball of code which offers an interface to a third party application. The third party application has a COM assembly which MUST be used to create new entries. This process involves two steps: generate a new object (basically an ID), and update that object with new field values.
Because COM interop is so slow, we only use that to generate the ID (and related objects) in the database. The actual update is done using a regular SQL query.
What I am trying to figure out if it's possible to use NHibernate to do some of the heavy lifting for us, without bypassing the COM assembly. Here's the code for saving something to the database as I envision it:
using(var s = sessionFactory.OpenSession())
using(var t = s.BeginTransaction())
{
MyEntity entity = new MyEntity();
s.Save(entity);
t.Commit();
}
Regular NH code I'd say. Now, this is where it gets tricky. I think I have to supply my own implementation of NHibernate.Id.IIdentifierGenerator which calls the COM assembly in the Generate method. That's not a problem. What IS a problem is that the COM assembly requires initialisation, which does take a bit of time. It also doesn't like multiple instances in the same process, for some reason.
What I would like to know is if there's a way to properly access an external service in the generator code.
I'm free to use any technique I want, so if it involves something like an IoC container that's no problem. The thing I am looking for is where exactly to hook-up my code so I can access the things I need in my generator, without having to resort to using singletons or other nasty stuff.
Assuming you want a single instance of the COM component for all of your entities, a Service Locator (implemented with any IoC container) is the way to go.
Your (trivial) IIdentifierGenerator implementation would have to call the locator in its constructor to get the component instance, and call the corresponding method on the component on Generate.
One thing to consider: is the COM component thread-safe? If not, you should synchronize the Generate calls.
Usage of the generator is simple, just use the fully-qualified name as your generator class.
So, I was reading the Google testing blog, and it says that global state is bad and makes it hard to write tests. I believe it--my code is difficult to test right now. So how do I avoid global state?
The biggest things I use global state (as I understand it) for is managing key pieces of information between our development, acceptance, and production environments. For example, I have a static class named "Globals" with a static member called "DBConnectionString." When the application loads, it determines which connection string to load, and populates Globals.DBConnectionString. I load file paths, server names, and other information in the Globals class.
Some of my functions rely on the global variables. So, when I test my functions, I have to remember to set certain globals first or else the tests will fail. I'd like to avoid this.
Is there a good way to manage state information? (Or am I understanding global state incorrectly?)
Dependency injection is what you're looking for. Rather than have those functions go out and look for their dependencies, inject the dependencies into the functions. That is, when you call the functions pass the data they want to them. That way it's easy to put a testing framework around a class because you can simply inject mock objects where appropriate.
It's hard to avoid some global state, but the best way to do this is to use factory classes at the highest level of your application, and everything below that very top level is based on dependency injection.
Two main benefits: one, testing is a heck of a lot easier, and two, your application is much more loosely coupled. You rely on being able to program against the interface of a class rather than its implementation.
Keep in mind if your tests involve actual resources such as databases or filesystems then what you are doing are integration tests rather than unit tests. Integration tests require some preliminary setup whereas unit tests should be able to run independently.
You could look into the use of a dependency injection framework such as Castle Windsor but for simple cases you may be able to take a middle of the road approach such as:
public interface ISettingsProvider
{
string ConnectionString { get; }
}
public class TestSettings : ISettingsProvider
{
public string ConnectionString { get { return "testdatabase"; } };
}
public class DataStuff
{
private ISettingsProvider settings;
public DataStuff(ISettingsProvider settings)
{
this.settings = settings;
}
public void DoSomething()
{
// use settings.ConnectionString
}
}
In reality you would most likely read from config files in your implementation. If you're up for it, a full blown DI framework with swappable configurations is the way to go but I think this is at least better than using Globals.ConnectionString.
Great first question.
The short answer: make sure your application is a function from ALL its inputs (including implicit ones) to its outputs.
The problem you're describing doesn't seem like global state. At least not mutable state. Rather, what you're describing seems like what is often referred to as "The Configuration Problem", and it has a number of solutions. If you're using Java, you may want to look into light-weight injection frameworks like Guice. In Scala, this is usually solved with implicits. In some languages, you will be able to load another program to configure your program at runtime. This is how we used to configure servers written in Smalltalk, and I use a window manager written in Haskell called Xmonad whose configuration file is just another Haskell program.
An example of dependency injection in an MVC setting, here goes:
index.php
$container = new Container();
include_file('container.php');
container.php
container.add("database.driver", "mysql");
container.add("database.name","app");
...
$container.add(new Database($container->get('database.driver', "database.name")), 'database');
$container.add(new Dao($container->get('database')), 'dao');
$container.add(new Service($container->get('dao')));
$container.add(new Controller($container->get('service')), 'controller');
$container.add(new FrontController(),'frontController');
index.php continues here:
$frontController = $container->get('frontController');
$controllerClass = $frontController->getController($_SERVER['request_uri']);
$controllerAction = $frontController->getAction($_SERVER['request_uri']);
$controller = $container->get('controller');
$controller->$action();
And there you have it, the controller depends on a service layer object which depends on
a dao(data access object) object which depends on a database object with depends on the
database driver, name etc