DAOs Aren't Dead - But They Either Collapsed Or Disappeared
What if:
- The vendor of your database changes?
- You will have to replace your relational database with LDAP, flat files or object oriented storage?
- You will have to replace JPA with iBatis or even plain JDBC?
- You will have to replace your local storage with remote service calls?
The questions above are interesting and valid, but how likely is such a case? The architecture (even real live) is all about probability. If something is likely to happen, we react to it. If something is rather unlikely, we tend to ignore it. In my past projects, only in (very) few cases we had to switch the database vendor.We never switched from a relational database to a flat file or LDAP. I guess it is unlikely in the future as well. Interestingly the real problem during the migration from one relational DB to another was not the leaky encapsulation of SQL statements or hiding classes behind interfaces, rather than different locking behavior, which just cannot be hidden or abstracted with Java-means. Most projects are concerned about the answers for the questions above and introduce DAOs, which are especially leaky by nature. Locking behavior is one thing, semantic behavior is even harder to abstract. A typical DAOs operates "per Value", whereby OR-tools work with managed / attached objects. If you start to emulate such a behavior for e.g. flat files, you will either fail, or eventually implement Hibernate 5.0.
My post "JPA/EJB 3 Killed The DAO" was not precise enough. In some situations even the classic DAOs are still a good solution, but for the vast majority of all cases and projects are absolutely not. If you are developing a product, like e.g. a CMS software, it is essential to be as flexible as possible to survive in a heterogenous market. A DAO could help you to abstract from the particular data source, but such abstractions are leaky by nature. The variations of different data sources is already predefined by the system requirements of your product. You have to protect your business logic from being dependent on the particular data store implementation. DAO can help you to abstract from the variations. Even so, in most commercial products a particular database tend to be better supported, than the other. Just look at the Hibernate, Toplink or openJPA support for a particular database.
In a usual projects there is no need to introduce a DAO as as standalone component. It is enough to rely on your common sense and try to remain DRY. Then you could easily use the EntityManager directly in your Service layer and if you notice some repetition or duplication just to refactor the code into one place. This can happens without any ceremony - you either delegate the calls to a class which encapsulates common queries, or you could even inherit such capabilities from a common class. You could call both strategies a "DAO" - in J2EE, however, you would start with a DAO in own layer, in Java EE a DAO can just evolve but is optional...
[see pages: 80, 141 and 261 in "Real World Java EE 6 - Rethinking Best Practices" book]
>> In a usual projects there is no need to introduce a DAO as as standalone component.
Adam, I disagree to this sentence ... though I agree with the whole rest of your article ;) ...
'usual project' for us means at least JUnit-Testable. And this (in my opinion) quite forces you (sooner or later) to introduce a DAO-layer, just because you need to 'delegate the calls to a class which encapsulates common queries'.
That of course is all that remains ouf 'our' DAOs ...
Furthermore this approach (i.e. forcing to use DAOs as encapsulation of queries) is
- no architecture overkill, it's just one more well defined class, with code, that'd be cluttered elsewhere anyway.
- totally in harmony with your article '15 tips on JPA Rich Domain Modelling' - if you really created such a domain model (JUnit-tests inclusive of course) you are forced to abstract the 'finding of objects' to a switchable class, to leave your higher level-tests without a db-connection.
Strange thing is - a lot of time ago we programmed with JDO1 - and without DAOs ...
I'm really afraid, that we do have migrate to JPA some time and now I wish we had DAOs ... ;)
All those PersistenceManagers everywhere in the code *shiver*
Posted by Stefan Hansel on February 05, 2009 at 10:52 AM CET #
>> In a usual projects there is no need to introduce a DAO as as standalone component.
Adam, I disagree to this sentence ... though I agree with the whole rest of your article ;) ...
'usual project' for us means at least JUnit-Testable. And this (in my opinion) quite forces you (sooner or later) to introduce a DAO-layer, just because you need to 'delegate the calls to a class which encapsulates common queries'.
That of course is all that remains ouf 'our' DAOs ...
Furthermore this approach (i.e. forcing to use DAOs as encapsulation of queries) is
- no architecture overkill, it's just one more well defined class, with code, that'd be cluttered elsewhere anyway.
- totally in harmony with your article '15 tips on JPA Rich Domain Modelling' - if you really created such a domain model (JUnit-tests inclusive of course) you are forced to abstract the 'finding of objects' to a switchable class, to leave your higher level-tests without a db-connection.
Posted by Stefan Hansel on February 05, 2009 at 10:54 AM CET #
"Your comment was marked as spam and will not be displayed. Comment has more than 1000 characters"
What a great spam detection ;) ...
post continued ...
------------
Strange thing is - a lot of time ago we programmed with JDO1 - and without DAOs ...
I'm really afraid, that we do have migrate to JPA some time and now I wish we had DAOs ... ;)
All those PersistenceManagers everywhere in the code *shiver*
Posted by stefan hansel on February 05, 2009 at 10:55 AM CET #
Stefan,
1. Yes, I have to disable the 1000 characters limitation somehow - it rollerweblogger default setting -> sorry for that.
2."usual project' for us means at least JUnit-Testable":
http://www.adam-bien.com/roller/abien/entry/schizophrenia_driven_design :-)
3. I (mis)use JUnit to test the Service with inherited or standalone DAO, which access a local (Derby, H2 or an virtualized image). I do not mock out the EntityManager. Because it is injected, however, you could easily mock it out.
If you really want to mock out your persistence layer - then a single generic DAO would be absolutely sufficient. In this case the testability and not the encapsulation are the main motivation behind DAOs. But then I would really insist on the tests in code reviews :-)
Posted by Adam Bien on February 05, 2009 at 11:20 AM CET #
I have to agree with Stefan. Unit-Testing is a main reason for keeping the DAO.
Testing your Business-Layer: Mocking a DAO is easier than mocking the EntityManager (and all the stuff that comes with it,like mocking a Query ...)
Testing your Data-Access: It has value to test your data-access separately (as you say yourself in '15 tips on JPA Rich Domain Modelling'). This is cleanly possible by institutionalizing DAO-Components that have their own Unit-Tests.
Also I still believe in 'Separation of Concern' to get a grip on complexity. I would argue that separating data-access out of your business code does usually decrease complexity of your business code.
Maybe we should think more in the sense of 'Repositories' (like in DDD) than 'DAOs'. Repositories are part of your domain-layer. They provide concrete business-semantics and are therefore on a higher abstraction level than an entity-manager...
Posted by Jonas on February 05, 2009 at 11:49 AM CET #
In my experience, DAO are not really used for abstraction over a repository we don't often change database (the ORM can take care of it anyway like hibernate's dialects) , nor we replace the database with a file repository or LDAP directory. We use DAO for :
- object design principles : 1 class 1 responsability. If data access code is put in the service layer it can quickly become bloated. Testing is also easier.
- having a library for data access that use domain objects and which can be used in different contexts like for batchs (to a certain limit) and
web applications.
Posted by Luc Dew on February 05, 2009 at 11:55 AM CET #
@All
if unit-testing is the driving force, then a single generic DAO, like e.g.: http://www.adam-bien.com/roller/abien/entry/generic_crud_components_with_java
would be enough. It is sufficient to deploy them once for all services (not a DAO for every service, or entity). It is rather a "Data Service", than a classic DAO. Then they almost disappeared :-),
regards,
adam
Posted by Adam Bien on February 05, 2009 at 12:20 PM CET #
I could live with a generic_crud_component for testing.
But then its matter of taste if you like to have a lot of constants for your named queries.
I personally like
SomeDAO {
findSomethingUsefull(String input1, String input2)
findLatestNegativeTransaction(Date searchFrom)
}
more.
Better place to have JavaDoc (for the functions) as well ...
I always have the 'Repository' from DDD in mind as well when thinking about what is left of the DAO.
Posted by stefan hansel on February 05, 2009 at 01:23 PM CET #
Stefan,
but something like: "findSomethingUsefull(String input1, String input2)" can be easily placed in the Service layer already. Otherwise most of the Service methods tend to be empty. The Service would delegate the invocations to a generic DAO. And you will get a dedicated place to write your javadocs. But keep in mind:
http://www.adam-bien.com/roller/abien/entry/javadoc_dry_for_comments_or
and
http://www.adam-bien.com/roller/abien/entry/how_to_javadoc_efficient_and
thanks for your feedback!,
regards,
adam
Posted by Adam Bien on February 05, 2009 at 01:39 PM CET #
To my mind now comes a pattern (officially supported by sun :D) called 'fast lane reader', see http://java.sun.com/blueprints/patterns/FastLaneReader.html
It's a bit outdated referencing entity beans, but anyway, why not say:
- all query-code naturally belongs to DAOs. This isn't any overhead, just some code structuring because we all like clean nice classes (interfaces are only introduced when really needed of course).
If needed we call those classes Repository ... you know - often it helps a lot just to rename stuff to make people comfortable
- all DAOs/Repositories are beans themselfes, a client just needing some data, doesn't need to ask some service ... it goes directly to the DAO (=fast lane reader adapted to ejb3)
- all remaining logic belongs to some service beans or is in the domain-objects themself
I see many advantages: it's as easy as doing queries directly in the service-classes, BUT ... any architect demanding 3-tier-architecture will be happy as well :D
Posted by stefan hansel on February 05, 2009 at 06:33 PM CET #
@Stefan,
"all DAOs/Repositories are beans themselfes, a client just needing some data, doesn't need to ask some service ... it goes directly to the DAO (=fast lane reader adapted to ejb3)"
Exactly - this is why I used "collapsed" in the title. DAOs collapse with Services for simpler user cases. For complex business logic DAO become just the product of refactoring.
"...any architect demanding 3-tier-architecture will be happy as well"
For such cases we should introduce some noop layers with fancier names like: orchestration, service and data service :-)
thanks again!,
adam
Posted by Adam Bien on February 05, 2009 at 06:57 PM CET #
"Then you could easily use the EntityManager directly in your Service layer and if you notice some repetition or duplication just to refactor the code into one place"
I tried this once and it killed testability. When I moved all data access code into DAOs, I was able to unit test my service layer in isolation. I use a GenericJpaDao, and for entities that had named queries I create a new DAO for them that extends the generic one, and adds methods for calling the named queries so that the caller doesn't see anything JPA.
Posted by Ryan de Laplante on February 06, 2009 at 03:28 AM CET #
Ryan,
...and what you did in the Service? :-)
regards,
adam
Posted by Adam Bien on February 06, 2009 at 10:27 AM CET #
Stefan, Adam:
In DDD Eric Evans talks about "Repositories" not DAO's that will do your suggested functionality...
Robert
Posted by Robert on February 06, 2009 at 11:40 AM CET #
Robert,
is an EntityManager already a repository :-)?
regards,
adam
Posted by Adam Bien on February 06, 2009 at 12:40 PM CET #
@Adam: What do you mean? The service knows nothing about the technologies that provide data access, it just uses DAOs. In fact there are about 5 DAOs the service uses. One of them talks to a JCA connector to a legacy EIS, and there are several other DAOs to read/write data to tables in several databases. The transaction demarkation is managed by the service. By completely decoupling the service from JPA and JCA connector I am able to cleanly unit test the service.
Posted by Ryan de Laplante on February 07, 2009 at 05:15 PM CET #
BTW the service is pure business logic. It is the brains of the system.
Posted by Release 4.0 Sprint 9 on February 07, 2009 at 05:16 PM CET #
@Ryan,
in your case: perfect. The service has dedicated responsibility. What I often see in projects are "empty" services which mainly delegate all calls to a DAO. In this case DAO and Services can just collapse. Btw. it is always a good idea to introduce DAOs for accessing JPA incompatible resources. Then you will even need "integrational' logic. The responsibility of the DAO is decoupling from particular technology - what is perfect. EntityManager is already a DAO - it not only decouples from SQL but even different JPA-providers. You will have good reasons to encapsulate a generic DAO with more specific DAO.
thanks for your comments!,
adam
Posted by Adam Bien on February 08, 2009 at 10:07 AM CET #
@Release 4.0 Sprint 9
and what is exaclty the business logic in CRUD usecases or simple masterdata management?
thanks!,
adam
Posted by Adam Bien on February 08, 2009 at 10:09 AM CET #
"Release 4.0 Sprint 9" was actually me adding a second comment to clarify. I must have pasted my current sprint instead of my name :)
The business logic does many things. For example, you call into the service asking to find a reservation and it knows how to look through many data sources to locate it, load complete data from several other sources, translate error codes and exceptions, etc. so you have just one simple call to the service to load a reservation. It also has methods that performs business logic on input based on system settings and other live data.
Posted by Ryan de Laplante on February 08, 2009 at 10:39 PM CET #
One very important fact is missing here, what if you want to migrate from Java to .NET or maybe Ruby? dao wont have you, you better learn now .NET and Ruby, because you might migrate to it next month!!!!
Posted by raveman on February 09, 2009 at 10:59 AM CET #
Raveman,
your are right, however, the introduction of a persistence layer written in Groovy or linq-like Scala persistence framework is even more likely. In that case the DAO wouldn't help either...
regards,
adam
Posted by Adam Bien on February 09, 2009 at 11:41 PM CET #
Adam,
What you say makes good sense. Right now I have a separate DAO layer purely for the sake of separation of concerns but moving on I plan to seriously revisit this separation and potentially blur the lines.
Frankly I think most of the problems with DAO and JPA comes from the use of the JavaBeans pattern. If JPA uses constructor injection we'd be able to do some major cleaning.
Posted by Gili on February 10, 2009 at 06:36 AM CET #
Our I.T. project simply uses Spring/JDBC API which maps to a Data Transfer Object (POJO JavaBean) and then gets displayed with JSTL tags (or Struts tags for one step further when using a form bean). It's a simple design with DAOs and DTOs - what's superfluous and outdated about that? I spent most of my career creating web applications this way. If you work in a big I.T. organization, they are so resistant to change, scenarios 1-4 are unlikely. Besides, all our DAO calls are to database stored procedures instead of SQL statements.
Posted by Tom on October 16, 2009 at 02:41 AM CEST #
At first I think the idea not using DAO is extreme, but when think again it's possible to reduce DAO to reduce abstraction levels (especially when domain model is not much complex and the service is mostly delegation to DAOs) .
It's easy to grasp the whole architecture, also !
This is a brave approach against traditional JavaEE "more abstraction" way (which is the culprit of complexity)
Posted by Medicore-Ninja.blogSpot.com on September 20, 2010 at 01:08 PM CEST #
At first I think the idea not using DAO is extreme, but when think again it's possible to reduce DAO to reduce abstraction levels (especially when domain model is not much complex and the service is mostly delegation to DAOs) .
It's easy to grasp the whole architecture, also !
This is a brave approach against traditional JavaEE "more abstraction" way (which is the culprit of complexity)
Posted by Mediocre-Ninja.blogSpot.com on September 20, 2010 at 01:11 PM CEST #
Someone said :
"" In fact there are about 5 DAOs the service uses. One of them talks to a JCA connector to a legacy EIS ""
It's all about services. What we call DAO or Repositories, are SERVICES. But with different responsabilities.
In case of Repositories, they are persistance services.
The "3-tier architecture" sent us in the wrong way. There is no such things as "layers". The fact that business services depend on persistance services, does not imply "layers".
We have services with different concerns (persistance, business, ui controllers ...)
And here it is : if your EIS does not expose a pure CRUD service, you should not suffix your class with *DAO/*Repositories !!
= This is a remote service, with a proxy jca implementation ...
Posted by bugsan on July 13, 2011 at 04:22 AM CEST #