Background

At my current project, we’re currently working on the second release of a webapp that integrates with a couple of webservices. It features a N+1 architecture (pretty much like the J-Technologies reference architecture) integrated by Spring 2.5.x. In this release, we have to persist into a single-consumer Oracle database. Since we have a tight deadline but do want to experiment with various ORM implementations in the future, we ended up deciding to use JPA1.0 with Hibernate3 as persistence provider.

Purpose of this post

The combination Websphere / Hibernate / JPA did provide some – shall we say.. challenges? :) You have some different approach routes to wire these technologies together and having a lot of choices can be confusing. The purpose of this post therefore is not only to show how we integrated it all but also why we did it this way. That hopefully helps others who are starting up a project with these technologies.

And of course, some of the choices we made may be flawed. That’s the reason you can write comments below :)

I created an example project with our setup, see the bottom of this post ([1]). Remember that nowadays you can download a development version of Websphere 7 at no charge at IBM.com. But you can also run this project in jetty ofcourse.

Configuring JPA

Setting up the PersistenceContext

We decided to use Spring to bootstrap the JPA context. This is not the pure JEE way, I’ll get back to that in a later section. Spring’s LocalContainerEntityManagerFactoryBean does pretty much what the JEE container should do: bootstrap the JPA persistence context, but with the usual extra Spring flexibility.

persistence-spring.xml

	

org.hibernate.ejb.HibernatePersistence
		java:comp/env/jdbc/testDb
	

Spring application context

<util:properties id="jpaProviderProperties" location="classpath:jpaVendor.properties" />
<bean id="entityManagerFactory"
class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<!--
Non-standard filename so Websphere doesn't try to create the EM (and
throw errors due to Hibernate not being on the AS classpath)
-->
<property name="persistenceXmlLocation" value="classpath:META-INF/persistence-spring.xml" />
<property name="jpaProperties" ref="jpaProviderProperties" />

</bean>

The location of the persistence-spring.xml is important as it is used to find the Entities you want to persist (more info, see here). Ignore the jpaVendor.properties for now, I’ll get to that in the section on integration tests.

Transaction management

Choosing a transaction manager

In the end, we chose the most future-proof option: a JTA Transaction manager (TM). But we started out with these options:

1. A (in JPA-speak) RESOURCE_LOCAL transaction manager. Spring provides the JpaTransactionManager. Cannot handle transactions for more than one XA resource (db, web service call, etc).
2. A container-provided JTA transaction manager. Websphere provides the UOWManager and Spring providers a wrapper around it, offering full JTA support – transaction suspending, distributed transactions, the works. It can be created through the container-independent <tx:jta-transaction-manager /> in the spring context.
3. What is all this transaction manager dogma? Do we need it? What’s wrong with using plain old autocommit?

First: do we need a TM at all? Recent experiences at a different project showed that without a TM the use of connection pools is less efficient resulting in sizeable performance penalties, although it’s not completely clear why. So configuring a transaction manager seems the sensible thing to do. Since JTA is the default choice when running JPA in a JEE container and we had no good reason to reject this option, we went ahead with the default approach. Also, I have the feeling we might have to support WS-Transaction somewhere in the future (yes, YAGNI ;)).

Transaction boundaries

Next: how to configure the transaction boundaries. That’s pretty standard Spring-tx stuff (since we’re not using EJB3). We use Spring’s @Transactional annotations on the service layer (the transaction boundary in our application) and the rest is taken care of by Spring’s <tx:annotation-driven />. Works like a charm.

Hibernate configuration

Hibernate did give some problems with this setup. Since Spring is managing transactions, the only thing Hibernate should do is join the transaction. In order to do this Hibernate needs to look up the transaction manager. Sadly, neither the IBM UOWManager nor the Spring wrapper implements the TransactionManager interface. In the end, this configuration in jpaVendor.properties did the trick:

hibernate.transaction.manager_lookup_class=org.hibernate.transaction.WebSphereExtendedJTATransactionLookup

Bottom line: Hibernate is synchronizing to the transaction using a pre-WAS6.1 mechanism, and Spring is using the >=WAS6.1 way. Also, the Hibernate implementation has it’s problems: setRollbackOnly may be called in application code according to the spec, but this causes a nasty stacktrace in the log. But: it does work correctly. The whole story is explained in detail in this thread.

On a sidenote: something confused me for a while: Hibernate3 configures itself based on the persistence.xml configuration. I noticed that it switched its transaction factory implementation automatically to ‘JoinableCMTTransactionFactory’. CMT is Container managed persistence, and we don’t use EJB2, since this seemed incorrect. After some time I read the javadoc here, and it turns out that this is in fact correct behaviour:

The term ‘CMT’ is potentially misleading here; the pertinent point simply being that the transactions are being managed by something other than the Hibernate transaction mechanism.

Opening the entity manager on each request

We also added a filter (OpenEntityManagerInViewFilter) in the web.xml that sets up a EntityManager that the request starts en destroys it when the request ends (comparable with the OpenSessionInViewFilter for Hibernate).

Running integration tests / Jetty

Our data layer is isn’t unittested, it’s integration tested (as explained by yet another Vincent here). As a rule of thumb, we always try to keep the configuration as identical to the deployment configuration as possible. In this case, the JPA context is set up using the exact same application context file and persistence.xml. The only thing that’s different are the jpaVendor.properties (in tests, I want Hibernate to use the H2 dialect and to automatically create the DB schema). They are read from the classpath (see sping XML snipplet at the beginning of this post): so you can have separate jpaVendor.properties for your test (src/test/resources) and production (src/main/resources) environment [2].

The resources normally provided by the JEE container (through JNDI) need to be mocked in test mode. This is a nice way to accomplish that:

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations = { "classpath:spring-config-domain.xml" })
@TransactionConfiguration(defaultRollback = true)
public class UserDaoTest {
[...]
	@BeforeClass
	public static void beforeClass() throws Exception {

		// Suppress the 'close', since we use a single connection in unittest
		// anyways. The connection is terminated upon destruction in afterClass
		singleConnectionDataSource = new SingleConnectionDataSource(
				"jdbc:h2:mem:", true);
		singleConnectionDataSource.setDriverClassName("org.h2.Driver");
		final TransactionAwareDataSourceProxy transactionAwareDataSourceProxy = new TransactionAwareDataSourceProxy(
				singleConnectionDataSource);

		contextBuilder = new SimpleNamingContextBuilder();
		contextBuilder.bind("java:comp/TransactionManager",
				new JotmFactoryBean().getObject());
		contextBuilder.bind("java:comp/env/jdbc/testDb",
				transactionAwareDataSourceProxy);
		contextBuilder.activate();
	}

	@AfterClass
	public static void afterClass() throws Exception {
		contextBuilder.deactivate();
		singleConnectionDataSource.destroy();
	}
[...]
}

So, a datasource is registered in JNDI that keeps one connection open at all times (which is pretty handy when using a single threaded H2 DB) and a JOTM (jottum!) TM is used instead of the Websphere TM. The nice thing is the Spring configuration is identical, the <tx:jta-transaction-manager /> finds the JOTM TM now instead. Hibernate does need some manual configuration:

hibernate.transaction.manager_lookup_class=org.hibernate.transaction.JOTMTransactionManagerLookup

Another bonus: if you want to run your app in a lite JEE container like Jetty for quick deployment, you can use pretty much the same config for that!

Discussion – what about JPA by the JEE book?

According to the EJB3 spec, the container should be in control of the PersistenceContext, roughly as follows:

• Container detects META-INF/persistence.xml on webapp startup.
• Container boots the JPA context based on the provided configuration.
• Container sets the PersistenceContext on objects requiring it.
• Container exposes the PersistenceContext via JNDI (as defined in the web.xml).

Now try this on Websphere 7 (and 6.1 with the EJB3 feature pack, for that matter). As expected (remember: don’t feed the trolls), Websphere fails if you want to use your own persistence provider (in our case: Hibernate3). According to IBM this should be possible by setting the classloader to PARENT_LAST, but alas, that didn’t seem to work as documented. Messing with heavy weight containers like Websphere is tiresome: you have stop/start redeploy until you’ve gone cross-eyed. So, the pure JEE configuration for JPA in Websphere wasn’t convenient for us (which doesn’t say it’s impossible) and I personally don’t see that as a problem – because it’s just easier to do it the Spring way – which can be easily tested in an integration test, instead of deploy-time.

Final words

So, is this the best solution? THE way of configuring JPA/JTA/Hibernate/WAS? Of course not. The is no right answer here I think, it all depends on the environment you’re in. But by explaining why this seems to be a nice fit for our situation, I hope to help those we need to use these technologies but can’t seem to see the forest for the trees.

Thanks for reading!

Links

[1] example jpa-spring-hibernate-was project
[2] You could also accomplish this by setting a hibernate.properties on your classpath as Hibernate picks this up automatically, but this a more explicit way (but who cares about explicitness anymore in these times of @Autowired magic? :-)).
Keywords: Websphere 6.1 7, JPA, EJB3 , JTA, Transaction management, Spring, Hibernate, Hibernate3

A common problem with typecasting a lazy loaded entity to its child is a ClassCastException. This exception occurs because the dynamic created proxy implements the baseclass and has no knowledge about its subclasses.

Suppose we have a class B which extends A and a class C which has class A as a member as shown below.

The following test will fail with a ClassCastException on the last line.

The methods save(), clearSession() and retrieve() are just helper methods which implement the Hibernate session methods save(), clear() and get().

A search on the Internet shows a couple of solutions for this problem.

1. Using interfaces as a proxy in the hibernate mappings. This will result in accessing the object through its interface only so all methods must be exposed in the interface. I don’t want to expose every public or protected method in the interface which are not intended for use by external parties.
2. Using the Visitor Pattern to access the correct childclass. This means that users of these objects must write a lot of code just to use some getters. I don’t want to burden someone else with a local Hibernate problem.
3. Using ((HibernateProxy)object).getHibernateLazyInitializer().getImplementation() whenever a typecast of an object to its child class is needed.

All of the solutions mentioned above are not suitable for me so I decided on another solution which is a variant of solution 3.

With a slight modification of the method getA() in class C, exposing the HibernateProxy is avoided.

Now the test completes without failure.

This has to be done for every getter which can return a lazy loaded proxy.

And if you (like me) don’t want Hibernate code in your domain model, you can move this code to the persistence layer and use dependency injection to use it.

It’s still not an elegant solution (IMHO there isn’t one), but its the best usable for me.

As promised in my previous post I will blog some more about JPA and how to use it. In this post I will go into the locking features of JPA 2.0 including the new pessimistic lock options.

This post will:

• introduce the new locking features of the JPA;
• introduce both pessimistic locking and optimitic locking concepts;
• give a quick recap about JPA versioning;
• talk about the connection between locking and design;
• and finally compare both locking strategies.

JPA 2.0 now supports Pessimistic Locking:

A great omission of JPA 1.0 was the lack of pessimistic locking. Therefore it was necessary to fall back on the support of the underlying implementation to use the JPA is situation where pessimistic locking was required. This can happen when JPA shares a database with another process which does not know or supports versioning based optimistic locking.

JPA 2.0 now supports the following locking modes:

• OPTIMISTIC                                    (==READ in JPA 1.0)
• OPTIMISTIC_FORCE_INCREMENT         (==WRITE in JPA 1.0)
• PESSIMISTIC_READ
• PESSIMISTIC_WRITE
• PESSIMISTIC_FORCE_INCREMENT       (A hybrid of both strategies)
• READ                                               (kept around for backward compatibility)
• WRITE                                             (kept around for compatibility)

Object can be locked by using the find() or refresh() operation of the EntityManager. For example:

Order order = entityManager.find(Order.class, 10, LockModeType.PESSIMISTIC_READ);
 
SELECT ID, PRODUCT, AANTAL, VERSION, orderId FROM ORDER_TABLE WHERE (orderId = ?) FOR UPDATE.

As you can see the FOR UPDATE is added to the select query telling the database to get an exclusive lock on this selected row.

You can also use the lock() method on the entityManager and specify a Lock Mode on Queries (JPQL / Named and Criteria).

Now that we know how to unleash the power of pessimistic locking we need to learn how to use it well.

Locking Strategies: a quick recap….

Locking is a means to prevent data form becoming corrupted because two different processes are editing the same data. If we use locking correctly no two processes can edit (or if required) access the same data. Thus data can never be inconsistent.

As already mentioned there are two locking strategies: optimistic locking and pessimistic locking. I will describe both strategies and give an overview of their pro’s and con’s.

Pessimistic locking

This strategy is the standard locking provided by the database. It will protect data by limiting access to a single process. This is achieved by keeping track of all the currently active locks. If another process wants to access locked data it will have to wait until the other process releases the lock. Of course this introduces a whole range of potential errors like lock timeouts and deadlocks.

This locking strategy is called pessimistic because of the assumption that locking is always necessary to avoid corruption. Based on that assumption it introduces significant overhead in order to keep track of which process is assessing which data. Compare pessimistic locking to a traffic light. It will only allow vehicles to pass when it knows for sure that no one will be in the way.

Using pessimistic locking has some pro’s:

• The database is in charge and protects your data. Independent from application logic.
• A process or thread can only proceed if it has the right locks. Thus it is guaranteed that there will be no conflicts once the lock is acquired.
• Processes are put on hold until they can acquire the lock. (This blessing can also be a curse because a process can overwrite data the moment the lock is released. This feels like a missing update but is technically the correct behaviour. But as long as you read and write in the same transction you’re data is never stale.)

No pro’s without con’s:

• Keeping track of all those locks introduces significant overhead. Even if there is no data being accessed simultaneous the database still locks.
• The locking can lead to deadlocks and lock time out. These errors are hard to recover from and take a long time before the calling process is informed.
• Must be supported by the database.

So pessimistic locking depends on the database restricting access to data. But this comes at high overhead and hard-to-recover errors.

Optimistic locking

As pessimistic locking is embedded in the DBMS, optimistic locking is a strategy that by-passes the database. It will detect conflicts only when they occur. This is done introducing a version number to every table you want to protect. If you read data you will get the version number. If you alter the data you first check the version number again, and when holding the previous read value, update the record and increment the version number. If some one has “changed the data right from under you” you will see a different version number and know that there is a conflict. I will refer to the optimistic lock procedure as check&update.

This strategy is called optimistic because it never bothers to lock. It assumes that process will not bother each other until they do.

Using optimistic locking has some big pro’s:

• There is no (at least very little) overhead involved in locking.
• Optimistic locking is fast and easy to use, especially because it works implicitly. If you specify a @Version the upate&check will be performed automatically.
• It’s very efficient.
• It is database independent. No special features are required.

There are also some drawbacks:

• It will only detect conflicts, not prevent them. When it occurs it’s the application that must resolve the conflict. For example by showing the user a diff or an option to override the current version in the database.
• If a conflict occurs only one process is allowed to proceed. The others have their database transaction rolled back. This is far more expensive than waiting until you get the database lock.
• It will only work if everyone accessing the database plays by the versioning rules. The database does not enforce it.
• It can be considered ‘unfair’ as the process that writes the data first wins, opposed to the process that first acquired the lock.
• Sometimes optimistic locking is not sufficient. Locking a complete table to protect against insert for example.

So optimistic locking depends on the calling processes to respect the versioning rules. This makes it possible to detect conflicts and eliminates the need to keep of all the locks and gives Optimistic locking a huge advantage.

However when conflicts occurs it is up to the application to patch things up.

JPA support for optimistic locking

JPA supports optimistic locking based on versioning right from the first release. All you need to do is declare an attribute of your class with a @Version annotation.

For example:

  @Entity
  public class Order {
 
      @id @GeneratedValue
      private Integer id;
 
      @Version
      private Integer version;
}

The above code will result in a Order table with a primary key and version column. JPA will check and update the version after every change to Order.

More on JPA versioning can be found here.

Lock scope.

At first glance versioning seems to be the preferred strategy. It’s easy to use and with little overhead. However there is one more aspect to take into account when dealing with locking. That is what I call ‘the lock scope’.

Versioning will only lock (check&update) records that were changed. Databases will only lock records you tell it to lock by doing a SELECT … FOR UPDATE. Both procedures prevent processes from corrupting the database. But it will not prevent breaking business rules!
Let’s look at the following example:

This is a typical Order->OrderLine example. Order has a set of OrderLines which keep a price and quantity for every single item in Order. Let’s assume that there is a business rule that the total amount of money of an Order must stay below $10.000. This is easily achieved by adding a check on order to make sure that every addition/alteration of OrderLines will not break this rule.
A problem occurs when another process comes in and adds OrderLines to an Order at the same time. No single process knows all OrderLines. To make the check work in a concurrent environment you need to make the OrderLine updates in a serial order. In other words: the Order needs to be locked before any additions can be made to OrderLines. This ensures that the business rule can be enforced.

The JPA can achieve this by using one of the two lock levels: OPTIMISTC_FORCE_INCREMENT or PESSIMITC_WRITE. Both will give you a exclusive lock to make sure no other process can edit the same data.

This example illustrates that there are situations in which you need to think ahead about locking. Both versioning and database locks won’t help you out-of-the-box . You need to determine the right lock scope. Determining the lock scope is a business question and needs to be defined based on the functional design and then translated to technical requirements.

Choosing a locking Strategy.

Ok .. now you know about the pro’s and con’s of both locking strategies. You know how to use them technically and you know you need to think about the lock scope. So which strategy is for winners?

As you probably have guessed there is no straightforward answer.

Optimistic locking has little overhead and is easy to use, especially because it works implicitly in JPA. But you need to make sure that everyone using the database uses the same versioning approach. It’s also more expensive in terms of conflict resolving.

Optimistic locking is the preferred strategy if:

• You’re application has a private database.
• All the applications using the database know and use versioning.
• It is unlikely that there will be a lot of conflicts. (eg. Users editing the same data.)

Pessimistic locking will protect data on the database level. It will prevent conflicts by putting the process in a queue to wait for the lock. If there are a lot of collisions this gives a better change of more processes making it through. However having to keep a large lock administration involves a lot of overhead even if there a no conflicts.

So pessimistic locking it the preferred strategy if:

• Other non-versionized processes will edit the data you need to lock.
• You predict / see that there will be a lot of colissions.

For those among us unable to choose, JPA offers a hybrid solution. If you use the lock option PESSIMISTIC_FORCE_INCREMENT both Pessimitic and Optimisic locks are acquired at the same time. Offcourse you’re cutting of both your hands when using this option for every database call…. “Just to be sure .. “. You’ll end up with the bad from both locking strategies. But this hybird option can be a life saver when a particular table or operation must be protected at a database level and still has to participate in versionized transactions.

More information on locking

• JPA 2.0 Concurrency and locking – Shows transcipts of most lokcing possibilities.
• Explanation of Optimistic Concurrency Checking – Talks some more about optimisitic locking.

And don’t forget to think about the lock scope!

This is the second installment of my blogs about the JPA. Next time we’ll go into the new Criteria API of JPA 2.0.

• Special thanks to Martijn Blankestijn for the Order example and Jouke Stoel for test reading.

Thursday 10 December the people from EclipseLink release version 2.0 of their Object Relation Mapping framework. Besides other improvements this release includes the reference implementation of JPA 2.0.  This means that the JPA 2.0 and JSR-317 are now final!

The second release of the Jave Persistence API adds a lot of new features to the JPA framework. The team responsible for this release now claims to serve 95% of the persistence needs of Java programmers. You can get the final version of JSR-317 or download the reference implementation on the eclipselink website. Two older blogs by Mike Keith introduces all the new features here and here.

Version 2.0 now has (among others) support for:

• Increased locking possibilities;
• Support for Criteria Queries and a MetaModel;
• (A little) support for second level caching;
• Basic Element and embeddable collections;
• Orphan removals for sets;
• Derived Identifiers (using @ManyToOne as past of your primary key);
• Ordered list.

With these new features the Java Persistence API becomes even more useful. In the next  few weeks I will blog some more about the new features and go deeper into some of the new features and how to use them to your advantange.

So stay tuned.