My last article triggered a lot of responses of people, which I, of course, try to answer.

One question, by Tofarr, was:

You are not caching the contructor! class.newInstance() internally looks for the no arg constructor each time – I would be curious to see how doing a Constructor c = class.getConstructor() would affect the timing…

This is not for 100% true, since (at least in my Java version) Class.newInstance() caches the no-arg constructor after it’s first lookup. But, the suggestion is worth trying.

So, let’s go!

Caching and instantiating the Class

In the original article, this was the task with the cached Class object.

Caching and invoking the Constructor

So I added a simple implementation which caches the Constructor object, like this:

The code should speak for itself. We don't just cache the Class object, but also the Constructor object. Caching only the Constructor in not enough, since we still need the Class to reflectively lookup methods. This wouldn't be necessary if we cache the Methods. But, for the sake of a fair comparison, both implementations don't cache the methods.

Running the benchmark

Let's run it, using the following command:

In the previous article, I didn't show you the run command, so let's catch up.

• -jar
  Nothing special, I just packaged the benchmark in a JAR to easily transfer it across machines
• -server
  This option is used to enable the high performance "server" VM. Since Java SE 5, Ergonomics are used to automatically pick an appropriate VM, based on the characteristics of the hardware, OS, etc. We explicitly choose server to make the test more reliable.
• -XX:+TraceClassLoading
  Output each loaded class to the console. Used to make sure that the test isn't disrupted by class loading, which would skew the results.
• -XX:+PrintCompilation
  Output a line to the console when a method is compiled by the VM. Used to make sure that the test isn't disrupted by dynamic compilation, which would skew the results.
• -XX:+PrintGCDetails
  Output a detailled message to the console with each garbage collection run. Used to check that no major collections occur during a test, which would skew the results.
• -Xms512m -Xmx512m -XX:PermSize=256m -XX:MaxPermSize=256m
  I purposely set high values to delay any garbage collection, because that could decrease performance on the later runs.

The results

NOTE: I've changed the benchmark since the previous article and the results presented here aren't comparable with the old ones!

Hey, that's odd. Constructor.newInstance() is more expensive than Class.newInstance(). Strange... (ps. the result is the same on many runs on multiple machines)

Fixing the problem

But... Constructor.newInstance() is a varargs method. This means a new array is created when it is invoked. Let's fix this line:

into this:

We also need to put this thingie somewhere:

This way, we pass in our own array, which is null, but this is enough to disable the creation of a new array.

Note that I didn't just pass in a null literal. This would lead to a compiler warning and makes the code less obvious.

The results:

Yeah, now we're faster, but just a little bit.

Wrapping up

So, let's combine this with the other optimizations (method caching) and see where we end up:
"Java HotSpot(TM) Server VM (build 10.0-b23, mixed mode)"

(these are the totals of a *lot* of runs)

And, for the fun of it, some OpenJDK results from my little Atom 330 Ubuntu machine: "OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode)":

So, invoking Constructor.newInstance(args...) is slightly cheaper than invoking Class.newInstance().

And, Reflection has gotten pretty damn fast if you ask me.

Don't forget to pass in a null array to prevent the JVM from automagically creating one for you.

Note: Class.newInstance() and Constructor.newInstance() behave differently with regards to Exceptions. See the Class and Constructor documentation for details. The only thing I'm gonna say here, is that Constructor is the preferred way.

In this article, I’ll show you some ways to turn slow reflective code into faster code.

Reasons to use reflection

As some of you may know, many frameworks and libraries use reflection to do their thing. Reflection can be a solution for many sorts of issues, like:

• Configuration. Many frameworks use text-based configuration, like XML. Classes are often configured in text files and need to be instantiated at some point. Also, method names (like event handlers or callbacks) might be configured in text files.
• Frameworks in general. Whether you configure your classes in an external text file, annotations or some kind of Java config, the framework can never know the classes it invokes. This framework therefore almost always needs to invoke Class.newInstance() or something similar (Constructor.newInstance() is actually better) to invoke the application classes. And, if you don’t want to put any restrictions on the application code, like requiring application classes to implement a predefined interface, you definitely need to use reflection to invoke methods and access fields.
• Backwards compatibility. For example, the BeanValidator in MyFaces needs to behave in two different ways, depending on the available libraries. It uses the method ValueExpression.getValueReference(), but only if it is available (available since Unified EL 2.2). Otherwise, it uses a home-brewn mechanism. Reflection is needed, both for compile and at runtime. Otherwise, the code will throw a NoSuchMethodError when ValueExpression.getValueReference() is not available. Reflection is the way to determine method-existence on classes.
• Tooling, or other kinds of code that need to inspect classes at runtime.
• Simple class modifications, like making private methods accessible. OR-Mappers like Hibernate do this for field access, since fields are usually private.
• Non-Java templating engines, like FreeMarker or Facelets. Since templates are not Java bytecode, they rely on reflection to access Java stuff.

Reflection in JSF

As you may know, JSF also uses reflection a lot. Managed Beans, unlike i.e. Struts Actions, don’t have a specific interface, but (mostly) follow the JavaBean convention. This requires the framework to use reflection a lot, for example while resolving EL expressions to method calls.

Unfortunately, reflection is slow. Note that I’m talking about nanoseconds per invocation here. Few reflective invocations in a single web request is nothing to be afraid about, especially compared to database and network latency. But, resolving expressions is a heavily used operation in any JSF request.

Profiling also shows that a significant portion of a JSF application is spent on reflection.

I say, time to improve this. But first, we need measurements to see what the problem is.

Reflection performance, the test setup

So, reflection is slow, right? What does slow really mean and how slow is it?

Let’s start with a simple, but typical, example of how reflection is used in most frameworks.

Consider a simple JavaBean:

A simple JavaBean with a String property. This JavaBean’s methods are to be invoked by the framework.

Now, let’s first write a simple program to access it without reflection. Not a single framework invokes application code directly, since it’s not compiled against it, but this will be our baseline for comparison.

A reliable concurrent test harness requires a lot of boilerplate code, but the essence is between the two comments. It invokes the bean methods in a plain Java way.

On my laptop, with 1000 threads and 1000 iterations, it takes 68ms to complete.

Measuring reflection performance

Now, let's try it out using reflection. After all, not a single framework is compiled against the application code, so the method above is the ideal, but impossible.

We add the following task to the test harness:

This code should look pretty familiar. It loads a class, given its name, instantiates it and invokes its methods, all using the standard Reflection API.

Don't mind the arithmetic stuff. It's used to prevent the JIT from removing too much (dead) code, which would render the test case useless.

Let's include it in the test, so the main method looks like this:

The results are obvious:

So, full blown reflection is about 50 times slower. Not good, right? Let's try to improve this.

Optimizing reflective code: Caching classes

But, of course we can do better. Let's tune the reflective approach a bit.

The first optimization is caching the Class object. Classes are thread safe and can be cached and reused safely.

Caching a Class object saves a lot of repeated overhead, like contacting the SecurityManager, ClassLoader, etc. So this should be a significant improvement.

The code looks like this:

And we of course add the task to the test.
Performance got better with about 40%

So, we can safely conclude that in general, it's a good idea to cache Class instances if you plan to instantiate them a lot. Don't cache too much, because such loitering objects are the number one reason of Java memory leaks. But, a "Map<String, Class>" with a fixed amount of classes (like all your controllers) should do no harm here.

Optimizing reflective code: Caching method handles

There is more reflective code that can be optimized. The Method instances are also good candidates for caching. After all, you obtain the same Method handle every time and invoke it with an instance parameter. So, let's cache the getter and setter methods, like this:

The code is getting a lot messier with every tweak, but don't worry. There are other, more concise ways to do this. I chose this approach because of testability (writing a correct concurrent test is hard).

The results, however, don't lie:

Caching the two methods improves performance with about 400%. But we have two methods, so let's conclude a 200% improvement per method.

At this point, we've performance has improved about ten times, compared to the original reflection implementation. But, even this optimized version is still about 6 to 10 times slower than the original one.

Unfortunately, using standard reflection, we can't make much more improvements. The JIT simply has less ways to optimize reflective invocations than it has with normal bytecode (invokedynamic in JDK 7 improves this).

Optimizing reflective code: Caching instances

This optimization is only applicable to stateless objects! Using it on objects with state leads to correctness problems!

Let's assume that correctness is irrelevant so we can cache instances of the bean. The code would look like this:

And the results:

So, we've won about 100-200ms by not invoking the newInstance method. But, since creating objects also affects garbage collection (in this test we create one million beans per test) and we've refactored the local variable to a static final variable the performance implications of the newInstance call itself are not obvious. But, it's a fact that we've improved performance by caching the instance.

But note, this approach only applies to stateless or immutable objects. Wherever there's state, you need new instances.

Optimizing reflective code: Conclusion

In this article, we've seen how the performance of reflection differs from 'normal' Java bytecode performance.

We've also seen how some simple tweaks like caching Class and Method objects make reflective code a lot faster.

But still, even when caching instances, the performance of reflection doesn't even come close to the performance of plain Java bytecode (where the instances weren't cached).

There are examples of high performance reflection, for example in the java.util.concurrent.atomic.AtomicReferenceFieldUpdater class, but this kind of code should not be written by anyone except Doug Lea. (and b.t.w., it hurts portability)

So, should you use reflection in your application? It depends. If you're writing high performance stuff that's invoked very, very often, I would say no. If you're writing a simple method dispatcher that gets invoked once per web request in a normal web application, I wouldn't care about the performance of the reflective invocation. After all, we're still talking about nano-/milliseconds here.

In the next entry, I'll show you how you can use bytecode enhancement to get the same functionality as with reflection, but without a huge performance penalty.

Edit
Note: I just heard that in some cases, HotSpot is able to inline through reflection calls, enabling it to apply all it's JIT tricks on it, effectively reducing the invocation cost to zero! But it only works if it can prove if the invoked method is always the same, which is not always true unfortunately.
http://java.sun.com/products/hotspot/whitepaper.html#performance

I’ve been playing a bit with Bean Validation and thought I found an issue: Unit testing a ConstraintValidator. Specifically, parametrized ConstraintValidators.

First, what’s a parametrized ConstraintValidator? Simple, a Validator that takes a parameter to do its job. A simple example is a length validator that takes the minimum and maximum length as parameters and validates if the value is between those two.

Parametrized ConstraintValidators are everywhere, even Bean Validation itself contains them. For example: @Min(length) and @Max(length), but also normal ConstraintValidators, like @NotNull where you can customize the error message. However, this error message parameter is not very interesting with regards to unit testing individual ConstraintValidators, because the message is generated by the Bean Validation framework.

For example, consider this ConstraintValidator, incl. corresponding annotation:

In the future, Bean Validation may become the core mechanism for ensuring correctness in your system. For example, JPA 2.0 leverages Bean Validation annotations to generate additional DDL statements and JSF 2.0 automatically uses Bean Validation when it’s available. Note: JSF 2.0 uses a new concept, called “default validators” to implement this. A default validator is invoked on every input component on a postback.

But, how do we test this stuff? After all, you can’t instantiate an annotation.

I currently have 4 options:

• Using reflection to use a declared annotation
• Custom annotation implementation
• Dynamic proxies
• Mocking, for example using Mockito

Using a declared annotation
The first attempt is shown below. Here, I simply declare the annotation in my test case on a dummy instance field (an annotation doesn’t live on its own) and look it up using reflection. This way, Java takes care of instantiating and initializing the annotation.

I personally quite like this approach. The test data is inside the test case. Unfortunately not in the test method, that would be better, but since you can’t put annotations everywhere in your code, it’s not easy to implement this decently.

So let’s look at the next one…

Custom annotation implementation
Since Java annotations are just a special case of interfaces, they should be fairly straightforward to implement. However, for some reason, I didn’t know this until recently. But it’s actually not different from any other interface implementation. See for yourself.

Nothing special, right? But I don’t really like it. I could have also written it inline, but I don’t like that either:

I think the above one looks like garbage. So this is also a no-go.

Dynamic proxies
Java itself uses dynamic proxies at runtime to instantiate annotations. So why not also use a Proxy to mock the annotation? See for yourself.

However inline, the implementation looks very clumsy. Unfortunately this is probably as simple as it can get, since you DO need to define the inner class.

And it will become even worse when there are multiple parameters on the bean, since you need to check which method is invoked. This will make the implementation even worse.

Mocking using Mockito
But, creating and configuring proxies, isn’t this what mocking libraries do? Exactly, let’s try.

This looks pretty nice. And it scales well with multiple attributes.

I only doubt it will be the nicest solution when the annotations become more complex. I really don’t like annotations that contain annotation parameters. Or even worse, arrays of annotation parameters. But reality is, they exist. And if they exist, they also need to be tested. But, OTOH, people who nest annotations probably don’t mind unreadable unit tests.

A common problem with typecasting a lazy loaded entity to its child is a ClassCastException. This exception occurs because the dynamic created proxy implements the baseclass and has no knowledge about its subclasses.

Suppose we have a class B which extends A and a class C which has class A as a member as shown below.

The following test will fail with a ClassCastException on the last line.

The methods save(), clearSession() and retrieve() are just helper methods which implement the Hibernate session methods save(), clear() and get().

A search on the Internet shows a couple of solutions for this problem.

1. Using interfaces as a proxy in the hibernate mappings. This will result in accessing the object through its interface only so all methods must be exposed in the interface. I don’t want to expose every public or protected method in the interface which are not intended for use by external parties.
2. Using the Visitor Pattern to access the correct childclass. This means that users of these objects must write a lot of code just to use some getters. I don’t want to burden someone else with a local Hibernate problem.
3. Using ((HibernateProxy)object).getHibernateLazyInitializer().getImplementation() whenever a typecast of an object to its child class is needed.

All of the solutions mentioned above are not suitable for me so I decided on another solution which is a variant of solution 3.

With a slight modification of the method getA() in class C, exposing the HibernateProxy is avoided.

Now the test completes without failure.

This has to be done for every getter which can return a lazy loaded proxy.

And if you (like me) don’t want Hibernate code in your domain model, you can move this code to the persistence layer and use dependency injection to use it.

It’s still not an elegant solution (IMHO there isn’t one), but its the best usable for me.

At the end of last year I encountered something odd in the java.util.Calendar. Now is odd behavior nothing to be surprised of in the Java Calendar but this particular oddness was really hard to spot.

I will therefore share it with you.

The code

Let’s first look a some code dealing with getting the last day of the month:

public class CalendarTest {
  public static void main(String[] args) {
    Calendar c = Calendar.getInstance();
    c.set(Calendar.MONTH, Calendar.FEBRUARY);
    int maxDayOfMonth = c.getActualMaximum(Calendar.DAY_OF_MONTH);

    System.out.println("last day of month = " + maxDayOfMonth);
  }
}

Please read the java doc for Calendar.getActualMaximum():

Returns the maximum value that the specified calendar field could have, given the time value of this Calendar. For example, the actual maximum value of the MONTH field is 12 in some years, and 13 in other years in the Hebrew calendar system.

So the code above can print two different values right …? 28 or 29 depending on the whether this year is a leap year. As you could have guessed that is another unexpected possibility. I can also print 31. Yes… really.

I all depends on the date on which this code is executed.

The problem is that Calendar.getInstance() will return a Calendar filled with the current date/time. Let’s assume the above code runs on the last day of January. The call to getInstance() will return 31-01-2009. The next step puts the month to February. This will result in a overflow as 31-02-2009 is invalid. Because of this Calendar will move the date to 3-03-2000. And March has 31 days.

There is a bug report (status: Closed, Not a Defect) in the SDN which told me that the observed behavior was correct. I was not the first person surprised by this and I’m guessing I will not be the last.

I was …

1. … lucky I wrote a decent unit test.
2. … lucky to be running said unit test om 31 December.
3. … unlucky for having to work on the last day of the year.

Otherwise the bug I created would properly have slipped through the QA cycles and would have ended up in production. There it would only be visible on the last three days of every month if someone would specify a date in February.

What about lenient?
The Calendar.setLenient function does protect against invalid input. Let’s quote the the Calendar java doc about Leniency:

Leniency
Calendar has two modes for interpreting the calendar fields, lenient and non-lenient. When a Calendar is in lenient mode, it accepts a wider range of calendar field values than it produces. When a Calendar recomputes calendar field values for return by get(), all of the calendar fields are normalized. For example, a lenient GregorianCalendar interprets MONTH == JANUARY, DAY_OF_MONTH == 32 as February 1.

When a Calendar is in non-lenient mode, it throws an exception if there is any inconsistency in its calendar fields. For example, a GregorianCalendar always produces DAY_OF_MONTH values between 1 and the length of the month. A non-lenient GregorianCalendar throws an exception upon calculating its time or calendar field values if any out-of-range field value has been set.

So lenient does protect against the programmer/user creating a invalid date in a way that the following code will result in an exception:

    Calendar c = Calendar.getInstance();
    c.setLenient(false);
    c.set(Calendar.MONTH, Calendar.FEBRUARY);
    c.set(Calendar.DAY_OF_MONTH, 31);
    System.out.println(c.getTime());

But the exception is only thrown when c.getTime() is called. Calls to getActualMaximum() still work and return 31. So lenient is not useful here.

Lessons learned

The quick fix for this problem is to set the DAY_OF_MONTH to 1 (or any number between 1 and 28) before setting the month.

I also learned that Calendar.setLenient() will not protect you from this error. It will only stop you from getting an invalid Date. It does not protect against overflows.

Sould I use JODA time?

At the end of this post I have a question for you. I have no experience with JODA time always preferring to use the standard Date/Time API unless there was a problem. But perhaps I should reverse my views and use JODA time unless I’m not allowed too? What do you think … is it time to stop  using the default Calendar API and use JODA time instead? Or should I wait for Java 7 with JSR 310?

As promised in my previous post I will blog some more about JPA and how to use it. In this post I will go into the locking features of JPA 2.0 including the new pessimistic lock options.

This post will:

• introduce the new locking features of the JPA;
• introduce both pessimistic locking and optimitic locking concepts;
• give a quick recap about JPA versioning;
• talk about the connection between locking and design;
• and finally compare both locking strategies.

JPA 2.0 now supports Pessimistic Locking:

A great omission of JPA 1.0 was the lack of pessimistic locking. Therefore it was necessary to fall back on the support of the underlying implementation to use the JPA is situation where pessimistic locking was required. This can happen when JPA shares a database with another process which does not know or supports versioning based optimistic locking.

JPA 2.0 now supports the following locking modes:

• OPTIMISTIC                                    (==READ in JPA 1.0)
• OPTIMISTIC_FORCE_INCREMENT         (==WRITE in JPA 1.0)
• PESSIMISTIC_READ
• PESSIMISTIC_WRITE
• PESSIMISTIC_FORCE_INCREMENT       (A hybrid of both strategies)
• READ                                               (kept around for backward compatibility)
• WRITE                                             (kept around for compatibility)

Object can be locked by using the find() or refresh() operation of the EntityManager. For example:

Order order = entityManager.find(Order.class, 10, LockModeType.PESSIMISTIC_READ);
 
SELECT ID, PRODUCT, AANTAL, VERSION, orderId FROM ORDER_TABLE WHERE (orderId = ?) FOR UPDATE.

As you can see the FOR UPDATE is added to the select query telling the database to get an exclusive lock on this selected row.

You can also use the lock() method on the entityManager and specify a Lock Mode on Queries (JPQL / Named and Criteria).

Now that we know how to unleash the power of pessimistic locking we need to learn how to use it well.

Locking Strategies: a quick recap….

Locking is a means to prevent data form becoming corrupted because two different processes are editing the same data. If we use locking correctly no two processes can edit (or if required) access the same data. Thus data can never be inconsistent.

As already mentioned there are two locking strategies: optimistic locking and pessimistic locking. I will describe both strategies and give an overview of their pro’s and con’s.

Pessimistic locking

This strategy is the standard locking provided by the database. It will protect data by limiting access to a single process. This is achieved by keeping track of all the currently active locks. If another process wants to access locked data it will have to wait until the other process releases the lock. Of course this introduces a whole range of potential errors like lock timeouts and deadlocks.

This locking strategy is called pessimistic because of the assumption that locking is always necessary to avoid corruption. Based on that assumption it introduces significant overhead in order to keep track of which process is assessing which data. Compare pessimistic locking to a traffic light. It will only allow vehicles to pass when it knows for sure that no one will be in the way.

Using pessimistic locking has some pro’s:

• The database is in charge and protects your data. Independent from application logic.
• A process or thread can only proceed if it has the right locks. Thus it is guaranteed that there will be no conflicts once the lock is acquired.
• Processes are put on hold until they can acquire the lock. (This blessing can also be a curse because a process can overwrite data the moment the lock is released. This feels like a missing update but is technically the correct behaviour. But as long as you read and write in the same transction you’re data is never stale.)

No pro’s without con’s:

• Keeping track of all those locks introduces significant overhead. Even if there is no data being accessed simultaneous the database still locks.
• The locking can lead to deadlocks and lock time out. These errors are hard to recover from and take a long time before the calling process is informed.
• Must be supported by the database.

So pessimistic locking depends on the database restricting access to data. But this comes at high overhead and hard-to-recover errors.

Optimistic locking

As pessimistic locking is embedded in the DBMS, optimistic locking is a strategy that by-passes the database. It will detect conflicts only when they occur. This is done introducing a version number to every table you want to protect. If you read data you will get the version number. If you alter the data you first check the version number again, and when holding the previous read value, update the record and increment the version number. If some one has “changed the data right from under you” you will see a different version number and know that there is a conflict. I will refer to the optimistic lock procedure as check&update.

This strategy is called optimistic because it never bothers to lock. It assumes that process will not bother each other until they do.

Using optimistic locking has some big pro’s:

• There is no (at least very little) overhead involved in locking.
• Optimistic locking is fast and easy to use, especially because it works implicitly. If you specify a @Version the upate&check will be performed automatically.
• It’s very efficient.
• It is database independent. No special features are required.

There are also some drawbacks:

• It will only detect conflicts, not prevent them. When it occurs it’s the application that must resolve the conflict. For example by showing the user a diff or an option to override the current version in the database.
• If a conflict occurs only one process is allowed to proceed. The others have their database transaction rolled back. This is far more expensive than waiting until you get the database lock.
• It will only work if everyone accessing the database plays by the versioning rules. The database does not enforce it.
• It can be considered ‘unfair’ as the process that writes the data first wins, opposed to the process that first acquired the lock.
• Sometimes optimistic locking is not sufficient. Locking a complete table to protect against insert for example.

So optimistic locking depends on the calling processes to respect the versioning rules. This makes it possible to detect conflicts and eliminates the need to keep of all the locks and gives Optimistic locking a huge advantage.

However when conflicts occurs it is up to the application to patch things up.

JPA support for optimistic locking

JPA supports optimistic locking based on versioning right from the first release. All you need to do is declare an attribute of your class with a @Version annotation.

For example:

  @Entity
  public class Order {
 
      @id @GeneratedValue
      private Integer id;
 
      @Version
      private Integer version;
}

The above code will result in a Order table with a primary key and version column. JPA will check and update the version after every change to Order.

More on JPA versioning can be found here.

Lock scope.

At first glance versioning seems to be the preferred strategy. It’s easy to use and with little overhead. However there is one more aspect to take into account when dealing with locking. That is what I call ‘the lock scope’.

Versioning will only lock (check&update) records that were changed. Databases will only lock records you tell it to lock by doing a SELECT … FOR UPDATE. Both procedures prevent processes from corrupting the database. But it will not prevent breaking business rules!
Let’s look at the following example:

This is a typical Order->OrderLine example. Order has a set of OrderLines which keep a price and quantity for every single item in Order. Let’s assume that there is a business rule that the total amount of money of an Order must stay below $10.000. This is easily achieved by adding a check on order to make sure that every addition/alteration of OrderLines will not break this rule.
A problem occurs when another process comes in and adds OrderLines to an Order at the same time. No single process knows all OrderLines. To make the check work in a concurrent environment you need to make the OrderLine updates in a serial order. In other words: the Order needs to be locked before any additions can be made to OrderLines. This ensures that the business rule can be enforced.

The JPA can achieve this by using one of the two lock levels: OPTIMISTC_FORCE_INCREMENT or PESSIMITC_WRITE. Both will give you a exclusive lock to make sure no other process can edit the same data.

This example illustrates that there are situations in which you need to think ahead about locking. Both versioning and database locks won’t help you out-of-the-box . You need to determine the right lock scope. Determining the lock scope is a business question and needs to be defined based on the functional design and then translated to technical requirements.

Choosing a locking Strategy.

Ok .. now you know about the pro’s and con’s of both locking strategies. You know how to use them technically and you know you need to think about the lock scope. So which strategy is for winners?

As you probably have guessed there is no straightforward answer.

Optimistic locking has little overhead and is easy to use, especially because it works implicitly in JPA. But you need to make sure that everyone using the database uses the same versioning approach. It’s also more expensive in terms of conflict resolving.

Optimistic locking is the preferred strategy if:

• You’re application has a private database.
• All the applications using the database know and use versioning.
• It is unlikely that there will be a lot of conflicts. (eg. Users editing the same data.)

Pessimistic locking will protect data on the database level. It will prevent conflicts by putting the process in a queue to wait for the lock. If there are a lot of collisions this gives a better change of more processes making it through. However having to keep a large lock administration involves a lot of overhead even if there a no conflicts.

So pessimistic locking it the preferred strategy if:

• Other non-versionized processes will edit the data you need to lock.
• You predict / see that there will be a lot of colissions.

For those among us unable to choose, JPA offers a hybrid solution. If you use the lock option PESSIMISTIC_FORCE_INCREMENT both Pessimitic and Optimisic locks are acquired at the same time. Offcourse you’re cutting of both your hands when using this option for every database call…. “Just to be sure .. “. You’ll end up with the bad from both locking strategies. But this hybird option can be a life saver when a particular table or operation must be protected at a database level and still has to participate in versionized transactions.

More information on locking

• JPA 2.0 Concurrency and locking – Shows transcipts of most lokcing possibilities.
• Explanation of Optimistic Concurrency Checking – Talks some more about optimisitic locking.

And don’t forget to think about the lock scope!

This is the second installment of my blogs about the JPA. Next time we’ll go into the new Criteria API of JPA 2.0.

• Special thanks to Martijn Blankestijn for the Order example and Jouke Stoel for test reading.

Thursday 10 December the people from EclipseLink release version 2.0 of their Object Relation Mapping framework. Besides other improvements this release includes the reference implementation of JPA 2.0.  This means that the JPA 2.0 and JSR-317 are now final!

The second release of the Jave Persistence API adds a lot of new features to the JPA framework. The team responsible for this release now claims to serve 95% of the persistence needs of Java programmers. You can get the final version of JSR-317 or download the reference implementation on the eclipselink website. Two older blogs by Mike Keith introduces all the new features here and here.

Version 2.0 now has (among others) support for:

• Increased locking possibilities;
• Support for Criteria Queries and a MetaModel;
• (A little) support for second level caching;
• Basic Element and embeddable collections;
• Orphan removals for sets;
• Derived Identifiers (using @ManyToOne as past of your primary key);
• Ordered list.

With these new features the Java Persistence API becomes even more useful. In the next  few weeks I will blog some more about the new features and go deeper into some of the new features and how to use them to your advantange.

So stay tuned.

Afgelopen DeVoxx is een stemming geweest waar de bezoekers konden stemmen op hun favoriete nieuwe JDK 7 features.
De grote wijzigingen zijn natuurlijk allang bepaald, zoals !closures (niet dus), JVM optimalisaties voor scripttalen (JSR 292) en Java Module System (JSR 294), maar voor JDK 7 willen ze ook een aantal kleinere wijzigingen faciliteren.
Stephen Colebourne heeft op zijn blog een mooie samenvatting van de poll geplaatst.
Zoals je kunt zien, zitten er 3 populaire tussen:

1. Null handling
2. Infer generics
3. Multi catch

Properties is ook populair, maar deze komt sowieso niet in JDK 7.
Deze 3 populaire features zal ik even kort laten zien:

Null handling
Iedereen heeft weleens dergelijke code gezien:

if (myObject != null) {
    if (myObject.getProp() != null) {
        if (myObject.getProp().getProp() != null) {
            return myObject.getProp().getProp().getString();
        }
    }
}

Zou het niet relaxed zijn als je het zo kunt schrijven?

return myObject()?.getProp()?.getProp()?.getString();

Er zitten nog wat vergelijkbare constructies aan vast, zoals een aanpassing aan de ternary operator, zie Alex Millers blog.

Infer generics
Waarschijnlijk hebben jullie deze constructie ook weleens gezien? Zo nee, dan heb je waarschijnlijk nog geen JDK 5+ collections gebruikt.

List<String> list = new ArrayList<String>();

Dit is nog niet zo vervelend, maar wat dacht je van deze variant?

Map<String, AtomicReference<SoftReference<SomeObject>>> list = new ConcurrentHashMap<String, AtomicReference<SoftReference<SomeObject>>>();

Vrij lelijk als je het aan mij vraagt. En in de meeste gevallen is het niet eens nuttig, aangezien links en rechts van de assignment hetzelfde staat.
Wat dacht je hiervan?

Map<AtomicReference<SoftReference<SomeObject>>> list = new ConcurrentHashMap<>();

De haken aan de rechterkant geven aan dat de compiler de typespecificatie links van de assignment naar de rechterkant gekopieerd moet worden.
Zie wederom Alex Millers blog voor de details.

Ten slotte de Multi catch
Jullie hebben ook vast weleens dergelijke catch clausules gehad neem ik aan.

try {
    // Do something
} catch (Exception1 e) {
    LOG.error("Something horrible happened", e);
} catch (Exception2 e) {
    LOG.error("Something horrible happened", e);
} catch (Exception3 e) {
    LOG.error("Something horrible happened", e);
} catch (Exception4 e) {
    LOG.error("Something horrible happened", e);
}

“catch (Exception)” is niet netjes, hebben we allemaal geleerd. Voor je het weet, vang je onterecht een fout af. Maar al die code duplicatie zit je toch niet lekker. Vandaar een nieuw alternatief:

try {
    // Do something
} catch (Exception1 | Exception2 | Exception3 | Exception4 e) {
    LOG.error("Something horrible happened", e);
}

Dat is een stuk beter he? Vooral in combinatie met Safe Rethrow kan dit erg krachtig zijn.
Ik ben nog wel benieuwd naar de details, want wat is het type van “e”? Gaan ze op zoek naar een common supertype, of wordt het duck typing?
En weer een linkje naar Alex Miller.

Dusss…
Dit zijn maar een paar van de toevoegingen waar momenteel aan gewerkt wordt. Meer info over “Project Coin” op Joe Darcy’s blog.
En natuurlijk weer Alex Miller.

De Bluetooth JSR-82 en de API daarbij zijn al enkele jaren oud (v1.1: 2002), maar voor mij was het een nieuwe. Hiermee kun je vanuit Java met bluetooth apparaten communiceren. Het leek me wel leuk om, al forenzend in de trein een eBook te lezen en tegelijkertijd te zien wie er allemaal elke dag met me meereist. Deze zelfde exercitie is natuurlijk ook uitstekend in de file of op vaste locaties uit te voeren…

Inmiddels hebben heel veel apparaten een bluetooth (‘bt’) module: GPS ontvangers, WiiMotes, smartphones, billboards, verkeerslichten, you name it. Elk ingeschakeld apparaat adverteert zijn nummer en meestal een (door de fabrikant of eigenaar ingestelde) naam. De leukste tot nu toe vond ik “smartföhn”, maar voornamelijk zie je de naam van de eigenaar, of het merk & type van het apparaat verschijnen. O, en laat ik vooral ‘Dennis Ordina’ niet ongenoemd laten! Welke collega met Nokia telefoon met bt adres 001A165A1686 zat op 4 februari om kwart voor zes in de intercity van Amsterdam Bijlmer naar Utrecht CS? Als laatste vermelding: één telefoon had als naam het 06-nummer. Lijkt me niet handig, behalve als je wilt dat Jan en alleman je belt…

De naam Bluetooth komt overigens van de 10e-Eeuwse koning van Denemarken en Noorwegen: Harald Blåtand (niet te verwarren met Blauwbaard!). Het logo is afgeleid van de runentekens van zijn initialen.
De laatste bluetoothversie (uit 2007) is 2.1+EDR (3Mbps), en er wordt gewerkt aan een high speed variant die moderne WiFi-snelheden haalt. De radius is afhankelijk van de ‘class’ 100, 10 of 1 meter.

Begrippen

Eerst vuur ik enkele begrippen op je af uit de ‘bt’ wereld, waarna wat codevoorbeelden langskomen.

• Protocol bluetooth is opgebouwd als OSI network stack van protocollen, waarvan de interessantste L2CAP (Logical Link Control and Adaptation Protocol), RFCOMM (Radio Frequency Communications) en OBEX (OBject EXchange) zijn. Het zijn op elkaar gestapelde transportprotocollen net zoals TCP en IP op elkaar gestapeld zijn. L2CAP is packet oriented en RFCOMM stream oriented. OBEX is session oriented.
• Profiel interfacespecificatie; ruwweg een netwerkprotocol op applicatieniveau. Er zijn er zo’n 25 gedefinieerd; voorbeelden zijn FTP (file transfer), HID (muis/keyboard), OPP (vCards), BIP (plaatjes) en PAN (piconet). JSR-82 omvat alleen de basisprofielen GAP, SPP, SDAP, GOEP en de daarbij horende protocollen SDP (GAP+SDAP), OBEX (GOEP), RFCOMM (SPP) en L2CAP. Andere profielen moet je zelf uitprogrammeren!
• Bt adres vergelijkbaar aan een Ethernet MAC-adres. Bestaat uit 12 hex digits, waarvan de eerste 6 verdeeld zijn onder de fabrikanten. Aan het adres kun je dus al zien of het bijvoorbeeld een Nokia of Motorola apparaat betreft!
• Master net zoals er bij TCP/IP-verbindingen servers (daemons) zich registeren op een bekende poort en clients hiernaar connecten, is bt ook client/server georiënteerd; de server heet ‘master’ en registreert (publiceert) services. Dit vind je terug in bt UUIDs en URLs.
• UUID uniek nummer om een service (gedefinieerd in een profiel) te identificeren, vergelijkbaar aan een server IP port number. In 3 lengtes: 4, 8 of 32 hex digits. Er bestaat een simpele conversie om 4/8-digit UUID’s naar volledige lengte te converteren: concateneren met het vaste nummer 00001000800000805F9B34FB. Enkele voorbeeldwaarden: OBEX File Transfer = 0×1106, Human Interface Device Profile (WiiMote!) = 0×1124.
• BCC Bluetooth Control Centre; een GUI om je device, services & security te configureren, en te zoeken naar & pairen met andere devices. Meegeleverd met je bt driver.
• URLs je maakt connecties met behulp van URLs in het bekende formaat. Server URLs bevatten altijd “localhost”, de service UUID en eventueel parameters zoals een naam. Client URLs bevatten het remote bt adres, het kanaal- of PSM [Protocol Service Multiplexer] nummer – vergelijkbaar met een client IP port number; is niet hetzelfde als een service UUID – en eventueel (security) parameters. Enkele voorbeelden:
o OBEX/GOEP
 Server btgoep://localhost:ed495afe28ed11da94d900e08161165f
 Client btgoep://00A3920B2C22:12
o RFCOMM/SPP
 Server btspp://localhost:;name=Sample SPP Server
 Client btspp://0050CD00321B:3;authenticate={true|false}; authorize={true|false};encrypt={true|false}
o L2CAP
 Client btl2cap://0050CD00321B:1003; ReceiveMTU=512;TransmitMTU=512
• Piconet mininetwerk van maximaal 8 bt devices – meer verbindingen ondersteunt bluetooth niet. Een netwerk van meerdere aan elkaar gekoppelde piconets heet een scatternet.
• Service class elk bt device kan met zijn service class aangeven welke service types hij ondersteunt (bijv. positioning, networking of audio) en van welk (sub)type hij is (bijv. Computer/Laptop of Peripheral/Joystick).
• Discovery scannen naar devices en/of services. Na discovery levert JSR-82 je URLs zodat je die niet zelf hoeft op te bouwen. Een bt device kan altijd (‘global’/’general’) discoverable zijn (GIAC), een beperkte tijd (‘limited’ – LIAC) of helemaal niet. Dit poor-mans beveiligingsmechanisme is bekend van WiFi routers. Ook daar is de fabrieksinstelling meestal niet de veiligste… Discovery is niet erg snel; een complete sweep van devices & hun services duurt afhankelijk van het aantal devices in bereik één tot meerdere minuten, dit onder andere omdat er geen parallelle service discoveries kunnen draaien.
• Service record bij discovery van een service ontvang je een lijst van attributen met onder anderen een leesbare naam en kanaal (RFCOMM&OBEX)- of PSM (L2CAP) nummer. Helaas vullen niet alle bt devices dit service record exact volgens de specs.
• Pairing pas nadat apparaten aan elkaar bekend zijn gemaakt middels een PIN-code (meestal 0000…) kan communicatie gestart worden. Helaas voor mijn tagger kan dit alleen via de BCC; de JSR-82 API voorziet er niet in om een PIN-code door te geven. Bij pairing wordt overigens ook een gedeelde 128b sleutel afgesproken, die daarna gebruikt wordt als autorisatie en/of encryptie aangezet worden. Dat laatste kan dan wel weer vanuit Java. Discovery is overigens wel mogelijk zonder te pairen; mijn tagger kan van elk device dat in bereik is en in bluetooth discovery mode staat (hetgeen bijna elke telefoon is, vanuit de winkel) de volgende attributen uitvragen: bt adres, naam, service class, services inclusief naam, UUID en URL. Als een pairing wel bestaat (in JSR-82 termen: het device is ‘trusted’), is het mogelijk om een connectie op te bouwen, bijv. simpelweg om de pingtijd te meten.

Codevoorbeelden

De betrokken Java packages zijn: javax.bluetooth, javax.obex en javax.microedition.io.
Wat betreft dat laatste package: JSR-82 heeft veel raakvlakken met J2ME (termen in het verlengde hiervan: MIDP, CLDC en CDC), wat logisch is omdat bluetooth in 1998 door Nokia bedacht is als draadloze communicatie van mobieltjes naar andere apparaten. Mocht je iets met J2ME & bt willen doen (laat me je resultaten weten!): telefoons kunnen zelf ook JSR-82 implementeren, maar niet alle telefoons met bt & Java doen dat! Zie de bluecove Wiki voor een lijst.

De API is vrij klein. De belangrijkste klassen zijn DiscoveryAgent, LocalDevice en RemoteDevice.

Twee problemen die ik ben tegengekomen met deze API:
1. Een afgrijselijke klasse is DataElement. Dit is een wrapper voor diverse soorten datatypes. Je zult er helaas mee moeten leven.
2. Als je alleen het bt adres van een device weet, dan moet je eerst discoveren. Je kunt namelijk niet op basis van dit adres een RemoteDevice (laten) instantiëren. Maar als je een URL kent kun je wel meteen connecten! Vreemd…

Nu volgen enkele zeer kleine & korte codevoorbeelden – check voor enkele compleet werkende applicaties de Eclipse workspace op de Ordina Wiki. (Helaas zijn de syntax highlighting & tabs verloren gegaan bij kopiëren vanuit Word naar WordPress…)

// setup: get local device & discovery agent
LocalDevice localDevice = LocalDevice.getLocalDevice();
DiscoveryAgent discoveryAgent = localDevice.getDiscoveryAgent();

// start global device inquiry
MyDiscoveryListener discoveryListener = new MyDiscoveryListener();
boolean limited = false;
boolean started = discoveryAgent.startInquiry(
limited ? DiscoveryAgent.LIAC : DiscoveryAgent.GIAC,
discoveryListener);

// retrieve name of first remote device in list
RemoteDevice remoteDevice = discoveryListener.getDiscoveredDevices().get(0).getRemoteDevice();
String deviceName = remoteDevice.getFriendlyName(true);

// search for services using RFCOMM on the first remote device
UUID[] uuids = new UUID[]{new UUID(BluetoothConstants.PROTOCOL_RFCOMM)};
int transId = discoveryAgent.searchServices(null, uuids, remoteDevice,
discoveryListener);

// retrieve a connection URL using the first service record
ServiceRecord record = discoveryListener.getServiceRecords().get(transId)[0];
String url = record.getConnectionURL(ServiceRecord.NOAUTHENTICATE_NOENCRYPT, false);

// open L2CAP, RFCOMM & OBEX connections
L2CAPConnection l2capConnection = (L2CAPConnection) Connector.open(url);

StreamConnection rfcommConnection = (StreamConnection) Connector.open(url);

ClientSession obexConnection = (ClientSession) Connector.open(url);
HeaderSet hsConnectReply = obexConnection.connect(null);
if (hsConnectReply.getResponseCode() != ResponseCodes.OBEX_HTTP_OK) {
// 196 = not found, 204 = precon failed, 160 = ok
LOG.error(“expected OBEX_HTTP_OK but got ” + hsConnectReply.getResponseCode());
}

Hoe service registratie werkt (dit is: opzetten van een server connectie die wacht op clients) kun je terugvinden in de Eclipse workspace.

OBEX – OBject EXchange

OBEX is ‘geleend’ van de IrDa (infrarood) wereld, vandaar dat de API een eigen package javax.obex heeft. Binnen bt werkt OBEX bovenop RFCOMM, maar OBEX ondersteunt ook IrDa en TCP/IP als transportlaag.

Ondersteunde diensten zijn o.a. FTP (phone2PC), Object Push (phone2phone – zogezegd gebruikt bij toothing, Sync, basic imaging en basic printing.

Het protocol doet denken aan HTTP. Herkenbare concepten zijn:
• Headers 12 gedefinieerde parameters, o.a. name, type en length
• Sessions op basis van connectie ID
• Content types text/vcard, x-obex/folder-listing, etc.
• Commands CONNECT, PUT, GET, SETPATH, ABORT, CREATE-EMPTY, PUT-DELETE en DISCONNECT
• Error codes OBEX_HTTP_OK, OBEX_HTTP_NOT_FOUND, etc.

Voorbeelden van uit te wisselen data zijn: vCard, vCalendar, vMessage, vNotes en platte bestanden.

De belangrijkste 4 klassen/interfaces en hun supertypes zijn:
• ClientSession > Connection
• HeaderSet
• Operation > ContentConnection > StreamConnection > OutputConnection & InputConnection > Connection
• SessionNotifier > Connection

JSR-82 implementaties & bt stacks

Bluecove is momenteel de enige actief onderhouden, bruikbare en gratis JSR-82 implementatie, maar werkt gelukkig redelijk goed. Bluecove heeft wel een bluetooth stack nodig. Voor Windows wordt widcomm aangeraden en BlueSoleil ten zeerste afgeraden – dit kan ik beamen! De bluetooth stack wordt meegeleverd met je chipset, dus kiezen kun je niet. Gelukkig bevat mijn Ordina laptop (HP Compaq 6910p) een widcomm ingebouwd. Wel eerst even een service pack van 100MB installeren! Voordat ik dat deed werd de widcomm niet herkend en ben ik met een USB bt stikkie van cygnet aan het klooien geweest. Helaas zat daar BlueSoleil bij en dat ging niet erg lekker.

Op Linux kun je gebruik maken van de BlueZ stack. Ik heb ‘m niet geprobeerd – laat me je ervaringen weten!

Een mogelijk client programma is Nokia PC Suite, die bijvoorbeeld. een remote file browser biedt. Helaas installeert Nokia zoveel dingen achter je rug om (koppelt onder andere de JAR extensie aan een Nokia application installer!) dat ik deze afraad.

Overigens werkt ook de combinatie widcomm & bluecove niet vlekkeloos. Enkele keren heb ik een JVM crash gehad, een vastlopende BT stack (een volledige herstart helpt, maar soms is killen van BTStackServer.exe voldoende), een BCC die mijn connecties afpakte (alleen een unpair/pair-actie verhelpt dit), et cetera. Als je de documentatie van bluecove mag geloven, ligt dit eerder aan de slechte staat van bluetooth stacks, danwel aan een mismatch tussen stacks en JSR-82, dan aan bluecove zelf. Toch krijg ik een beetje een déjà-vu met JMF en Java Serial: ook dit zijn Java extensies waarvoor eigenlijk nooit een goede (reference) implementatie is gekomen, en door Sun een beetje aan hun lot lijken te zijn overgelaten.

Documentatie

Er bestaat een bluetooth SIG die alle specs publiceert; onder andere die van alle profielen. Helaas moet je voor de meest informatie lid zijn, en dat kunnen alleen betalende bedrijven of universiteiten. Ik heb ze gecontacteerd voor een simpele lijst met standaard UUID nummers, en kreeg pas na lang aandringen antwoord – natuurlijk pas nadat ik mijn code af had. De online documentatie & tutorials die ik wél gevonden heb is helaas vaak slecht, oud, weinig diepgaand en lastig te vinden – ik heb hier en daar door open source code moeten snuffelen voor ontbrekende informatie (zoals voornoemde UUID nummers).

Voor mijn source code, verwijzingen naar boeken, meer technische informatie, een lijst van geregistreerde UUIDs, attributen en service classes, en extra URLs verwijs ik je naar de Ordina Wiki pagina die ik hiervoor opgezet heb.

Mocht je zelf ook aan de gang gaan met Java bluetooth, dan hoor ik graag van je terug wat je resultaten zijn. Codebijdragen zijn natuurlijk welkom op de Wiki!

Hedzer Westra

Het valt me op projecten al een tijdje op dat collega’s compilatiefouten krijgen op @Override. Deze fouten kreeg ik zelf nooit. Gisteren tijdens de JSF cursus gebeurde het weer. Weer twee mensen die tegen deze fout aan liepen. Bij de rest van de groep ging het goed, maar bij deze twee niet, ondanks dat ze dezelfde stappen doorliepen als de rest.

In de code kwamen een stuk of 20 compile errors naar voren, allemaal gerelateerd aan @Override. De melding was:

The method [METHODNAME] of type [CLASSNAME] must override a superclass method

Het probleem is een aanpassing die in Java 6 is gemaakt. In Java 5 mag deze annotation alleen op een method staan die een method uit een klasse override. Of de method in de superclass abstract is en of de superclass zelf abstract is, maakt niet uit. Wat wel uitmaakt in Java 5, is dat een method die een interface method implementeert, niet met @Override geannoteerd mag zijn.

Dit mag dus niet in Java 5.

interface MyInterface {
    void doSomething();
}
 
class MyImplementation implements MyInterface {
    // Error in Java 5: The method doSomething() of type MyImplementation must override a superclass method
    @Override
    void doSomething() {}
}

Het vervelende was dat Eclipse die annotaties neer had gezet omdat mijn Compiler Compliance Level op 6 stond en ik een Java 6 compiler gebruikte, maar niet iedere cursist gebruikte Java 6.

Leuk detail: We hebben het hier over een (per ongeluk) niet gedocumenteerde feature van Java 6. Zie Peter Ahé’s weblog.
http://blogs.sun.com/ahe/entry/override
http://blogs.sun.com/ahe/entry/override_snafu