On my current project (online e-banking application for a medium/large scale bank), we needed to add Cross Site Scripting (XSS) protection afterwards. Well, actually, we had a working XSS protection mechanism in place, but the security auditors pointed out the implementation had flaws. But we’ll get to that later.

Cross Site Scripting?

First, let me explain what Cross Site Scripting (XSS) is. XSS is a hack where malicious code (JavaScript/HTML) is injected into a trusted site. With trusted, I mean the user trusts the site, which is usually the case with e-banking websites. At least, I hope so.

A simple example is a web site which contains a search box, like the following.

<form action="/search.do" method="GET">
  <input type="text" name="searchString" value="Enter something here" />
  <input type="submit" value="Search" />
</form>

When the user submits the form, a request is made to the following URL: /search.do?searchString=[THE_USER_INPUT]

The application responds with a page, containing the results, including a summary of the search criteria, and also the search form, prefilled with the search string the user entered, like here.

<!-- Some HTML -->
 
<form action="search.jsp" method="GET">
  <input type="text" name="searchString" value="${param.searchString}" />
  <input type="submit" value="Search" />
</form>
 
<table id="results">
  <!-- The search results -->
</table>
 
<!-- The rest of the site -->

So far, so good, right?

No, wrong!!!

Suppose a hacker enters the following value into the searchString box:

"/><script>alert("hello")</script><p id="

You’ll see an alert box with the message: “Hello”. You have injected a script into your web page.

But why would anyone hack his own browser session? That’s not the danger here. The danger is that a hacker can create an URL, like

http://www.yoursite.com/search.do?searchString="/><script>alert("hello")</script><p id="

and send it to other people.

When they click on it, the script gets executed in their browser.

Of course, an alert window is harmless, but what if I inject some DOM scripting to create a form where the user must enter his or her credentials for the attacked site? Or an image to automatically trigger a “Button click” (this is actually a case of Cross Site Request Forgery)? With such a leak, I can do almost anything on your page, without you knowing.

Let’s see what happens when the login page of your bank is vulnerable to XSS. Phishing becomes easy as stepping on kittens!

As you’re probably now aware of, XSS is a great “enabler” for all kind of other hacks. And the fun part is, hackers can spot XSS flaws easily and often using automatic tools. Not scared yet? In that case, I hope you’re not working on the website where I perform my electronic payments! You should be scared for these kind of leaks.

Countermeasures

In theory, it is very easy to protect your website against XSS. Just escape all variables, according to the escaping rules that apply for the markup/script language you are about to put the variables in. Apache Commons Lang has a utility for this purpose: the class StringEscapeUtils with its method escapeHtml and some others. escapeHtml is probably the method you’ll need most, since most variables are outputted in HTML tags.

Why is this fix enough? Well, just test it out with the example above. We modify the JSP to escape the input before rendering it to the client, as shown below.

<%
  String param = request.getParameter("searchString");
  if (param == null) param = "";
  String escaped = org.apache.commons.lang.StringEscapeUtils.escapeHtml(param);
%>
<form action="search.jsp" method="GET">
  <input type="text" name="searchString" value="<%=escaped%>"/>
  <input type="submit" value="Search"/>
</form>

The difference is that I added an escaping routine here. Nothing fancy, except that the “obvious” happens. The user input is shown on the screen in exactly the same way the user has entered it.

You see, no validation or black-/whitelists are needed, which is great, since you don’t want to bother users with security.

Some caveats, which become quite annoying later on.

• Timing is essential with regards to escaping. You cannot do it on your input parameters, since you’ll then end up with HTML encoded texts in your backend systems (that was the flaw I was talking about in the introduction). You also don’t want to be bothered with HTML encoding in your Java business logic, so input filtering is not the way to go. You definitely need to filter on the output, just before you render your page back to the client.
• You may not escape a variable twice, since the user ends up with HTML codes in his/her UI. When escaping > twice, it first becomes > and after the second pass, it becomes >, which is not only wrong with regards to usability, it also opens up some interesting new security holes.
• When you persist user input unencoded, which you should, you also need to escape all variables that come from there before rendering the web page. Essentially, you just need to escape every variable that is inserted into the web page, since it may (directly or indirectly) be manipulated by a malicious user.

Implementation

So, we need a way to escape all user input just before it is rendered. What are the options?

• Custom JSP function
  Probably the easiest way to escape special characters is by implementing a JSP custom function, like the JSF “outputText” component:
  

  <h:outputText escape="true" value="<script>" />

  . This one is easy to implement. It’s just a burden to edit all 200 JSP files (and the same amount of JSP Tag files) in our project and add the function at all places where a variable is used. The second issue is that you also need to be sure you don’t escape a variable twice, since that results in strange output and, ironically, new security holes. This is especially the case with tag attributes. Do you escape in the tag or in the calling code? You’ll need a strategy there. Finally, and probably most importantly, this solution will probably become a maintenance nightmare. JSPs are cluttered with these custom functions and future maintainers may never forget one when making changes. Something more generic would be nice.

• ELResolver?
  ELResolvers are added to JSP with version 2.1. We use Tomcat 6, so that’s great. However, the ELResolver mechanism differs greatly from, for example, the JSF VariableResolver mechanism. VariableResolvers are implemented using decoration, which makes them very useful to intercept all variable access and modify the responses. ELResolvers are implemented as a CoR and the implementation makes it impossible to create a custom filtering mechanism for all variables. It is also not the right place for this kind of logic. Why? Imagine a JSP Tag file. In the calling JSP file, you use EL to pass an argument to the tag and in the JSP Tag file, you use EL to put the argument in the HTML. With an ELResolver solution, variables get escaped twice. Not good.
• Tomcat hooks, AOP, Javassist?
  There are other ways to hook into the JSP/Servlet lifecycle. Containers provide vendor specific hooks to hook into the lifecycle, but unfortunately, no Tomcat hooks to help us here. But we’re not defeated yet! We can use AOP or a bytecode library to enhance some core Tomcat classes. Of course, we’re entering the black magic room here, but if it works… Unfortunately, these are also no real options. We can for example intercept all calls that go into the EL runtime by proxying all EL calls, but this has almost all of the issues as a custom ELResolver. Darn!
• Servlet Filter?
  But what’s wrong with a plain old Servlet Filter which escapes HTML special tags afterwards? The issue with a Servlet Filter is that it can’t possibly see the difference between legitimate and injected markup. The Filter Either escapes all markup (including your own) or nothing (which renders it useless). After the JSP has been executed, you’ll end up with HTML. If there is an XSS attack, you are simply too late when you use a Servlet Filter. So, it has to do with timing. Let’s step back.
• Object graph XSS filtering!
  There are two issues with most solutions presented.
  • Timing: The solution is too early or too late, making it useless or wrong.
  • no. executions: The solution has the risk of escaping the variable more than once, making it wrong and unsafe.

  So, what’s the best point in the request processing lifecycle to do the escaping? It turns out that just before the JSP is executed is the ideal moment. It’s ideal, because you’re not too early, so you won’t notice anything in your Java code. It’s also ideal, since you have full control over execution, making it easier to execute once and only once.
  

Object graph XSS filtering

The graph walker sounds like a solution, however, it’s a difficult one to implement.

The reason is the flexibility of our MVC framework. It’s allowed to pass arbitrary object graphs to the JSP. This is an issue, since you can’t just iterate over all attributes and escape them. For example, some attributes may be collections, maps or domain objects, containing possibly unsafe strings.

So we need an object walker. What features should it have?

• Cycle detection, to prevent infinite loops.
• Ability to modify arbitrary objects, either using reflection (POJO’s) or API (Collections).

Implementation sources

A hack and slash implementation of this solution is attached. What do you guys think of it? Does it look like a workable solution?

Ps. I’m aware of the fact that I haven’t implemented any pattern or followed any best practice. It’s just a simple PoC.

Click here to download the ObjectWalker sources

Spring Roo (Milestone2) is the newest Spring brain child. This time however not a (integration) framework, but a real application generation tool.

Using a command line tool (Yes really) you not only tell Roo to generate code for you, but you also use it to modify your generated code.
For example with a simple:

create project -topLevelPackage nl.ordina.compcomp

a project is created.

And with a:

new persistent class jpa -name ~.domain.Competence

a JPA class is generated.

Adding attributes to the JPA class is also done wit a command like:

add field string description

Web interface generation is done by:

new controller automatic ~.web.CompetenceController

In short this is it. A web application with CRUD functionality. The fun part is however the code that is generated. One statement from the Spring Roo website that really intrigued me, is the ability of Roo to once generate your application and the allow you to continue without using Roo. How did the solfe the generation gap and how is it that I can continue developing the application without Roo.

So lets see what Roo has generated:

package nl.ordina.compcomp.domain;
 
import javax.persistence.Entity;
import org.springframework.roo.addon.entity.RooEntity;
import org.springframework.roo.addon.javabean.RooJavaBean;
import org.springframework.roo.addon.tostring.RooToString;
 
@Entity
@RooEntity
@RooJavaBean
@RooToString
public class Competence {
 
private String description;
}

Thats it. But you say where is the rest then. Or do they have some fancy runtime framework that acts on the beautiful annotations. Well yes and no. Besides the java class a number of AspectJ files are also generated. And these files contain the extra generated code. The generation gap is solved by adding functionality using AOP. The aspect files are generated over and over again. The java class file however is only generated once and can be modified by the programmer.

If you like to continue without Roo, simply tell it to Roo. What will happen is that the Roo annotation are removed from your java files and the Aspects will be merged into your class. At that point the Roo dependency is no longer needed.

Is this the way to go? Who knows must tell. I don’t. The code is still in Milestone phase so it is not written in stone and may contain some bugs or missing functionality. The reasoning behind Roo and some implementation however choices I personally find interesting.

What do you think? (more…)

So, you’ve probably been using Java 5 for a while now..? How do you like the new features?

One of the new things is autoboxing. Java is slowly realizing that having primitives wasn’t the best idea. Having only objects instead of primitives (like modern languages, for example Scala) feels more natural and fits OO-programming.

To ease the pain Java introduced this bandage: Autoboxing.

Autoboxing means automatic translation (in some cases) of primitives to the corresponding objects.


So you can do:

int primitiveFifty = 50;
Integer objectFifty = primitiveFifty;

Even better, it allows you to automagically do int-calculations with objects:

Integer calculate = 10;
calculate += 20;

Java translates everything into int, then back to Integer without you noticing anything. This is great! (NOT!)


Autoboxing works great with simple examples like this, but it gets spooky when you use it in combination with widening and narrowing. Let me give you an example:

Byte b = 1;

This is clear, the ‘int’ 1 will be translated without noticing to a Byte. In the following example, the output is too as expected:

Byte b = null;
 
Byte n = 10;
if(false) {
    n = 1;
} else {
    n = b;
}
System.out.println(n);

The result of this piece of code will be ‘null’. Of course, the false will be executed, and then ‘n = b’ will be called. This causes n to be null.


Lets inline this if-then-else to do the same, but shorter:

Byte b = null;
 
Byte n = 10;
n = (false) ? 1 : b;
System.out.println(n);

The code is 100% the same as the example above isn’t it, we only inlined the if…? We do if(false) and the result is either ‘n = 1′ or ‘n = b’…?


Wrong!


When you execute the code you’ll get a NullPointerException!


Why is this? Let us translate the examples back to Java 1.4 code, without using autoboxing:

Byte b = null;
 
Byte n = Byte.valueOf(new Integer(10).byteValue());
if(false) {
    n = Byte.valueOf(new Integer(1).byteValue());
} else {
    n = b;
}
System.out.println(n);

As we can see, Java does a lot to go from int to Byte.

The translation of our second example is less inituative. The following ‘rules’ are from the language specification for the conditional ‘(Operand1 ? Operand2 : Operand2);’.


We have the following code in Java 5: (false) ? (int)1 : (Byte)null);

In our case, the second (int) and third (Byte) are both convertable to numeric types. The third sub-rule applies to our case:


“If one of the operands is of type Byte and the other operand is a constant expression of type int whose value is representable in type byte, then the type of the conditional expression is byte.”


Now we can translate the inline-code to the Java 1.4 variant:

Byte b = null;
Byte n = new Byte((byte)10);
n = Byte.valueOf((false)? (byte)1 : b.byteValue());
 
System.out.println(n);

As you can see, Java tries to get the byte value in both cases. To do this on b, it needs to call .byteValue(), which results in a NullPointerException…


So how do you safely inline it? You have to make sure the two values in the conditional have the same type as the type you expect, in our case:

Byte b = null;
Byte n = 10;
 
//Make sure we have Byte = ()?Byte:Byte;
n = (false)? (Byte)(byte)1 : b; 
 
System.out.println(n);

So, be alert when using autoboxing and widening/narrowing in the same piece of code. Happy coding!

Don’t code what you can generate

At my currrent project I ended up generating run-time proxies for my Business Objects with CGLIB. As this is somewhat unusual for a straightforward enterprise application, I still wonder about the pros and cons of this approach.

The main thing the application does is product configuration – given a definition of some tree of products, it lets the user create and edit product attributes, add, remove and edit subproducts according to that definition. It is a requirement that there is an external definition and it is expected to change regularly, so as much as possible of the product logic should be defined there.

A fragment of some imaginary domain objects definition to illustrate the case:

<productdefinition id="someProduct">
 <date id="startingDate" nullable="false">
 <enumeration id="area">
  <enum default="true" value="World">
  <enum value="Europe">
 </enumeration>
 <boolean id="allInclusive" defaultvalue="false">
 <range id="limit" min="10" max="5000" step="10" defaultvalue="1250">
 <subproduct type="someSubproduct">
  <multiplicity mincount="0" maxcount="1" defaultcount="1"/>
 </subproduct>
<productdefinition>

<productdefinition id="someSubproduct">
 <text id="reason" pattern="\a{1,30}"/>
</productdefinition>

Whatever the Java implementation:

• When creating SomeProduct some values have to be set initially: setArea(‘World’), setLimit(1250) etc. A properly initialized instance of SomeOtherProduct should be added as a subproduct.
• someProduct.setStartDate(null) or someProduct.setArea(“Japan”), someProduct.setLimit(10000) should all cause an error with an error code because the provided value is not acceptable according to the definition.
• Rendering a GUI for editing a product attribute and processing the user input is uniform for all product attributes
• etc. etc.

An intuitive approach was to try something PropertyBean-like with named properties:

interface Product {
 Object getPropertyValue( String propertyName);
 void setPropertyValue( String propertyName, Object propertyValue);
 AttributeDefinition getAttributeDefinition( String propertyName);
. . .
}

class ProductImpl implementing Product { . . . }

The implementation is straightforward and a lot of code fits this model perfectly. Still, there is code in the use case implementations which looks somewhat clumsy and prone to errors with this interface:

someProduct.setPropertyValue("someSubproduct[0].reason", "Lousy service");

An example without nested properties looks even more awkward, you can imagine. Forget code completion, type checking, tool refactoring facilities. The natural Java way of doing this is:

someOtherProduct.getSomeProduct().get(0).setArea("Europe");

And you need to invent some trics to write the equivalent of:
<form:input path=”someProduct.someSubproduct[0].reason”/>

The interface that I actually would like for the use case implementations is:

interface SomeProduct {
 Date getStartingDate();
 void setStartingDate( Date date) throws BadValueException; // Null not accepted
 List getSomesubproduct();
 SomeSubproduct addSubproduct();
 void deleteSubproduct( Subproduct p);
. . .
}

interface SomeSubproduct() {
 String getReason();
 void setReason( String reason);
}

Given existing ProductImpl instances, coding the implementations of those interfaces would be a repetitive task, all getter/setters/addXXX/deleteXXX etc. will look alike. A code generation library could generate this trivial code for us. CGLib is a code generation library (a lot of tautology here) and I know it is good enough for Hibernate, so I thought it should do the job for me too. With a productImpl instance and a desired interface, the CGLib code needed to produce the proxy is surprisingly little:

class ProductProxyFactory {

 public static Object createProxy(ProductImpl productImpl, Class clazz) {
  Enhancer e = new Enhancer();
  e.setSuperclass(clazz);
  Callback callback = new ProductProxyCallback(clazz.getSimpleName(),
    productImpl, productMapping);
  e.setCallback(callback);
  return e.create();
 }
}

public class ProductProxyCallback implements MethodInterceptor {
 private ProductImpl productImpl;

 public Object intercept(Object object, Method method,
                         Object[] args, MethodProxy  proxy) throws Throwable {
  String methodName = method.getName();
  if ( methodName.startsWith("get")) {
    String propertyName = findMappedPropertyName(methodName); // find mapped property from method name
    Object propertyValue = productImpl.getPropertyValue(propertyName );
    Class subProductClass = findSubProductClass(propertyName ); // property value might be a product too
    if (subProductClass != null) {
      return createSubproductProxy(result, subProductClassName, mapped);
    } else {
      return result;
  } else {
   // Process setXXX, addXXX, deleteXXX methods and methods inherited from Object
 . . .

Let’s see an example:

// There is some old code that still creates product using the clumsy Product interface
ProductImpl productImpl = new ProductImpl( someProductDefinition);
// and sets some data
productImpl.setProperty( "startDate", today);

// Let’s create a proxy implementing SomeProduct interface:
SomeProduct someProduct = (SomeProduct)createProxy( productImpl, SomeProduct.class);
Date date = someProduct.getStartingDate()

The code above is schematic, of course, in reality the application does not have to create any proxies explicitly. Not bad, the old classes have received a new face – another interface to the same data, and the only code that has to be written for each new interface is the interface definition itself.

Pros & Cons

- This approach adds complexity. This part of the application has confused more than one project members joining the team – ‘Hey, why can’t I find the implementation of SomeProduct?”. Is it worth maintaining this extra complixity for the needs af small audience – the developpers of one application?
- The code above is schematic, the real implementation has to take care of more details – external mapping file, caching proxy instances etc. The real code is already more complicated than the example and could potentially become too complicated, as business analysts add new requirements to the application.
- Object identity issues pop up if you use proxies and this case is not different – it is possible to create many proxies working on the same underlying productImpl.
- Some OOP has gone here. Try to override SomeProduct.getStartDate() or to set a breakpoint on it. I guess it happens when you mess up with AOP.

Let’s not forget the pros:

- No typing, retyping and copy-pasting of trivial code or introducing errors there. Changing a product is limited to editing tha Java interface and a mapping file.
- There is one well-defined place where all products are defined. There is one well-defined class where the behaviour of all products is defined, and no lazy or evil programmer can override it.

Is that DSL?
I wonder how this approach relates to the DSL concept of trading generality for expressiveness. The starting point is undoubtedly a domain-specific XML. While many DSL tools emphasize the code-generation phase, here objects are generated at run time. An obvious difference is that one can not inspect/change/override the generated code. While this is a DSL according to scholars (see http://ftp.cwi.nl/CWIreports/SEN/SEN-E0309.pdf), it is probably not the sort that is hot right now. Would it be more DSL if Java code was generated? I am curious on your thoughts on this.

You’ve probably heard people talking before about ‘cloud computing’.
But what exacly is this cloud computing you might ask?

To figure this out I decided to create my own Google App Engine project and find out about cloud computing along the way.

Wikipedia states that cloud computing is:

“Cloud computing is a style of computing in which dynamically scalable and often virtualized resources are provided as a service over the Internet. Users need not have knowledge of, expertise in, or control over the technology infrastructure in the “cloud” that supports them”

So, cloud computing doesn’t only have a vague name, the description isn’t very helpful either. But the key ingredients are “scalable” and “virtualized” and “as a service”. And you don’t have control over the infrastructure…

Lets take a look at Google App Engine. It was released in April 2008 as a platform for developing (Python)cloud applications. App Engine hosts these applications virtually on many machines, the programs are distributed and scaled across a vast amount of servers. And now, since the beginning of this year Google App Engine also supports Java!

The concept is simple, you get to build an application, preferably using Google’s App Engine-eclipse-plugin. And the JRE you build on is a slightly stripped-down version to make it usable on a cloud, like a sandbox.

Because you don’t know on what kind of servers your application will run on, or on how many servers, Google has decided you can’t do the following:

• Start threads
• Go to the Filesystem, no I/O
• Open sockets directly, but you can open connections through HTTP/HTTPS
• Make calls to System (like exit();, gc(); etc)

And there is more. Because of these restrictions you can’t access a database! So you can’t use something like Hibernate and some Oracle/MySQL/Postgress machine. To still be able to save/persist objects Google has teamed up with Datanucleus. Using JDO or (a stripped down version of) JPA you can persist and retrieve objects on the cloud.

With this in mind I started making my own application. And the frameworks I wanted to use are:

• Wicket (Web Framework)
• Spring IOC
• Spring ORM (for transaction management, using annotations)
• JPA (instead of the default JDO)

The first problems I encountered was getting Wicket to load. Because of the sandbox-restrictions there are a couple of things you can’t do. For example, Wicket can’t save temporary data to disk (what it normally does). And there are problems with Wicket being in ‘development-mode’ where is wants to start Threads to poll for changed resources.

A good overview on what it takes to get Wicket working is explained here:
http://www.danwalmsley.com/2009/04/08/apache-wicket-on-google-app-engine-for-java/

Next up was installing and running Spring. This was relatively easy at first. The core Spring code ran pretty much as expected. I added the JARs to my project and added this to the web.xml:

	<!-- Spring -->
	<context-param>
		<param-name>contextConfigLocation</param-name>
		<param-value>classpath:applicationcontext-*.xml</param-value>
	</context-param>
	<listener>
		<listener-class>
			org.springframework.web.context.ContextLoaderListener
		</listener-class>
	</listener>
	<listener>
		<listener-class>
			org.springframework.web.context.request.RequestContextListener
		</listener-class>
	</listener>

As you can see, I load up multiple XML files. I decided to go with the all-out-annotations method using Spring ORM. This proved to be pretty challenging…

With these annotations you are able to do the following in the code:

@Repository("loginDao")
@Transactional
public class LoginDaoImpl implements LoginDao {
 
	@PersistenceContext
	private EntityManager entityManager;
	... (and more)

As you can see I’m using Spring to inject my EntityManager into the DAO. But you can’t just load the entity manager in Google App Engine, you need a specific piece of configuration. I used the following XML:

	<bean id="data.emf"
		class="org.springframework.orm.jpa.LocalEntityManagerFactoryBean">
		<property name="persistenceUnitName" value="transactions-optional" />
	</bean>

	<bean class="org.springframework.orm.jpa.JpaTemplate">
		<property name="entityManagerFactory" ref="data.emf" />
	</bean>

	<bean id="transactionManager"
		class="org.springframework.orm.jpa.JpaTransactionManager">
		<property name="entityManagerFactory" ref="data.emf" />
	</bean>

To tell Spring to scan for these annotations you need to add the following lines in your applicationcontext:

	<context:annotation-config />
	<context:component-scan base-package="nl.redcode.*" />

And now the problems start… The first problem is that Google App Engine doesn’t support all core classes. When loading these annotations Spring will load its PersistenceAnnotationBeanPostProcessor. But it contains the following piece of code:

try {
	return (EntityManagerFactory) lookup(jndiName, EntityManagerFactory.class);
}
catch (NamingException ex) {
	throw new IllegalStateException("Could not obtain 
		EntityManagerFactory [" + jndiName + "]from JNDI", ex);
}

And the Exception we get is:

org.springframework.beans.factory.BeanCreationException: Error creating
bean with name
'org.springframework.context.annotation.internalPersistenceAnnotationProcessor':
Initialization of bean failed; nested exception is
java.lang.NoClassDefFoundError: javax/naming/NamingException
	at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:480)
	at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:409)
	at java.security.AccessController.doPrivileged(Native Method)
	at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:380)
	at
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:264)
	at
...etc

And when I looked at the white-list it was clear, NamingException isn’t part of the sandbox Google App Engine uses. So I started to google how to solve this. The first thing I encoutered was somebody who added the following lines to his/her applicationContext:

	<bean id="org.springframework.context.annotation.internalPersistenceAnnotationProcessor"
		class="java.lang.String" />

This piece of code, when executed before the annotation-scan, loads a String in the Spring Container under the name “internalPersistenceAnnotationProcessor”. This causes Spring to ignore its own instantiation of the PersistenceAnnotationBeanPostProcessor and we don’t get the Exception anymore.

But this causes some more damage we don’t want in the application. Before my Dao’s received a valid EntityManager, but they are Null now…!

So I took the code of the original Spring PersistenceAnnotationBeanPostProcessor and replaced all the instances of NamingException with just Exception. This removed the dependency to NamingException. I called this new bean “AppEngineJPAPostProcessor”. This is how I configured it in the applicationContext:

	<bean id="org.springframework.context.annotation.internalPersistenceAnnotationProcessor"
		class="nl.redcode.springhack.AppEngineJPAPostProcessor" />

The EntityManager(Factory) is now created, it gets injected into the DAO’s, they have transactions using annotations and everybody is happy!

When I got a little further in my project I decided to deploy my application to the cloud and test it online. Deploying your application to App Engine is very simple, just push the “Deploy” button in the Eclipse plugin and you only need to enter your credentials and a version-number of your release!

But then the old BeanPostProcessor bit me in the back again. On the server I got the following Exception when deploying:

java.lang.SecurityException: Unable to get members for class org.springframework.jndi.JndiLocatorSupport
	at com.google.apphosting.runtime.security.shared.intercept.java.lang.Class_$10.run(Class_.java:357)
	at com.google.apphosting.runtime.security.shared.intercept.java.lang.Class_$10.run(Class_.java:347)
	at java.security.AccessController.doPrivileged(Native Method)
	at com.google.apphosting.runtime.security.shared.intercept.java.lang.Class_.getMembers(Class_.java:347)
	at com.google.apphosting.runtime.security.shared.intercept.java.lang.Class_.getDeclaredMethods(Class_.java:174)
	at org.springframework.util.ReflectionUtils.doWithMethods(ReflectionUtils.java:460)
	at org.springframework.util.ReflectionUtils.doWithMethods(ReflectionUtils.java:443)
	at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.findAutowiringMetadata(AutowiredAnnotationBeanPostProcessor.java:299)
	at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessMergedBeanDefinition(AutowiredAnnotationBeanPostProcessor.java:179)

It seems the runtime is a bit more strict then the development server Google App Engine uses. For some reason it doesn’t like the JndiLocatorSupport. This is excepted because Google App Engine, due to the nature of the cloud, prohibits the use of JNDI.

Soon I found the problem, the only reference to JndiLocatorSupport is my own BeanPostProcessor:

/**
 * Rewritten for use in Google AppEngine
 *
 * @author Roy van Rijn
 */
public class AppEngineJPAPostProcessor extends JndiLocatorSupport implements
		InstantiationAwareBeanPostProcessor, BeanFactoryAware {
 
	private Map persistenceUnits;
	...

When I removed the ‘extends’ part there were two pieces of code that stopped working, they both look like this:

	try {
	return (EntityManagerFactory) lookup(jndiName, EntityManagerFactory.class);
}

It seems this PostProcessor always does a JNDI lookup to find the correct EntityManager! But we can’t do this if we don’t have access to the JndiLocatorSupport methods anymore. So I decided to hack a little bit in this code, my solution was to load the EntityManagerFactory and EntityManager from the Spring container:

try {
	return ((JpaTransactionManager)beanFactory
		.getBean("transactionManager"))
		.getEntityManagerFactory();
/*	return (EntityManagerFactory) lookup(jndiName,
		EntityManagerFactory.class);*/
} catch (Exception ex) {
...

This solved all the problems and with the changes to the BeanPostProcessor I’m now able to use all the Spring annotations, for persistency and transactions, in Google AppEngine.

I’m now still in the middle of developing my application on Google App Engine, but it seems that Google App Engine works like a charm. The problem is, most frameworks can’t really cope with the sandbox out-of-the-box. But with (some) minor patches and tweaking most frameworks will run using Google App Engine. The only major problem I’m having (which can’t be solved) is Datanucleus. I chose to use JPA because it is much richer and has more features then JDO, but Datanucleus hasn’t implemented much of these features yet.

For example, I had the following annotation on a field: @Column(unique=true)
Datanucleus threw “java.lang.UnsupportedOperationException: No support for uniqueness
constraints“.

Also, I created a query with this: “username = :username OR emailAddress = :emailAddress”
But Datanucleus doesn’t support the operator “OR”.

Other things DataNucleus can’t currently do:

• Many-to-many relationships
• Joins in a query (WHAT??)
• Aggregation queries (group by, having, sum, avg, max, min)
• Polymorphic queries. You cannot perform a query of a class to get instances of a subclass. Each class is represented by a separate entity kind in the datastore.

So, we’ve seen what a cloud is, what Google App Engine is, and I explained some tweaks/patches needed to get Wicket and Spring working.

Why use Google App Engine? It is free (up to some CPU/mail/data limits) and your project runs on a cloud, and thus is very scalable. You don’t have to worry about the environment or server. Your application will scale when its needed and everything is pre-installed and ready to run.
But be prepared for difficult classloading issues and missing classes. You just can’t expect all frameworks to be working out-of-the-box with the sandbox-limitations. Also don’t expect much support with persisting, the JPA-support is very minimal and won’t let you do much more then persisting and retrieving single objects.

Enough blogging, now its time for me again to tinker on my application, maybe I’ll tell about it here in the future!

JavaOne 2009 has just started. The worlds leading Java conference will be held from 2-5 June in San Franciso (US). If you are interested you can follow the general sessions via the JavaOne homepage, due to the time difference in Europe you can watch and listen to the general sessions around dinner time. Check it out: http://java.sun.com/javaone/

I`ve just finished watching the opening session which contained a lot of promotion for JavaFX. Sony talked about Java enabling interactive Blu-ray applications. Verizon anounced a partnership with Sun for  applications for their wireless network.

Larry Allison, CEO of Oracle, was the last guest speaker. He talked about how excited he was about the Oracle takeover of Sun and Java being the key platform for current and future Oracle (Fusion) products. He also mentioned that Sun and Oracle maybe hitting the mobile/nettop market and launching a Google Android alike platform.

I`ll be posting some JavaOne updates this week.