Java Joe

This is my 2nd post about Mock Data Testing using Spring and HSQLDB.  If you missed my first post you can find it here.

I know that this concept is not really bleeding edge. The whole purpose of these two posts are to share an approach that I have used in the past to help me with my Test Driven Continuous development. I have not tried it with Hibernate or any other ORM. I have only used it with Spring JDBC Template and straight JDBC programing.

Architecture
First let me explain the requirements for your architecture to be able to support this type of mock tests.  A good architecture allows components to be unit tested.  If you need an entire environment stack to be able to unit test your code then your architecture is just a big mess.  Business logic layer needs to be separate from your data access.  Your interface layer needs to be seperate from your business logic layer.   This is true if you are doing a MVC web application or even a back end SOA implementation.  I’m not just talking about encapsulating these different layers in different classes, but also have these layers be loosely coupled.

1. Your Database queries should be externalized.  Do not hard code database queries or stored procedure calls.  I know that Hibernate allows you to annotate your code.  But can be a barrier to having your database access be portable.    However, you can get around this.  Configuration of your Database access should be loaded at run time.  Spring Beans do a nice job of handling this.  The Bean XML file gives you a great way to capture configuration information and load it at run time.  The Application Configuration allows you to load these files external to your actual business logic.
2. Your modules should be able to be run independent of any other infrastructure.  Make your modules Pojos.  Even if you plan to expose them as EJBs, MDBs or Web Services.  You can code your business logic as a Pojo.  This will allow simple clean unit tests that can run right out of the command line with no other application running.

Unit Test Implementation
So lets start with the Java code.. Below is an example of a simple Pojo that does some sort of business logic. (I left out the get/setters and import statements.)

public class BusinessPojo {

    private String sql;

    private DataSource dataSource;
    private JdbcTemplate jdbcTemplate;

    public String doSomething(String id) {
	SqlRowSet rowset = getJdbcTemplate().queryForRowSet(getSql(), new Object[] { id });

	return rowset.getString("Field1");
    }
}

This example defines 2 bean properties. The first is the SQL string that is executed, the second is the JDBC Datasource that the SQL is being executed against. The Pojo also uses the JDBC Template from the Spring framework. This is only to hide a lot of the messy JDBC code.

This is a very simple example. But it can easily be extended to take any number of input parameters and any number of returned values.
Also, this approach can be used with any framework that you can pass a JDBC Datasource and a SQL statement to.

Now let’s look at our test class:

package com.josephkampf.example;

import java.io.InputStream;
import java.net.URL;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.Statement;

import org.apache.commons.io.IOUtils;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;
import org.testng.Assert;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;

/**
 * Unit test for the BusinessPojo.
 *

 *
 * @author Joseph Kampf (me@josephkampf.com)
 */
public class BusinessPojoTest {

    private ApplicationContext ctx;

    @BeforeClass
    public void setup() throws Exception {
	setUpHSQLDB("BusinessPojoTest", "/TestBusinessPojoSchemaSetup.sql");
	setUpHSQLDB("BusinessPojoTest", "/TestBusinessPojoDataSetup.sql");
	this.ctx = new ClassPathXmlApplicationContext(
		"spring-TestBusinessPojo.xml");
    }

    /**
     * Sets up the in-memory HSQLDB using the database name and initial SQL
     * script to populate the database.
     */
    public void setUpHSQLDB(String databaseName, String initSqlScript)
	    throws Exception {

	Class.forName("org.hsqldb.jdbcDriver");

	final URL url = this.getClass().getResource(initSqlScript);
	final String sql = IOUtils.toString((InputStream) url.getContent());

	final Connection connection = DriverManager.getConnection(
		"jdbc:hsqldb:mem:" + databaseName, "sa", "");
	final Statement s = connection.createStatement();
	s.execute(sql);

	s.close();
	connection.close();

    }

    @Test
    public void testProcess() throws Exception {
	BusinessPojo pojo = (BusinessPojo) this.ctx.getBean("pojo");
	Assert.assertEquals(pojo.doSomething("ID1"), "Expected");
    }
}

I use TestNG. I Like TestNG because it is annotation driven. Let’s take a look at what is going on in this test outside of actually testing the Pojo.

Take a look at the setUpHSQLDB method.  This method takes a name and looks up via a JDBC URL an in memory HSQLDB instance.  If the instance does not exist, it is created.  So we can always reference this instance again by using the same JDBC URL.  We will see this later when we create our BusinessPojo Test Spring context file.  The method also executes some SQL scripts in the in Memory Database.  The first sets up the table structures.  The second addes the data.  I’m not going to go into the details of what is in these scripts.  I’ll leave some examples as an attachment to this posting.  I refer you to the HSQLDB online documentation for details.  But what is happening here is the database is being set up in a known state in the same JVM that our unit test is running in.  No extra processes are required.

Note that we could take the setUpHSQLDB method and create a Test Fixture.  This is actually how I have implemented it in my code.  However I leave it all in one class for convenience.  Another option would be to add the setUpHSQLDB method in your base class. (Not the way I would go.)

Now lets connect our Pojo to the HSQLDB instance we have created in our Unit Test.  Below is the Spring Configuration that is loaded by the Unit Test.

spring config:
<beans>
 <bean id="pojo" class="com.josephkampf.example.BusinessPojo">
   <property name="dataSource">
     <ref bean="dbcpDataSource" />
   </property>
   <property name="sql">
     <value>SELECT f1 as Field1 from sample_table where id = ?</value>
   </property>
 </bean>

 <bean id="dbcpDataSource" destroy-method="close">
   <property name="driverClassName" value="org.hsqldb.jdbcDriver" />
   <property name="url" value="jdbc:hsqldb:mem:BusinessPojoTest" />
   <property name="username" value="sa" />
   <property name="password" value="" />
 </bean>
</beans>

Notice the url of the DataSource.  It has the form of jdbc:hsqldb:mem:BusinessPojoTest.  Notice that this is the same JDBC URL we used in our Unit Test to load up the database.  So when the Spring template obtains a connection, it will be obtaining a connection to the in memory HSQLDB instance.

Non-Unit Test Implementation
Now when you actually go to a non-unit test environment, you will need a diffrent Spring Application Context file.  Below is an example of an Application Context that defines this Pojo to connect to an Oracle Database. This is an example of what you might use in an environment that is beyond Unit test.(Integration, QA, Stage, Production, etc.)

<beans>
  <bean id="processor" class="com.josephkampf.example.BusinessPojo">
    <property name="dataSource">
      <ref bean="oracleDataSource" />
    </property>
    <property name="sql">
      <value>SELECT oracleField1 as Field1 from oracle_sample_table where id = ?</value>
    </property>
  </bean>

  <bean id="oracleDataSource">
    <!-- Insert Oracle datasource configuration here. -->
  </bean>
</beans>

Notice that the SQL has changed to conform to the Oracle database. All that matters is that both SQL values have the same structure for the Select clause have the same fields by name and the Where clause have the same number and order of the bind variables.

Mock Data Testing is an important step to delivering on a Test driven and continuous development effort. You really should not need a running external database to be able to run your Unit Tests. Your Unit Tests should be using a known set of data so that you can do meaningful Assertions in your Unit Test code. You also need to be able to spend less time coding that Mock Data. Using SQL to define this test data is a more intuitive and faster mechanism than other solutions. Spring allows you to separate your concerns so that you can create testable units.

I wanted to share some interesting things that myself and my colleague Emil “Bud” Lefkof have done recently in the realm of Mock Testing using Spring and HSQLDB.

I expect this to be the first of threetwo posts on this topic. First I’ll explain the problem that this approach solves. In the second post I’ll explain the approach. In the last post I’ll discuss some additional concerns such as having to deal with Stored Procedures.

For an explanation of Mock Object Testing see Wikipedia who actually does a nice job of explaining what it is all about.

The reason that Mock Testing is important is that Unit tests should be repeatable and automated. Developers should be able to check out code and unit tests from a source control system and be able to run a script or command that builds and tests. Unit tests shouldn’t need to rely on databases, or other external dependencies. Plus the state of a database can become unknown. When doing test driven development you need to be able to run a test over and over again. You can’t worry about the data already being in the database.

I was first introduced to the concept of Mock Testing by my old friend and colleague Howard Spector during my time at FirstUSA/Bank One Card Services.  It was my first experience with Test Driven Development.  Back in those days we had our own Data Services Framework.  This was a framework that provided an interface that was an abstraction over Relational Database.

Because this framework was an interface we could use test implementations that would return mock data objects.  However it always felt clumsy.  Creating the test data was a manual time consuming task.  There were 2 ways to create the Mock Data objects. 

The first way is to hand code a factory that would set the values of the properties of data objects.

public List getDataPojos(String arg){
  List returnList = new LinkedList();

  DataPojo dataPojo1 = new DataPojo();
  dataPojo1.setString1("String 11");
  dataPojo1.setString2("String 12");
  returnList.add(dataPojo1);

  DataPojo dataPojo2 = new DataPojo();
  dataPojo2.setString1("String 21");
  dataPojo2.setString2("String 22");
  returnList.add(dataPojo2);

  return returnList;
}

Another option that has a little bit more flexibility is to parse a properties file or XML file.

With either approach there needs to be some sort of translation from the source to the Mock Data object that is essentially throw away code. This will cause the developer to spend more time writing code to create Mock Data objects than writing the actual tests or even the actual code.

In my next post I’ll detail how we can solve the issue of spending too much time creating Mock Data objects.