Java testing using XStream, AspectJ, Vsdf

Problem
A unit or integration test may require the creation of complex object structures. This is even more problematic in legacy systems that were not created with testability in mind. Faced with this, it may seem better in the short term to avoid programmatic tests and continue to rely on traditional test techniques.

Solution
A possible short term approach is to use object serialization to capture the required objects as they are used in the actual system. Then a test can rebuild the objects and use them in various tests. We will not elaborate on the shortcomings of this approach.

XStream
Their are many approaches in Java that can be used for object serialization: JAXB, JavaBean serialization, and so forth. All of them have issues. For example, many of them require that the object to be serialized conform to certain requirements, like the JavaBean specification. If the objects don’t there are ways around it, but this quickly becomes complex, not only must the top level object be ‘handled’ but then its nested object graph. The XStream library does not have these requirements and so can probably handle a large percent of the use cases.

Example Test
In listing 1 below is a unit test using XStream. A class called XApp contains a scenario1 method that we wish to test. To invoke scenario1, a lot of data must be created and structured into a required object hierarchy, here we just use a simple map to represent that. Thus, the test creates an object from an XStream XML serialization and then invokes the scenario1() method of the system under test.

[expand title=”Listing 1, test”]

package com.octodecillion.testing;

import static org.hamcrest.core.Is.is;
import static org.junit.Assert.assertThat;

import java.io.File;
import java.io.IOException;

import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.JUnit4;

import com.octodecillion.testing.XApp.Inventory;
import com.thoughtworks.xstream.XStream;
import com.thoughtworks.xstream.io.xml.StaxDriver;

/**
 * @author jbetancourt
 *
 */
@RunWith(JUnit4.class)
public class InventoryTest {
	/**
	 * @throws IOException 
	 * 
	 */
	@SuppressWarnings("boxing")
	@Test
	public void scenario1_test() throws IOException {
		Inventory inventory = (Inventory) new XStream(new StaxDriver())
				.fromXML(new File(
				"inventory-scenario1.xml"));

		XApp app = new XApp();
		app.setInventory(inventory).scenario1();
		assertThat(app.getInventory().stock.size(), is(1));
	}

}

[/expand]

AOP
Since there are no mappings or modifications to objects, serializing with XStream takes two lines of code. We can insert this source code (or via a utility) where we need it in the system under test to capture a state for future testing. However, this requires that we litter the code with these serialization concerns. We can remove them after we capture the object, of course.

An alternative is to use Aspect Oriented Programming. With AOP we can ‘advise’ at ‘joinpoints’ to capture objects. This can be done with Load-Time Weaving. The original source is unmodified and to recapture the same objects we just rerun the system and reapply AOP LTW.

In listing 2, the AspectJ AOP Java framework is used to create an aspect to capture the data object at the setScenarioData() method in the class under test, XApp.

[expand title=”Listing 2, object capture”]

package com.octodecillion.testing;

import java.io.File;
import java.io.FileWriter;

import com.thoughtworks.xstream.XStream;
import com.thoughtworks.xstream.io.xml.StaxDriver;

/**
 * Aspect to allow XStream streaming of application data object.
 * 
 * @author jbetancourt
 *
 */
public aspect XStreamObject {
	/**
	 * Capture the App inventory object during creation of data 
	 * and stream to file.
	 * 
	 */
	@SuppressWarnings("javadoc")
	after(final XApp app) : 
		execution( protected void setScenarioData()) && 
		!within(XStreamObject) && target(app) {
		
		try {
			FileWriter writer = new FileWriter(
					new File("inventory-scenario1.xml"));
			new XStream(new StaxDriver()).toXML(app.getInventory(), writer);			
			writer.close();			
		} catch (Exception e) {
			e.printStackTrace();
		}		
	}	
}

[/expand]

Storing using Vsdf
One problem with using serialization for tests is where to store these. If we have a complex app, which we do or else why go through all this, there can be a lot of objects and these can be in various states depending on the test scenario requirements.

In prior posts Simple Java Data File and very simple data file I presented a concept for just this scenario. With Vsdf, multiple serializations can be stored in one file. Thus we can store all streamed object in one file, or various serialized states of an object type can be stored per Vsdf file. Some advantages of this approach is the reduction in test files, ability to change or update individual serializations.

Example Class to test
In listing 3 below is the example class to test. It is more of a “Hello World” type of app.

[expand title=”Listing 3, target class to test”]

package com.octodecillion.testing;

import java.io.IOException;
import java.util.HashMap;
import java.util.Map;

/**
 * Simple XStream example.
 * 
 * @author jbetancourt
 *
 */
public class XApp {

	private Inventory inventory;
	
	/**
	 * The scenario that needs testing.
	 */
	public void scenario1() {
		// do the business case here ...
	}

	/** default constructor */
	public XApp() {
		setScenarioData();
	}

	/**
	 * The Application entry point.
	 * @param args
	 * @throws IOException 
	 */
	public static void main(final String[] args) throws IOException {
		new XApp();
	}

	/**
	 * set up the data for scenario1
	 */
	protected void setScenarioData() {
		inventory = new Inventory();
		inventory.add("322", new Vehicle() {
			@Override
			public int getNumWheels() {
				return 4;
			}

		});
	}

	/**
	 * @author jbetancourt
	 *
	 */
	public class Inventory {
		/**		 */
		Map<String, Vehicle> stock = new HashMap<String, Vehicle>();

		/**
		 * @param id
		 * @param veh
		 */
		public void add(final String id, final Vehicle veh) {
			stock.put(id, veh);
		}

	} // end class Inventory

	/**
	 * @author jbetancourt
	 *
	 */
	public interface Vehicle {

		/**
		 * @return num of wheels
		 */
		public int getNumWheels();

	} // end Vehicle

	/**
	 * @param inventory
	 * @return the app object
	 */
	public XApp setInventory(final Inventory inventory) {
		this.inventory = inventory;
		return this;
	}

	/**
	 * @return the inventory
	 */
	public Inventory getInventory() {
		return inventory;
	}

} // end class XApp

[/expand]

Issues
Sounds great but what happens when a class that was serialized is modified, like gets new fields? How do we handle versioning and so forth?

Links

Java JMockIt mocks via Spring DI

How to use Dependency Injected mock objects to allow Integration Testing of a Java application.

When working with some legacy code bases, the introduction of Dependency Injection has limitations. Thus, various forms of ‘instrumentation’ will be required to reach the “last mile”. A major difficulty with legacy code is instantiation of objects with the “new” operator.

One form of instrumentation is a modern Mocking framework, like JMockIt, another is the use of AOP with, for example, AspectJ.

As part of an evaluation of a possible approach I looked into using DI for data driven Integration Testing. Is it possible to use declarative specification of JMockIt mock objects? That is, can they be specified in a Spring XML bean configuration file and loaded via Spring Framework DI? This is in lieu of Using AspectJ to dependency inject domain objects with Spring or some other framework.

Short answer, yes. Useful technique? Probably not. But …

I was able to solve this by looking at the JMockIt source code for the mockit.internal.startup.JMockitInitialization class.

From JMockIt source repository:

final class JMockitInitialization
{
  ... snipped ...
  private void setUpStartupMocksIfAny()
   {
      for (String mockClassName : config.mockClasses) {
         Class<?> mockClass = ClassLoad.loadClass(mockClassName);

         //noinspection UnnecessaryFullyQualifiedName
         if (mockit.MockUp.class.isAssignableFrom(mockClass)) {
            ConstructorReflection.newInstanceUsingDefaultConstructor(mockClass);
         }
         else {
            new MockClassSetup(mockClass).setUpStartupMock();
         }
      }
   }
  ... snipped ...
}

See the “Using mocks and stubs over entire test classes and suites” in the tutorial for further info.

Turned out to be easy to create a Spring XML configuration and a simple mocked “hello world” proof of concept. The web server was instrumented dynamically!

Updates
One thing I could not figure out was how to destroy existing Mocks that were created in the Tomcat app server. Calling Mockit.tearDownMocks() had no effect. Admittedly, JMockIt is for use in JUnit based tests, so this may be pushing it.

Further reading

  1. Hermetic Servers This shows that the above idea is useful and not unique.

Behavior counters for improved JUnit tests

A weak unit test may give false assurance if it is not responsive to API and test data changes.

Introduction
Recently I came across and even contributed some unit tests that were low quality. Not that they did not test what we wanted. When certain classes were changed the tests did not break, they seemed to still be testing the original API. Since I can’t duplicate the actual tests in this blog post, I’ll try to make up something very simple to illustrate the issue.

Example
In listing one below, a Revenue class is designed to merely illustrate the concept. The class contains a simple list of revenue per salesperson, for example.

Listing 1, example SUT

package com.octodecillion.junit;

import java.util.ArrayList;
import java.util.List;

public class Revenue {
   private List<Integer> users;

   public List<Integer> getUsers() {
     return users;
     // future bad change:
     //return new ArrayList<Integer>();
   }

   public void setUsers(List<Integer> users) {
    this.users = users;
   }
}

In listing 2 below we access this list and assert that each revenue is positive. Yeah, doesn’t make any business sense, bear with me.

Listing 2, A JUnit test of the Revenue class

package com.octodecillion.junit;

import static org.junit.Assert.*;
import java.util.ArrayList;
import org.junit.Before;
import org.junit.Test;

/**  */
public class RevenueTest {
	private Revenue revenue;
	private ArrayList<Integer> list;

	@Before
	public void setUp() throws Exception {
	 list = new ArrayList<Integer>();
	 revenue = new Revenue();
	}

	@Test
	public void should_have_positive_revenue() {
	  list.add(25);
	  list.add(99);
	  revenue.setUsers(list);
	  	
	  for (Integer tax : revenue.getUsers()) {
	    assertTrue(tax > 0);
	  }
	}
}

Problem
Listing 2 above will correctly test the Revenue class. But, what would happen if the Revenue’s class getUsers() method is accidentally changed to return a new List that has no entries? The test as written will still pass! Since the list has no entries the for loop will be skipped and the assertion will not be executed. We can protect against this by having a test of the size of the list, like so:

assertTrue(revenue.getUsers().size() > 0);

But, in a complex test there may be a limit on how extensive is the proactive testing. Or thru lack of skill or error the test could have been written incorrectly. Worse case, unit or integration tests are bogus, and the real problems will be found in production. What could be done?

Writing tests for the tests would be a “turtles all the way” scenario. We need to put inline behavioral coverage in the tests. By contrast, in normal behavioral testing with Mock Objects, the system under test (SUT) will have the behavioral testing, for example, Behavior-based testing with JMockit.

Solution
Listing 3 below, shows an alternative: we use an execution counter. For each block that must be executed within the unit test, not within the SUT, we increment a counter. At the end of the test we assert that the counter has a specific value. This ensures that no block is inadvertently skipped due to SUT changes or test data values.

Listing 3, JUnit test using execution counter

package com.octodecillion.junit;

import static org.junit.Assert.*;
import java.util.ArrayList;
import org.junit.Before;
import org.junit.Test;

public class RevenueTest {
	private Revenue revenue;
	private ArrayList<Integer> list;
	private InvokeCounter invokeCounter= new InvokeCounter();

	@Before
	public void setUp() throws Exception {
	  list = new ArrayList<Integer>();
	  revenue = new Revenue();
	}

	@Test
	public void should_have_positive_revenue() {
	  list.add(25);
	  list.add(99);
	  revenue.setUsers(list);
		
	  for (Integer tax : revenue.getUsers()) {
	    invokeCounter.increment();
	    assertTrue(tax > 0);
	  }	
		
	  invokeCounter.assertCounterMin(1);
	}

}

Implementation
Listing 4 below is possible implementation of an invocation counter class. This was updated (2012-11-19) to allow named counters.

Listing 4, Invocation counter implementation Gist

[expand title=”Click to expand source”]

[/expand]
Alternatives
The main alternative to the above technique is Of course, writing better tests. Another alternative is to use code coverage reporting for the JUnit tests. In a ‘test infected’ organization this would work. The reports are scrutinized and thresholds set and so forth. Alas, this is a passive approach or relies too much on manual intervention.

Summary
Discussed was a possible weakness of unit tests. An approach using invocation counters to assert block coverage of a test was presented.

Links

  1. Code Coverage
  2. JUnit
  3. junit.org seems to be down
  4. JUnit wikipedia entry
  5. JMockit
  6. Unit Testing

Test Coverage Using JMockit

Plenty of coverage reporting tools out there. What this one also includes is Path coverage.

The JMockit Unit Testing library continues to astound. One new thing I discovered is its Coverage reporting.

Code Coverage
Code coverage is simply a measurement of what code has been actually run when tests are executed. There are many such measures, ramifications, and tools. Like testing itself, code coverage measurement is probably not done enough, or misused.

“Code coverage tells you what you definitely haven’t tested, not what you have.” — Mark Simpson in comment

Path Coverage
Plenty of coverage reporting tools out there. What this one also includes is Path coverage. This is different then branch or line coverage. Paths are possible execution paths from entry points to exit points. If you visualize a methods statements in a directed graph, paths are a enumeration of the possible edges traversed when that method is invoked. So, Path coverages is inclusive of Branch coverage. Well, I’m not a testing expert, so this may be way off.

Very surprising results. For example, you run a coverage report with a tool such as Cobertura or Emma and feel very happy that you exercised every line and branch with your tests. Then you run the same tests but use JMockit Coverage and discover your tests didn’t cover all the paths! Not only that your line coverage wasn’t so great either.

Report
JMockit explicitly gives you a report showing:

Path
Measures how many of the possible execution paths through method/constructor bodies were actually executed by tests.
The percentages are calculated as 100*NPE/NP, where NP is the number of possible paths and NPE the number of fully executed paths.

Line
Measures how much of the executable production code was exercised by tests. An executable line of code contains one or more executable segments.
The percentages are calculated as 100*NE/NS, where NS is the number of segments and NE the number of executed segments.

Data
Measures how many of the instance and static non-final fields were fully exercised by the test run. To be fully exercised, a field must have the last value assigned to it read by at least one test. The percentages are calculated as 100*NFE/NF, where NF is the number of non-final fields and NFE the number of fully exercised fields.

— from the JMockit coverage report HTML page

Other information is found by using the full HTML output option.

Example
A sample JMockit coverage report is here. Of course you can drill down into various parts of the html page. Like when you click on an exercised line you will get a list of what invoked that line.

Worth it?
Are the metrics such as Path coverage that this tool generates accurate? Is JMockit coverage a replacement for other tools such as Cobertura? I don’t know. For most projects, the resources would probably make the use of coverages generated by multiple tools prohibitive.

Evaluation
One possible approach to evaluating coverage tools is to just use actual real results of the target application. Use the list of bugs and correlate to a coverage tool report. Where were the bugs? Which tool gave the least measure for this location? True, a ‘bug’ is not always a code problem or limited to one ‘unit’, which is a what a unit test is targeted to.

Further Reading

Use JMockit to Unit test logging output

A very simple method of unit testing the output logging of an app is shown using state-based mocking approach and JMockit toolkit.

A very simple method of unit testing output logging is presented using a state-based mocking approach and JMockit toolkit.

Context
Java language. Logging frameworks such as Log4j, commons-logging, java.util.logging, and SLF4J. Application logs.

Introduction
Unit test logging output? Isn’t that going overboard? Most likely. However, there can be some reasons why in certain parts of a code base you’d better:

  1. Audit requirements
  2. Use of Structured Logging.
  3. Logging adheres to standards such as Common Event Expression (CEE) language. (Note, CEE has been cancelled.)
  4. Ability to maintain a system
  5. Log maintenance tools get correct data
  6. Reduce technical debt
  7. SIEM

Solution
In listing 1, a simple class uses the java.util.Logger to log. We want to make sure this class will always log this in the future, i.e., a regression test. In order to qualify as a unit test, the system under test (SUT) should be isolated. Thus, parsing an actual logging output file would not be optimal.

Listing 1

" java.util.logging.Level;
" java.util.logging.Logger;

/**  Example class that logs. */
public class Service {
	Logger logger = Logger.getLogger(this.getClass().getName());
	
	public void serve(){		
		logger.log(Level.INFO,"Hello world!");
	}
}

In listing 2 we use JMockit to mock the java.util.Logger. This should also work for other logging frameworks. JMockit has two approaches for applying mock techniques: Behavior-based and state-based.

We apply state-based below (just cause that is the one I’m starting to get the hang of). The Arrange, Act, Assert (AAA) pattern is still used, but the Assert step is in the mock object. We apply a simple ‘equals’ test. Of course, based on what we expect in the log message, a regex may be more useful.

Listing 2

import static org.hamcrest.CoreMatchers.*;
import static org.junit.Assert.*;

import java.util.logging.Level;
import java.util.logging.Logger;

import mockit.Mock;
import mockit.MockUp;
import mockit.Mockit;

import org.junit.After;
import org.junit.Before;
import org.junit.Test;

/**
 * JUnit test for Service class.
 * @author jbetancourt
 */
public class ServiceTest {
	private Service service;

	@Before
	public void setUp() throws Exception {
		service = new Service();
	}
	
	@After
	public void tearDown() throws Exception {
		Mockit.tearDownMocks();
	}

	@Test
	public final void we_are_logging_correctly() {	
           // Arrange
           mockLogger();

           // Act
           service.serve();	
	}

	private void mockLogger() {

		new MockUp<Logger>() {
			@SuppressWarnings("unused")
			@Mock
			public void log(Level level, String msg) {
				assertThat("Hello world!", is(equalTo(msg)));
				assertThat(level, is(equalTo(Level.INFO)));
			}
		};
	}
}

Extensions
The above technique just tests that the log parameters are correct. This doesn’t check that the log output to file itself is correct. That is a different concern. Since the actual output logging is controlled by various configuration options, a unit test may not make sense. Would that be a functional test?

Of course, just checking that the message sent to the logger may not be enough. In this case a possible approach is to hook into the logging library to capture the final log output. In the java.util.logging API, one can add a new stream Handler to the Logger instance being used.

In listing 3 below a simple attempt is made to use a stream handler to capture the actual logger output used by java.util.logging.

Listing 3


public class ServiceTest{
  Logger logger = Logger.getLogger(ServiceTest.class.getName());
  private OutputStream logOut;
  private StreamHandler testLogHandler;

  @Before
  public void setUp() throws Exception {
     setUpLogHandler(logger);
  }

  /** */
  @Test
  public final void exception_log_has_all_info(){
    logger.log(Level.WARNING, "Hello world!");
    testLogHandler.flush();
    String captured = logOut.toString();
    Assert.assertTrue(captured.contains("Hello world!");
 }

 /** 
   Add stream handler to logger.  
   Will take more effort then this, e.g., may not have parent handler.
   */
  protected void setUpLogHandler(Logger logger) {
    logOut = new ByteArrayOutputStream();
    Handler[] handlers = logger.getParent().getHandlers();
    testLogHandler = new StreamHandler(logOut, handlers[0].getFormatter());
    logger.addHandler(testLogHandler);
  }
}

Software
* JUnit: 4.*
* JMockit: 0.999.11
* JDK: 1.6*
* Eclipse: IDE 1.7
* Git: 1.76.msysgit.0

Summary
Shown was a little technique that may come in handy one day. Though presented in the context of logging, it is really a simple application of state-based mock use.

Further reading


Carlos Santana/Mahavishnu John McLaughlin – The Life Divine

Unit Testing what will never happen?

Some developers refuse or are slow to test units for certain values or situations because, they claim, those values or situations will not occur. Is this wise?

Some developers refuse or are slow to test units for certain values or situations because, they claim, those values or situations will not occur.

Some reasons given are that the presentation tier or UI will prevent certain values; that other modules in the call chain already have error handling and validation. So why test something that won’t happen?

Let’s take an example. In the method/function below, the type will never be blank or null. Looking at the actual source code for the application will show this. So should a unit test be written that will invoke this method with a null, “”, or ” “?

public boolean service(final String type){
    // do stuff
    return result;
}

Yes!
1. Things change.
2. Bad stuff happens.
3. Development process will use invalid values.
4. More complete testing. Regression testing, path coverage, etc.
5. The method could now be used in a different call chain.
6. Its a public method, anything can invoke it, even a business partner via some remoting technology.
7. When invoked as part of other unit tests, it could have invalid values.

#3 is, I think, the most important. Did you ever do development and something not work and it turn out to be existing code that did not handle arguments correctly? That is wasted time and an aggravation. Sure, in the current production code, a blank string won’t be used, but during development, especially TDD, you scaffold code. Sometimes you don’t have values yet, so you just use a blank string.

Just the other day I tested a deployed production method that correctly tested the argument for an empty string, “”, and did the right thing. However, the code did not check for a blank string, ” “, and throws an exception. Unit testing would have shown this.

And, yes, this will never happen in our application. 🙂

Ok, you still don’t want to test something for null or invalid values? At least put this in writing in the Javadoc: “I (insert your name and address) am a great developer and stake my professional career that this method will never be invoked with invalid values.” The address is needed so us mediocre developers can hunt you down when the server crashes.

Further Reading

  1. Testing getter/setter using JUnit
  2. FindBugs and JSR-305

Off topic, some music …
If ” – Oregon (2009), from “Prime” CD.

Testing getter/setter using JUnit

Unit testing can be difficult. I give an example on how a property access test can fail.

So, I was thinking of testing my getter and setters in a JUnit test. Yea, I know it is not recommended, but I thought that I could write a simple reflective iterator that could do it without much fuss.

What I show below is that maybe getter/setters should be tested, especially in Java which really does not have “real” properties. Further, by investigating a potential solution, I show that naive testing will not work, and the reason is that testing IS hard to do.

Opponents of unit testing getter/setters are making a category error. They are equating the getter/setter idiom with the non-existent support for language level Properties in some languages, like Java.

A method named getFoo(), provides no guarantees on how it is getting the result, could be a field, could be a webservice, etc., and thus can fail like anything else.

 

Anyway, just to save time, and not have to code this, I did a quick search and found that this has been done many times before. So I looked at one of the solutions, Scott’s. Very nicely done! With one line you can test a class’s property getters and setters. Then I noticed a problem.

In his approach he invokes the setter and then the getter, finally comparing that the values are the same. For example, if the property is a boolean, the test class will set it true, then read it and the value better be true. However, what if the value prior to your set was true, the object is initialized with true for the field value?

For example, I took a class and set field x to true via the default constructor, then I modified the setX method to not set the field value.

public class Fubar {
  private boolean launch = true;
  
  public void getLaunch(){ 
      return this.launch;
  }

  public void setLaunch(boolean launch){  
      /* this.launch = launch; */ // broken!
  }
}

Code at: git clone git://gist.github.com/1408493.git gist-1408493

In the unit test the getLaunch method still returned true. The getter/setter test did not fail; the test was bogus. Not only that, the fact that the setLaunch method did not work illustrates that sometimes testing setters is warranted.

Thus, the version of the program that controls the sports strategy will ship and the boolean that aborts a play cannot be set to false! (Edited, removed joke about war stuff; you can’t be too careful with all the craziness in the world).

Updates
March 9 2014: I forked the original code with minor changes to a GitHub repository at: https://github.com/josefbetancourt/property-asserter.git
March 4, 2014: Just today a co-worker had an issue with a class. The getter was trying to return a value that was defined as transient. Thus, on deserialization the field was returning a null. The unit test could have provided a test for this. If a class implements Serializable, then it should include tests using serialization, and so forth.
March 2, 2014: I had a “heated” discussion with coworkers on this. I attempted to defend this post’s ideas. They asserted that this is a bogus example, and the issue is only applicable to boolean or primitives types. I contend that in fact it is very applicable where:

  • There are defaults of any kind.
  • There are methods that change object state. (Note: XUnit approach cannot test side effects unless they are part of the public interface)
  • The class will be maintained in the future.

That pretty much includes all non-trivial classes.

Feb 11, 2012: I’m using Scott’s solution in our test suite. I did modify the code a bit. Turns out that a JavaBean property is not so clear cut. If you have a getter and setter method pair but no actual target field, are those Javabean methods? Looks like the Java bean introspector will report these as bean accessors. Hmm.
June 7, 2013: There are two more issues with the above algorithm for testing getter/setters. First since the approach is for JavaBeans, there are issues with non-Javabeans.
    1. If the object doesn’t have a default constructor, one can’t automatically instantiate an object.
These are not insurmountable. The PropertyAsserter class has methods that take a class argument to use for testing a getter/setter.

 

In a recent presentation I gave on unit testing, I said that testing could be difficult. This is a great example of this. In this case, one should have followed analogous patterns in other fields. Thus, from semiconductor testing, for example, you would test gates by not just writing a 1 bit, you have to write 1 and 0 to make sure the gate is working, not stuck on high or low.

The bullet points

  1. Java Getter/Setters are a programmer’s agreement, easily broken.
  2. A getter/setter could be collaborating with external resources or services.
  3. A getter/setter could be part of a JavaBeans property event broadcasting requirement.
  4. If it can be automated, what’s the fuss?
  5. Never underestimated what could go wrong.
  6. Improves test coverage levels and get a pat on head from management.
  7. Would you not test a five cent metal washer on an interplanetary star ship?

Resources


Bill Evans Trio – Nardis – 19 Mar 65 (7 of 11)

Article on Bill Evans: http://www.chuckisraels.com/articleevans.htm

JMockit

JMockit is very useful. There are plenty of good reasons for using mocks.

Yesterday at work I gave a presentation on Unit Testing. It went well. 160 slides! And, no one passed out and hit the floor.

One thing I mentioned was mocking frameworks and how JMockit is very useful. Perhaps JMockIt represents the state of the art in Java based Mocking tools.

There are plenty of good reasons for using mocks:

“JMockit allows developers to write unit/integration tests without the testability issues typically found with other mocking APIs. Tests can easily be written that will mock final classes, static methods, constructors, and so on. There are no limitations.” — JMockit

JMockit is ” a collection of tools and APIs for use in developer testing, that is, tests written by developers using a testing framework such as JUnit or TestNG.”

I’ve used it for some tests. Since it uses Java instrumentation it can mock almost anything, especially those legacy untestable great OO classes. Best of all it has a very good tutorial.

The only ‘negative’, so far, is that JMockit does not, afaik, have many developers working on the project. That could also be a plus, of course.

Another mock tool is PowerMock.

Seems to me there are too many mock frameworks and they do pretty much the same things. Time for consolidation so that an API and a body of practice can shake out?

Further reading

  1. Mock object
  2. MockingToolkitComparisonMatrix
  3. Beyond EasyMock and JMock, try JMockIt !
  4. The Difference Between Mocks and Stubs
  5. The JMockit Testing Toolkit
  6. The Concept of Mocking
  7. PowerMock
  8. Unit Testing Using Mocks – Testing Techniques 5
  9. Making a mockery of CQ5 with JMockit


Off topic, some music …

Stefano Cantini – Blowin in the wind

jsUnit based tests won't run in other folders?

A fix for jsUnit tests not running in external folders.

This is why software is such a sometimes frustrating undertaking. I once had a jsUnit based test working. It ran fine in FireFox. Today it did not. Funny thing is that the existing tests that come with jsUnit still work. Other browsers fail too. (maybe).

It had to be a typo, bad url, or something. The jsUnit docs say that the problem is that I am not correctly giving the path to the jsUnitCore.js file. I tried everything. But, now I try to get smart. First copy the tests that are working to another folder. I copy failingTest.html to /temp/jsunit, and I also copy jsUnitCore.js there too.
Still doesn’t work.

I edit jsUnitCore.js, and add an alert to show that it is being executed or loaded. To make sure I really did it, I diff:

C:tempjsunit>diff failingTest.html javajsunittestsfailingTest.html

 
4,5c4,6
&lt;     &lt;meta http-equiv=&quot;Content-Type&quot; content=&quot;text/html; charset=UTF-8&quot;&gt;
&lt;     &lt;script type=&quot;text/javascript&quot; src=&quot;./jsUnitCore.js&quot;&gt;&lt;/script&gt;
---
&gt;     &lt;meta http-equiv=&quot;Content-Type&quot; content=&quot;text/html; charset=UTF-8&quot;&gt;
&gt;     &lt;link rel=&quot;stylesheet&quot; type=&quot;text/css&quot; href=&quot;../css/jsUnitStyle.css&quot;&gt;
&gt;     &lt;script type=&quot;text/javascript&quot; src=&quot;../app/jsUnitCore.js&quot;&gt;&lt;/script&gt;

C:tempjsunit>diff jsUnitCore.js javajsunitappjsUnitCore.js

13,14d12
&lt; alert(&quot;Maybe a career in sanitation?&quot;);
&lt;

Try the test and I see the message. So, I know that the library is being found and is running. That is weird. Must be the browser. I empty the cache. still nothing. Arrrrr.

Google? I find a reference to jsUnit forum about someone having a similar problem. No advice, but one person gives a url for the solution. Is it a good url, spam, or worse? Nope its legit, and real advice. It is the browser. Thank you Andrew! I should have checked the forum first.

Browsers, bah! I hope they don’t start showing up in embedded systems. That’s probably how WWIII will get started, some Javascript mistyping.

Updates
17JAN11: Spoke too soon. FireFox ver 4.0b9 is showing the problem again. I’ll try in my Ubuntu instance too.

Links
JSUnit 2.2 on Firefox 3.07 – FTW

jsUnit

Java Dev Using Embedded Groovy Console in Eclipse

Embed a script engine and have new ways of analyzing and developing code.

In development, simple but powerful tools to get at the essence of a code source in order to understand, test, and extend it is essential. This is even more important in Test Driven Development (TDD). Eclipse’s Java Debugger is for most situations, powerful enough. Eclipse has an Expressions View available in the debug perspective to execute snippets of code. The Java Development Toolkit (JDT) also has a Scrapbook facility that allows the creation, storage, and running of experimental code. And, of course, all the other features of the debugger are excellent.

However, when you need it, it’s possible to embed a script engine and have new ways of analyzing and developing code. In listing 1, an app shows the use of the ConsoleWaiter class. When the code executes the waiter.run() at line 43, it opens the Groovy Console which allows the use Groovy shell scripting in a GUI frame, see figure 1. When the console is closed the app continues executing.

Listing 1
/*
* File: ExampleApp2.java
* @author jbetancourt
* Date: 20101213T1718-5
*
*/
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

/**
*
* @author jbetancourt
*
*/
public class ExampleApp2 {
   static public String greeting = "Hello world!";
   private static final List<String> nodes;

   static {
        nodes = new
            ArrayList<String>(
                  Arrays.asList("one","two"));
   }

   private String title = "Example 2";

   public String getTitle(){
      return title;
   }

   /**
   * @param args command line args
   */
   public static void main(String[] args) {
      ExampleApp2 app = new ExampleApp2();

      ConsoleWaiter waiter = new ConsoleWaiter(app);

      waiter.setVar("greet", greeting);
      waiter.setVar("nodes", nodes);
      waiter.setVar("title", app.getTitle());
      waiter.run();
      System.out.println("Done!");
   }

}
screen capture of Groovy console
Console screen capture, click to view
Another screen capture, click to view

This is awesome. In one project I had to examine the contents of a Properties object. Did it have an “email” value? I was stumped when using the Eclipse debugger, it did not show all entries in the Map, at the end was “…”. Sure, I could use the Expressions window, but with the Console I could not only do a get(key), but iterate using Groovy style closures and much more.

The magic that enables this is the ConsoleWaiter.groovy class shown below in listing 2 that was written by John Green. Since a Groovy script is a Java class underneath, in Eclipse you can call Groovy from Java easily (some compiler magic).

Listing 2
/**
 * File:  ConsoleWaiter.groovy
 */

import groovy.lang.Binding;
import groovy.ui.Console;

/**
 * Provides a wrapper for the console.
 *
 * Based on source by John Green
 * Adapted from:  http://www.oehive.org/files/ConsoleWaiter.groovy
 * Released under the Eclipse Public License
 * http://www.eclipse.org/legal/epl-v10.html
 *
 * I added methods to allow use from Java.
 *
 * The run() method launches the console and causes this thread
 * to sleep until the console's window is closed.
 * Allows easy interaction with the objects alive at a given
 * point in an application's execution, like in a debugger
 * session.
 *
 * Example 1:
<pre> * new ConsoleWaiter().run()
 *</pre>
 *

 * Example 2:
<pre> * def waiter = new ConsoleWaiter()
 * waiter.console.setVariable("node", node)
 * waiter.run()
 *</pre>
 */
class ConsoleWaiter {
 Console console
 Object source
 boolean done = false;

 /** */
 public ConsoleWaiter(Console inConsole){
    this.console = inConsole
 }

 /** */
 public ConsoleWaiter(Object source){
    console =
    new Console(getClass().classLoader,
    new Binding())
    this.source = source
    console.setVariable("source", source)
 }

 /** */
 public void setVar(String key, Object value){
    console.setVariable(key, value)
 }

 /** 	 */
 public void setVar(String key, List values){
    console.setVariable(key, values)
 }

 /** 	 */
 public void setVar(String key, Object[] values){
    console.setVariable(key, values)
 }

 /** 	 */
 public void run() {
    console.run()
    // I'm a little surprised that this exit() can be private.
    console.frame.windowClosing = this.&amp;exit
    console.frame.windowClosed = this.&amp;exit
    while (!done) {
       sleep 1000
    }
 }

 /** 	 */
 public boolean isDone(){
    return done;
 }

 /** 	 */
 public void exit(EventObject evt = null) {
    done = true
 }

 /** 	 */
 public Console getConsole(){
    return console;
 }
}

Eclipse Integration

The disadvantage of this approach is that you have to put extraneous code inside the tests or target class. Not only is this tedious and slow, what if code is accidentally deployed like this? A better approach is to just set a breakpoint in the code, and then have the ability to open a script console at that breakpoint, in the Java Stack Frame, that has access to the execution context and classpath. Is there an Eclipse add-in that does this? If not, there should be.

Conclusion

Shown was a simple example of embedding a Groovy console in Java code to allow scripting. Of course, this is not a new idea. It was even mentioned in an older JVM scripting language, Beanshell. Note that it is possible to instead of using a GUI console, to use the Groovy shell, InteractiveShell class. In the reading list below this approach is taken to allow remote scripting of a server hosted application.

Updates

  • Oct 10, 2011:
    Interesting tool that could be relevant: YouDebug.
  • March 20, 2012: Not exactly same scenario, but the concept of an embedded script console is found in many products. Jenkins CI Server has one and it uses Groovy. Jenkins Script Console

Further reading