JMockit is very useful. There are plenty of good reasons for using mocks.
Yesterday at work I gave a presentation on Unit Testing. It went well. 160 slides! And, no one passed out and hit the floor.
One thing I mentioned was mocking frameworks and how JMockit is very useful. Perhaps JMockIt represents the state of the art in Java based Mocking tools.
“JMockit allows developers to write unit/integration tests without the testability issues typically found with other mocking APIs. Tests can easily be written that will mock final classes, static methods, constructors, and so on. There are no limitations.” — JMockit
JMockit is ” a collection of tools and APIs for use in developer testing, that is, tests written by developers using a testing framework such as JUnit or TestNG.”
I’ve used it for some tests. Since it uses Java instrumentation it can mock almost anything, especially those legacy untestable great OO classes. Best of all it has a very good tutorial.
The only ‘negative’, so far, is that JMockit does not, afaik, have many developers working on the project. That could also be a plus, of course.
Seems to me there are too many mock frameworks and they do pretty much the same things. Time for consolidation so that an API and a body of practice can shake out?
A simple example is used to show how to use Aspect Oriented Programming to provide fault injection.
A simple example is used to show how to use Aspect Oriented Programming to provide fault injection.
Intro
A few months ago I was looking at Java code and thinking there must be a better way to see if this will really work, if there are any issues in how the program would react to exceptional conditions. I could write some unit tests or force some problems to see if the exception handling is correct and the log file output is really useful. But is there another way?
I got a solution after a while of thinking: Insert faults. Now how to do that in an easy and maintainable way? Not a new idea of course. After doing a web search I found some interesting references.
Funny, today at work I was extending an application and one of the support objects was failing. Turned out to be an index out of bounds problem. The method had adequate exception handling but, of course, an index out of bounds scenario was not tested. In this case the problem would not occur in production (hmmm), but it definitely occurred during development, and was not handled correctly.
Exception Handling
There are many references of correct exception handling strategies and best practices. However, when creating applications, the developer must still make critical creative decisions about how to handle an exception. That decision may not be a wise one. And even if it is, may later prove to have been very unwise since the system requirements and implementation may have changed due to maintenance or requirement evolution.
Testing Approaches
Using various testing methodologies a subsystem can be exhaustively tested for exception handling. Yet, most testing methods require that the subsystems be isolated in some manner. This, though very valuable, can give fatal false confidence that the system will behave in a predictable manner in responding to exceptional conditions.
The prime example of this is within the Unit Testing methodologies. Since Unit Tests are tests of isolated components, they do not test actual exception handling in the live system. A perfect component can still contribute to system failure when it is used incorrectly or uses other components incorrectly. For example, a unit test can show that class X’s methods correctly handle and when necessary throw the perfect exception. That means nothing if components that interact with X don’t correctly use those exceptions: they swallow, mask, or mistakenly catch them when there is nothing they can do.
Thus, one needs to use Integration Testing to test assemblages of components. But, we are still back with the same shortcoming as with Unit Testing. Thus we would next need various forms of system testing. And, that system testing must be a White Box test. A Black Box test wherein an interface is exercised with various inputs (such as in web test systems or by QA personnel), will not necessarily invoke the full internal API/SPI between components that could be part of a complicated call chain. Why? Because, the interface, such as a browser client, will have (we hope) the validation, security, and logging layers implemented so that by the time a system test workflow enters the deep system components within the client or middleware, there is no way to influence the target component state into programmatic intentional failure mode.
If there is no way to exercise a run time trajectory, then why bother? The reason is that exceptional conditions are exceptional. Resources can be exhausted, systems not available, and so forth. The rule of Fail Fast may not be enough if collaborating components are not exceptionally responsive.
Fault Injection
To see if a system is responding to exceptional conditions, one must wait for those conditions or create them. Analogously to Fuzz Testing we can dynamically and randomly insert faults. In Fuzz Testing, inputs are manipulated to a system under test. In Fault Injection we have to manipulate inputs inside the real system, within the components themselves. And as in Fuzz Testing, we can employ various strategies to do so, such as randomized faults, etc. Of course, this system is not the deployed production system, it is the real system in a test environment, an internal full deployment.
Some approaches are: source modification, source macros, annotations, dependency injection (IOC), and Aspect Oriented Programming (AOP). Of course, there are many other techniques as found in open/closed projects, commercial products, and the research community.
Fault Injection using AOP
Since using the real system is required, a viable approach is to use the Instrumentation already available in the Java system. We could dynamically insert code that forces exceptions. AOP as implemented by the AspectJ language can already do this.
The major advantage of using AOP is that the source is not changed in any way. AspectJ is the state of the art in the Java ecosystem for using AOP.
Other approaches
Monitoring
BTrace (a Java oriented approach comparable to DTrace) is very interesting. However, since, like DTrace, it is meant to be safe, defect injection may not be doable. But, see this post on an unsafe mode for BTrace.
Source modification
One way of injecting failure is just to edit the source code and add strategic code that will cause the instigating failure. An advanced form of this is Mutation Testing. Code changes could be: arguments with wrong values, nulls, deliberately thrown exceptions, etc. For example, one can simply create a method that throws a runtime exception:
*** DON'T DO THIS ***
public static void REMOVE_XXX_FROM_XXX_SOURCE(){
if(true){
throw new NullPointerException(
"nt****** DELIBERATE RUNTIME EXCEPTION*****n");
}
}
Then insert invocations to this method at strategic points in the code, deploy to a test environment, and see if the expected results are obtained, such as logging output that can identify the cause, or that no side effects are recorded, such as incorrect data storage.
This is a low tech and error prone approach. Most importantly, this is dangerous. In the mad development rush to meet deadlines and also eat lunch, this code could make it into production! Even with that horrific method name, it will wind up in production! One would have to create deployment filters that stop the process if any “fault” inducing code is included. Another reason why this is not a good approach is that it doesn’t scale very well. Forcing exceptions is just one type of verification. Another is setting values outside of expected ranges or states. Thus, one would need many different kinds of source code insertions.
Of course, Unit and Behavior based testing tools sets can supply these required verifications if included into the system as Built-In Self-Tests (BIST).
Built-In Self-Test
In the hardware realm, BIST as found in, for example, IEEE 1149.1 JTAG, has been very successful. On the software side, there is ongoing research on how to implement BIST-like capability. This would make Brad Cox’s concept of the “Software IC” even more powerful.
Macros
A somewhat viable approach to source code fault insertion is instead of inserting the faulting code, insert “include” macros in the original source code that indicates what fault insertion should be done at a location. A fault injection preprocessor can scan and insert the fault framework’s invocations to accomplish the requirement. The source code build process can then simply enable or disable the use of the preprocessor. This would however still require source code modification and an extra maintenance nightmare. When the code changes the macros may also require change.
Annotations
Instead of macros, we could also use Annotations. Annotations could explicitly state the runtime exceptional behavior “service level agreements” at the component level. These SLA could then be systematically manipulated to test if they really hold at the system level.
Dependency Injection
One can also inject exceptional behavior by dependency injection, using programmatic, declarative, or annotations, faulted components could be inserted (or created via Aspect Oriented Programming) into the target system.
AOP Demonstration
In listing one below, an ATM class uses a service to do some banking.
/**
* Driver class for example.
* For a real system see: https://bitbucket.org/aragost/javahg/
* @author jbetancourt
*/
public class ATM {
private BankService service = new BankService(1000);
/**
* Perform a banking action.
*/
public void transaction(){
{
service.deposit(100L);
service.withdraw(200L);
}
service.statement();
}
/** Application entry point */
public static void main(String[] args) {
new ATM().transaction();
}
}
Listing two is the BankService being used. Of course, a real service would be more complex.
/**
* Example service.
*/
public class BankService {
// Use Long to allow example's null for failure
// injection.
private BigDecimal balance;
public BankService(long savings){
this.balance = new BigDecimal(savings);
}
/** */
public void deposit(Long amount){
balance = balance.add(new BigDecimal(amount));
System.out.println("Deposit: " + amount);
}
/** */
public void withdraw(Long amount){
balance = balance.subtract(new BigDecimal(amount));
System.out.println("Withdraw: " + amount);
}
/** */
public void statement() {
System.out.println("Balance: " + balance);
}
} // end BankService
Report: pointcut usage
withdrawCut: true -> true
depositCut: false -> false
Deposit: 100
Exception in thread "main" java.lang.NullPointerException
at BankService.withdraw_aroundBody2(BankService.java:15
at BankService.withdraw_aroundBody3$advice(BankService.java:81)
at BankService.withdraw(BankService.java:1)
at Main.main(Main.java:16)
Exception at the deposit method.
Report:
pointcut usage
withdrawCut: true -> false
depositCut: false -> true
Exception in thread "main" java.lang.NullPointerException
at BankService.deposit_aroundBody0(BankService.java:9)
at BankService.deposit_aroundBody1$advice(BankService.java:81)
at BankService.deposit(BankService.java:1)
at Main.main(Main.java:15)
All pointcuts enabled.
Report:
Report:
pointcut usage
withdrawCut: true -> false
depositCut: false -> true
Exception in thread "main" java.lang.NullPointerException
at BankService.deposit_aroundBody0(BankService.java:9)
at BankService.deposit_aroundBody1$advice(BankService.java:81)
at BankService.deposit(BankService.java:1)
at Main.main(Main.java:15)
Implementation
Now to test the exceptional behavior we want to fault the deposit and withdraw methods.
First we have a way of specifying what we want to fault by using a JSON configuration file:
The “cuts” settings indicate which AspectJ “pointcut” to turn on. The “random” setting indicates if we want the exceptions to be randomly inserted into the code base at the enabled pointcuts.
The Abstract failure injection aspect. Subclasses (aspects) will supply the actual pointcuts to specify where to ‘do’ the injection.
/**
* FailureInjectionAspect.aj
* @author jbetancourt
*
*/
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
/**
* Abstract Aspect that determines if failure should occur.
* @author jbetancourt
*
*/
public abstract aspect FailureInjectionAspect {
private Map<String, Boolean> pointcutFlags = new HashMap<String, Boolean>();
private volatile boolean initialized =false;
/**
* Initialize Failure injection aspect.
* @throws Exception
*
*/
protected void init() throws Exception {
Config config = new Config();
config.configure();
pointcutFlags = config.getPointcutFlags();
initialized = true;
}
/**
* Get boolean value for pointcut name.
*
* @param name pointcut name.
* @return true if pointcut enabled.
* @throws IOException
*/
protected boolean isPointcutEnabled(String name) {
if(!initialized){
try {
init();
} catch (Exception e) {
throw new IllegalStateException(
"Could not initialize object",e);
}
}
boolean f = false;
Object val = pointcutFlags.get(name);
if (null != val) {
f = ((Boolean) val).booleanValue();
}
return f;
}
} // end FailureInjectionAspect.aj
Here is a aspect that uses a nulling type of injection.
/**
*
* Example of an aspect for nulling an argument to a method.
* @author jbetancourt
*
*/
public aspect AmountFailureAspect extends FailureInjectionAspect {
/** fault the deposit */
private pointcut depositCut(Long amount) :
execution(public void BankService.deposit(Long))
&& if(AmountFailureAspect.aspectOf().isPointcutEnabled("depositCut"))
&& args(amount)
&& !within(FailureInjectionAspect)
;
/** fault the withdrawal */
private pointcut withdrawCut(Long amount) :
execution(public void BankService.withdraw(Long))
&& if(AmountFailureAspect.aspectOf().isPointcutEnabled("withdrawCut"))
&& args(amount)
&& !within(FailureInjectionAspect)
;
/** Null the amount arg */
void around(Long amount) : depositCut(amount)|| withdrawCut(amount) {
amount = null;
proceed(amount);
}
}
Here is how to read the configuation JSON file. I use the JSON-Simple library.
Using AspectJ is much easier in a supporting IDE like Eclipse. Below is the “mess”, unless you love the CLI, of compiling and running in a command shell. Could be made clean by creating aliases, scripts, etc.
cd src
src>javac -d ..bin -cp ..jarsjson-simple-1.1.jar *.java
src>java -cp ..bin Main
Jun 9, 2011 3:18:24 PM Main main
INFO: Starting Main.....
Deposit: 100
Withdraw: 200
Balance: 900
Now we change the flag from false to true in the runtimeInjection.json file:
Now we run the same Main program. Since the pointcut flag is true, the advise is invoked and the list argument to the service(List) is set to null.
c:UsersjbetancourtDocumentsprojectsdevAspectsForNullTesting>java -cp "c:javaaspectj1.6libaspectjrt.jar;c:javaaspectj1.6libaspectjtools
.jar;c:javaaspectj1.6libaspectjweaver.jar;jarsjson-simple-1.1.jar;bin;." Main
Jun 9, 2011 3:25:04 PM Main main
INFO: Starting Main.....
Pointcut usage
Name Prior Current
withdrawCut: true -> false
depositCut: false -> false
c:UsersjbetancourtDocumentsprojectsdevAspectsForNullTesting>java -cp "c:javaaspectj1.6libaspectjrt.jar;c:javaaspectj1.6libaspectjtools
.jar;c:javaaspectj1.6libaspectjweaver.jar;jarsjson-simple-1.1.jar;bin;." Main
Jun 9, 2011 3:25:10 PM Main main
INFO: Starting Main.....
Pointcut usage
Name Prior Current
withdrawCut: true -> true
depositCut: false -> true
Jun 9, 2011 3:25:10 PM Config exec
INFO: Initialized: true
Exception in thread "main" java.lang.NullPointerException
at BankService.deposit_aroundBody0(BankService.java:19)
at BankService.deposit_aroundBody1$advice(BankService.java:26)
at BankService.deposit(BankService.java:1)
at Main.main(Main.java:16)
c:UsersjbetancourtDocumentsprojectsdevAspectsForNullTesting>java -cp "c:javaaspectj1.6libaspectjrt.jar;c:javaaspectj1.6libaspectjtools
.jar;c:javaaspectj1.6libaspectjweaver.jar;jarsjson-simple-1.1.jar;bin;." Main
Jun 9, 2011 3:25:13 PM Main main
INFO: Starting Main.....
Pointcut usage
Name Prior Current
withdrawCut: true -> true
depositCut: false -> false
Jun 9, 2011 3:25:13 PM Config exec
INFO: Initialized: true
Deposit: 100
Exception in thread "main" java.lang.NullPointerException
at BankService.withdraw_aroundBody2(BankService.java:25)
at BankService.withdraw_aroundBody3$advice(BankService.java:26)
at BankService.withdraw(BankService.java:1)
at Main.main(Main.java:17)
Updates
Feb 1, 2012: Just learned about Byteman.
April 25, 2013: A new article on using Byteman: http://aredko.blogspot.com/2013/04/fault-injection-with-byteman-and-junit.html
Example source code illustrating use of the HTTP server included in Java JDK 1.6. via Groovy to present a browser-based UI to a local application.
Code illustrating use of the HTTP server included in Java JDK 1.6. via Groovy to present a browser-based UI to an app.
Result
The code allows this usage:
main.SimpleServer.serve(0){ s, t, p ->
// handle the request, response here...
// now shut down,
s.stopServer()
}
Listing 1, How it’s used.
This will create a web server using the host “localhost” at an unused port p, and then automatically open the default browser at “http://localhost:p/”. The closure will be the application. Neat. Of course, you would only use this behind a firewall, etc.
The above code, as a I later discovered, is very similar to other frameworks. Here is a new one I just learned about:
See also: vert.io. “Effortless asynchronous application development for the modern web and enterprise”
See also “Java development 2.0: Ultra-lightweight Java web services with Gretty” link, for an approach using Gretty.
Hmmm, just noticed that this is the ‘look’ of a simple node.js example. No way to dupe node.js of course, it is a low level thing, but can streaming event based programming be done with Groovy? Perhaps with GPars. See this discussion node.groovy?.
A sample session running the example code and using telnet is:
telnet localhost 21224
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
GET /?reply=24 HTTP/1.1
HTTP/1.1 200 OK
Content-length: 15</pre>
<h1>Wrong!</h1>
<pre>Connection closed by foreign host.
The Eclipse console contains:
+++++++ Simple Server ++++++++++++++++++
uri: [/?reply=24]
protocol: [HTTP/1.1]
query: [reply=24]
path: [/]
params: [[reply:[24]]]
-------------------------
Stopping server ... stopped!
The browser will show:
The app as seen on browser
Scenario
You have to supply a GUI for running a local Java application. One example could be the setup of a product build. Builds, though usually executed with Ant, Maven, or Gradle, may still require the user to select some target parameters or build type. Why not just use the browser? It can present a more modern interface and with good design, allow more fault tolerant use then that with command line or multiple prompt boxes.
This approach also allows future remote build server use, since the designed browser UI can be reused. The browser is ubiquitous and creating web pages is relatively easy (though can be a hair pulling nightmare sometimes). And, with the use of AJAX and high-level frameworks, like Dojo, JQuery, or React, browser clients can more easily duplicate the usability of dedicated thick client apps. HTML5 (see HTML Cheatsheet) is also helping to obliterate any remaining reasons for using a thick-client app.
An embedded server as used here, is great when the task is ad hoc, short lived, or single user. In the provided example, once the input is received the server shuts down. For more complex or ubiquitous use a standard server or more powerful embedded server should be used.
Embedded Server
For a local application that accesses system resources and uses the browser as the UI, using an embedded server is the simplest approach. Pure Javascript, Applets, and other means are very complex and may run against configuration issues with Browser security settings and so forth. In the Java world, that would mean using the more popular Tomcat or Jetty servers.
However, Java 1.6 now includes a light-weight HTTP Server. So, the configuration and requirements are more tractable. Nothing more to download and programmatic configuration concerns are minor. Note that the Java HTTP server does not offer many features, one augments this with custom code. For example, parsing of the request query is not present.
That the package of this server is com.sun… is problematic. Will it stay around, become part of the javax packages, etc? Should the JDK even have this built in? According to this post by M. MacMahone
… is that the API and implementation are a fully supported, publicly accessible component of Sun’s implementation of Java SE 6. It does mean however, that the packages are not formally part of the Java SE platform, and are therefore not guaranteed to be available on all other (non Sun) implementations of Java SE 6.
Incidentally, the Groovy language has an import system, Grape, that can also make use of Tomcat or Jetty as transparently as using the JDK embedded server. See the Further Reading below for an example using Groovlets.
This code illustrates
(more for my future reference)
The list below was some of the things the code used and the final code listed here may no longer have them.
com.sun.net.httpserver API use.
Using JQuery in external js files.
How to stop the server.
With AJAX post
Timeout
HTTP context
Using ScheduledExecutorService to limit runtime.
A console ASCII spinner.
Groovy GString use.
Launching the default browser.
Selecting an unused port.
Simplistic state machine configuration.
Detecting Java version.
AJAX using JQuery.
Groovy object construction from script.
Use of Closure.
Basic authentication
Quasi Anonymous class use in Groovy
access to resources
Of course, not great example of the above, but … Warning, code is not production ready, etc. There is no real exception handling!
How it works.
TODO: Give some idea what all that code does.
The index.html creates a simple form with three buttons (submit, ping, and end), an input field, and a ‘console’ output area.
– submit: send the answer to the server which then gives feedback, correct or wrong. The server then deliberately shuts down.
– ping: sends an AJAX request to the ping context which just sends back the time, the response is appended to the console.
– end: sends an AJAX request to the ‘stop’ context. The server responds with ‘stopping server …’, then shuts down. All buttons are disabled.
Why Groovy
Groovy is a dynamic JVM based language whose most prominent feature is that it extends the Java syntax to be more usable, i.e., less wordy,. From the example, a method that prints the contents of a map can be defined as:
The Groovy In Action book is pretty thorough. Chapter 1 makes a good case, and may still be available here.
Why not Groovy? Well, being dynamic can be rough, especially if it impacts the ability of a IDE to offer the features that come from using Java, like completions, etc. The Eclipse plug-in is getting much better, and I read the IntelliJ IDEA groovy support is top-notch. But, the worlds most popular language, JavaScript, is dynamic, and it hasn’t bothered too many people (maybe end users?).
Source
When I first wrote this, I created a complicated “app” with a bunch of class files and so forth. Then I simplified it. Later I said, yuck. Finally I decided that this should just be a simple one method invocation as shown at the beginning of this post. Consequently, a bunch of stuff in this sample code is guarded by conditionals and it works but sure can be simplified.
While coding I ran into a strange classloader issue see this post. There are a few files in this demo
SimpleServer.groovy: provides the ‘facade’ to HTTPServer. Handles the “/” context.
AppContext.groovy: An example of an “app” that gets mapped to a context path.
/**
* File: AppContext.groovy
* Date: 20110320T1952-05:00
* Author: jbetancourt
*/
package main
import java.text.DateFormat;
import java.text.SimpleDateFormat
import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpHandler;
/**
* The context that hosts the application.
*
* @author jbetancourt
*
*/
class AppContext implements HttpHandler{
def SimpleServer server;
def static RUNNING = 'running'
def currentState
def transitions = [:]
/** */
AppContext(SimpleServer server){
this.server = server
currentState = server.getServerProperty("initialState")
def statesProperty = server.getServerProperty("transitions")
}
/**
* Handle the given request/response.
*
* The first response is the input form. The subsequent
* request evaluates the answer if any, and then the server
* is stopped. Any exception will also stop the server.
*
*/
@Override
public void handle(HttpExchange t) throws IOException {
try{
def uri = t.getRequestURI()
def final query = uri.getRawQuery()
def path = uri.getRawPath()
def params = server.parseQuery(query)
showInfo([query:query,uri:uri,path:path,currentState:currentState,params:params])
def mode = params.get("mode")
if(atState("running")){
if(mode){
if(mode[0] == "mathInput"){
def reply = params.get("reply")
evaluateAnswer(t,reply)
transitionNextState()
}else if (mode[0]=="ping") {
server.sendString(t,"huh?")
}
}
}
if(atState("end")){
server.stopServer()
}
}catch(Exception ex){
ex.printStackTrace()
server.stopServer()
throw ex;
}
}
/**
* And, send response.
*/
def evaluateAnswer(t,answer){
def reply
try{
reply = (answer[0] != '"23"') ? "Wrong!" : "Correct!"
}catch(Exception ex){
reply = "wrong"
}
server.sendString(t,"</pre>
<center>
<h1>$reply</h1>
</center>
<pre>")
}
/** */
def showInfo(info){
println "++++++++ AppContext +++++++++++++++++"
info.each{ k,v ->
println k + (v ? ': [' + v +']' : ": []")
}
println "-------------------------"
}
/** */
def ping(t){
def now = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss").format(new Date())
server.sendString(t, "$now")
}
/** */
def setTransitions(s){
s.split(",").each{ tran ->
def kv = tran.split(":")
transitions.put(kv[0],kv[1])
}
}
/** */
def atState(s){
return currentState == s
}
/** */
def transitionNextState(){
print("currentState[$currentState],")
def ns = transitions[currentState]
currentState = (ns ? ns : "end")
println(" next state=[$currentState] ")
}
} // end AppContext class
Summary
Shown was a simple example of using the JDK 1.6 embedded HTTP server using the Groovy language.
Updates
April 15, 2011: Added some beginnings of code to handle image resources.
August 4, 2011: Just saw an old post on same subject. “Groovy++ in action: DSL for embedding HttpServer”. Added it to references. That one creates a DSL to use the embedded HTTP server. Nice.
I just manually deleted the compiled Groovy classes, then ran the app. Works. Now I rerun still works. Now add a simple blank line to one of the sources and run. It died. Classloaders are different for other Groovy classes.
In an Eclipse Groovy project, the order of compilation is causing Groovy classloader conflicts.
I just manually deleted the compiled Groovy classes, then ran the app. Works. Now I rerun still works. Now add a simple blank line to one of the sources and run. It died. Manually delete the output class files, works again. 🙁
Strange I don’t have “rebuild automatically” turned on. Also strange, in the project properties, for builders, only the Java builder is listed.
The console output when I get the error:
Caught: groovy.lang.MissingMethodException: No signature of method: static main.SimpleServerSupport.keepAlive() is applicable for argument types: (main.QuestionServer, java.lang.Long) values: [main.QuestionServer@b32ed4, 60000]
Possible solutions: keepAlive(main.QuestionServer, java.lang.Long)
The following classes appear as argument class and as parameter class, but are defined by different class loader:
main.QuestionServer (defined by ‘groovy.lang.GroovyClassLoader$InnerLoader@16921fd’ and ‘org.codehaus.groovy.tools.RootLoader@42e816’)
If one of the method suggestions matches the method you wanted to call,
then check your class loader setup.
at main.QuestionServer.main(QuestionServer.groovy:72)
Haven’t solved this yet. I can’t find any reference to something like it. Maybe I just have a config issue?
I now remember that the plugin adds hooks into the Java compiler. So is the compiler used different when you run the app with “run as Groovy script” then when Java builder executes, that is what the Classloader names would suggest? But, that makes no sense, since at run time, the classloader hierarchy has nothing to do with the compile, or does it, since Groovy precompiles at runtime?
Yes, I uninstalled the Groovy feature and reinstalled.
Running the same program (script) at the command line is fine.
Todo: Post some code that exhibits the issue. The current code is still at the “throw at wall and see what sticks” stage.
Updates
20110328T2010-5: Solved it. I moved a method from a support class, into the main SimpleServer class. This method was using an Executor.newScheduledThreadPool(1) to start a thread to limit the server’s lifetime, it also did a little console spinner to show that there is something running. I don’t see how that is related to classloaders.
Environment
Windows 7 64bit Professional
Eclipse Helios Service Release 2
Groovy-Eclipse Feature 2.1.3.xx-20110317-1200-e36
Embed a script engine and have new ways of analyzing and developing code.
In development, simple but powerful tools to get at the essence of a code source in order to understand, test, and extend it is essential. This is even more important in Test Driven Development (TDD). Eclipse’s Java Debugger is for most situations, powerful enough. Eclipse has an Expressions View available in the debug perspective to execute snippets of code. The Java Development Toolkit (JDT) also has a Scrapbook facility that allows the creation, storage, and running of experimental code. And, of course, all the other features of the debugger are excellent.
However, when you need it, it’s possible to embed a script engine and have new ways of analyzing and developing code. In listing 1, an app shows the use of the ConsoleWaiter class. When the code executes the waiter.run() at line 43, it opens the Groovy Console which allows the use Groovy shell scripting in a GUI frame, see figure 1. When the console is closed the app continues executing.
Listing 1
/*
* File: ExampleApp2.java
* @author jbetancourt
* Date: 20101213T1718-5
*
*/
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
/**
*
* @author jbetancourt
*
*/
public class ExampleApp2 {
static public String greeting = "Hello world!";
private static final List<String> nodes;
static {
nodes = new
ArrayList<String>(
Arrays.asList("one","two"));
}
private String title = "Example 2";
public String getTitle(){
return title;
}
/**
* @param args command line args
*/
public static void main(String[] args) {
ExampleApp2 app = new ExampleApp2();
ConsoleWaiter waiter = new ConsoleWaiter(app);
waiter.setVar("greet", greeting);
waiter.setVar("nodes", nodes);
waiter.setVar("title", app.getTitle());
waiter.run();
System.out.println("Done!");
}
}
Console screen capture, click to viewAnother screen capture, click to view
This is awesome. In one project I had to examine the contents of a Properties object. Did it have an “email” value? I was stumped when using the Eclipse debugger, it did not show all entries in the Map, at the end was “…”. Sure, I could use the Expressions window, but with the Console I could not only do a get(key), but iterate using Groovy style closures and much more.
The magic that enables this is the ConsoleWaiter.groovy class shown below in listing 2 that was written by John Green. Since a Groovy script is a Java class underneath, in Eclipse you can call Groovy from Java easily (some compiler magic).
Listing 2
/**
* File: ConsoleWaiter.groovy
*/
import groovy.lang.Binding;
import groovy.ui.Console;
/**
* Provides a wrapper for the console.
*
* Based on source by John Green
* Adapted from: http://www.oehive.org/files/ConsoleWaiter.groovy
* Released under the Eclipse Public License
* http://www.eclipse.org/legal/epl-v10.html
*
* I added methods to allow use from Java.
*
* The run() method launches the console and causes this thread
* to sleep until the console's window is closed.
* Allows easy interaction with the objects alive at a given
* point in an application's execution, like in a debugger
* session.
*
* Example 1:
<pre> * new ConsoleWaiter().run()
*</pre>
*
* Example 2:
<pre> * def waiter = new ConsoleWaiter()
* waiter.console.setVariable("node", node)
* waiter.run()
*</pre>
*/
class ConsoleWaiter {
Console console
Object source
boolean done = false;
/** */
public ConsoleWaiter(Console inConsole){
this.console = inConsole
}
/** */
public ConsoleWaiter(Object source){
console =
new Console(getClass().classLoader,
new Binding())
this.source = source
console.setVariable("source", source)
}
/** */
public void setVar(String key, Object value){
console.setVariable(key, value)
}
/** */
public void setVar(String key, List values){
console.setVariable(key, values)
}
/** */
public void setVar(String key, Object[] values){
console.setVariable(key, values)
}
/** */
public void run() {
console.run()
// I'm a little surprised that this exit() can be private.
console.frame.windowClosing = this.&exit
console.frame.windowClosed = this.&exit
while (!done) {
sleep 1000
}
}
/** */
public boolean isDone(){
return done;
}
/** */
public void exit(EventObject evt = null) {
done = true
}
/** */
public Console getConsole(){
return console;
}
}
Eclipse Integration
The disadvantage of this approach is that you have to put extraneous code inside the tests or target class. Not only is this tedious and slow, what if code is accidentally deployed like this? A better approach is to just set a breakpoint in the code, and then have the ability to open a script console at that breakpoint, in the Java Stack Frame, that has access to the execution context and classpath. Is there an Eclipse add-in that does this? If not, there should be.
Conclusion
Shown was a simple example of embedding a Groovy console in Java code to allow scripting. Of course, this is not a new idea. It was even mentioned in an older JVM scripting language, Beanshell. Note that it is possible to instead of using a GUI console, to use the Groovy shell, InteractiveShell class. In the reading list below this approach is taken to allow remote scripting of a server hosted application.
Updates
Oct 10, 2011:
Interesting tool that could be relevant: YouDebug.
March 20, 2012: Not exactly same scenario, but the concept of an embedded script console is found in many products. Jenkins CI Server has one and it uses Groovy. Jenkins Script Console
Unless you start Eclipse with the “-showLocation” option there is no way to know what the current workspace location is?
Maybe someone knows the answer. Unless you start Eclipse with the “-showLocation” option there is no way to know what the current workspace location is. Another method is to use the Eclipse menu system: File -> Switch workspace -> Other. The workspace will be in the resulting dialog box.
There are also a few configuration settings available in the Eclipse -> About -> Show configuration:
osgi.install.area=file:/C:/java/eclipse-4.2/
osgi.instance.area=file:/C:/Users/jbetancourt/workspace-4.2/
osgi.instance.area.default=file:/C:/Users/jbetancourt/workspace/
One reason to know is that when I go to create a new project, do I specify the workspace or is the current workspace the one I want? I don’t remember if the workspace is shown or specified on project creation, will have to check.
Maybe I’m missing some internal Eclipse reason for this (similar to that crazy bug when the copyright name was changed inside Java) and there is no way for Eclipse to know this. Still, seems lame.
Update
10 JUL 2012: Eclipse Juno: -showLocation doesn’t seem to be working, doesn’t show location in title bar of Eclipse.
17 JAN 2011: Found the answer, see below. I was on the right track. If I read the FAQ entry correctly, the current project is accessible (even though there can be many active projects), it is just not exposed in the Eclipse UI? But, that doesn’t really show the workspace, of course.
In a prior post “Java Plain Old Concurrent Object” I wrote about a need for a higher level concurrency support in Java. Here I give the program code that I used to experiment with concurrency and learn a little more.
Summary
Presented are a few ways of coding a concurrent program using Java threading and also using a Communicating Sequential Processes (CSP) library. They are written in the Groovy language. Used as an example is a simple lottery number generation application.
In a prior blog post(Java Plain Old Concurrent Object) I wrote about a possible need for a higher level concurrency support in Java. Here I give the program code that I used in September 2007 to learn more then the rudimentary Java concurrency concepts.
I managed to use CSP and the usual Java concurrent library support, and recently was about to try Actors using the new GPars library. It was a good way to get a feel for each approach. Note that it was not enough to become a concurrency expert; that was not a goal; still studying Goetz’s book Java Concurrency In Practice.
My first use of multitasking code was in C++ code I did in 1992:
… technical hurdles with the product, so I put it aside until I could get back to it. One thing that I was proud of was learning a little bit of Fuzzy Logic and using it as the controller. I even wrote a graphical simulator in the C++ language; threading was fun stuff. Watching the fuzzy sets behave like analog surfaces or neural EEG waves gave me the idea for the biomimicry aspects. — An Adaptive Controller Using Fuzzy Memory
Problem
For code example I used lottery number play generator. Powerball is a U.S. multi-state lottery game. A Powerball play is composed of 5 white balls in range 1 – 55 and one red ball in the range of 1 – 42. As you can imagine the odds are extremely high that you could guess the Jackpot set (1 in 195,249,054). The jackpot gets very large and that is a great enticement to forgo any sanity and “you can’t win if you don’t play” you know.
If you enjoy the frills, ambiance, and clientele, gambling is a great way to throw away money. For more on Lottery see: Lottery Links back to top
Source Code
To simulate real random number generation, the program will continuously generate plays, but only produce the requested numbers of generated sets when the user clicks the enter key. I’m assuming a user waiting a certain amount of time adds a random element to the PRNG being used. Does it? I don’t think so, but that’s another topic.
A few points:
The problem could have been solved without using concurrent approach.
Example code use at your own risk. No correctness or quality guaranteed.
A Thread monitor view using VisualVM doesn’t show much, just six threads with four daemon ones.
We use the following batch file, which simply invokes the Main.groovy script.
@rem File: run.cmd
@rem Created 28 Dec 2008 J. Betancourt
call setEnv.cmd
groovy ..\src\main\groovy\net\coxmembers\jbetancourt1\Main.groovy %*
The main script driver is shown in listing 1. Two arguments are specified, how many dollars to lose and what type of concurrency approach to use. Executing it with no arguments would give:
c:Users\jbetancourt\Documents\projects\dev\ConcurrentGroovy-1bin>run
usage: usage -t type -d dollars [-h]
-d,--dollars How many dollars to lose
-h,--help Usage information
-t,--type Type of threading
Options for '-t' are:
"simple" ----> "Using simple Threads"
"lock" ----> "Using lock"
"executor" ----> "Using Futures"
"csp" ----> "Using JCSP"
"actor" ----> "Using Actors"
Example: run -d 5 -t simple
And, here is a sample run:
c:Users\jbetancourt\Documents\projects\dev\ConcurrentGroovy-1bin>run -d 5 -t simple
Setting up games 5 using "simple"
The odds of winning jackpot are: 1 in 146,107,962.00
Enter return key
Number of games generated are: 5
1: 7 11 14 23 34 PB: 9
2: 21 22 27 28 55 PB: 21
3: 2 9 30 37 46 PB: 30
4: 7 49 50 51 53 PB: 30
5: 9 18 24 40 42 PB: 32
Good luck!
Success?
Not really. A funny thing about this code, if you invoke it and quickly hit the return key, it doesn’t work correctly. Concurrent code can be hard to craft.
Example:
>run -d 5 -t executor
Setting up games 5 using "executor"
Press the Enter key on keyboard
Number of games generated are: 1
1: 18 20 22 27 41 PB: 25
Good luck!
Listing 1 Main driver class:
package net.cox.members.jbetancourt1;
import org.apache.commons.collections.buffer.CircularFifoBuffer
import org.apache.commons.collections.Buffer
import java.security.SecureRandom
import org.apache.commons.collections.BufferUtils
/**
* Generate Powerball plays.
* Required libraries:
* - Commons CLI
* - Commons collection
*
* A Powerball play is composed of 5 white balls in range 1 - 55
* and one red ball in the range of 1 - 42
* This script gets how many games to play, then
* continually generates this number of games
* and adds them into a circular buffer. This is
* repeated until the user hits a key, which
* interrupts the generation thread and dumps the results.
*
* A circular buffer is used since the results are
* only needed when the user hits a key,
* thereby making it "random". In actuality,
* since we are using a psuedo-random
* generator, this is still not true random.
* There is no way to improve the odds of
* picking the correct numbers for a lottery game.
*/
class Main{
def apps = [
"simple": [new GuessSimple(),"Using simple Threads"],
"lock": [new GuessWithLock(),"Using lock"],
"executor": [new GuessWithExecutor(),"Using Futures"],
"csp": [new GuessCSP(),"Using JCSP"],
"actor" : [new GuessActor(),"Using Actors"]
]
/** Main entry point */
static main(args){
def main = new Main()
def options = main.parseArguments(args)
if((options != null ) && options.d && options.t){
println "Setting up game" + (options.d>0? "s": "") +
" ${options.d} using "${options.t}""
def dollars = Integer.parseInt(options.d)
def games = BufferUtils.synchronizedBuffer(
new CircularFifoBuffer(dollars))
def threadType = ""
if(options.t){
threadType = options.t
}
def guess = main.apps[threadType][0]
guess.generate(dollars,games)
}
System.exit(0)
} // end main
/****************************************************************************
* Using CLI builder, parse the command line.
*/
def OptionAccessor parseArguments(String[] args){
def cli = new CliBuilder(usage: 'usage -t type -d dollars [-h]')
cli.h(longOpt: 'help', 'Usage information')
cli.t(longOpt: 'type', 'Type of threading',args:1)
cli.d(longOpt: 'dollars','How many dollars to lose',args:1,required:false)
def options = cli.parse(args)
if(options == null || options.getOptions().size() == 0 || options.h){
cli.usage()
println "nOptions for '-t' are: "
apps.each{
println("t"${it.key}"".
padRight(12) + " ----> "${it.value[1]}"")
}
println "nExample: run -d 5 -t simple"
return options
}
return options
}
} // end class Main
Listing 2 Base class:
<pre>package net.cox.members.jbetancourt1;
import org.apache.commons.collections.buffer.CircularFifoBuffer
import org.apache.commons.collections.Buffer
import java.security.SecureRandom
import org.apache.commons.collections.BufferUtils
/**
*
*/
class GuessBase{
def whiteBalls = [0]
def redBalls = [0]
def random = new SecureRandom()
//each element is an array of 0-4 white ball values and 5 is the red ball value.
def games //= BufferUtils.synchronizedBuffer(new CircularFifoBuffer(dollars))
def createBallSets(){
def i = 0
for(j in 1..55){
whiteBalls[i++] = j
}
i = 0
for(j in 1..42){
redBalls[i++] = j
}
}
/** Pick the set of white balls */
def pickWhite(){
def array = []
array.addAll(whiteBalls)
Collections.shuffle(array,random)
return array[0..4]
}
/** swap two elements in a list */
def swapElements(list,i,j){
def temp = list[i]
list[i] = list[j]
list[j] = temp
}
/** Pick the single red 'powerball'. */
def pickRed(){
int offset = random.nextInt(redBalls.size())
return redBalls[offset]
}
def showGames(){
def gamesSorted = games.sort(){a,b -> a[5] <=> b[5]}
def rownum = 1
println "Number of games generated are: ${games.size()}"
for (loop in gamesSorted){
print ((rownum++ + ": ").padLeft(8))
for(x in ( loop[0..4]).sort() ){
print ((x.toString()).padLeft(3))
}
println " PB:" + ((loop[5]).toString()).padLeft(3)
}
println "Good luck!"
}
} // end class GuessBase
Listing 3 Using simple Java Thread class:
package net.cox.members.jbetancourt1;
/**
* Example that uses a simple 'inline' thread.
*/
class GuessSimple extends GuessBase{
/** create d guesses and store into games array */
def synchronized generate(d,games){
this.games = games
createBallSets()
def worker = new Thread();
worker.setDaemon(true);
worker.start{ // Groovy allows closure here.
while(!Thread.currentThread().isInterrupted()){
def w = pickWhite()
def r = pickRed()
games.add((Object)(w+r))
random.setSeed(System.currentTimeMillis());
}
}
println "The odds of winning jackpot are: 1 in 146,107,962.00nEnter return key"
new InputStreamReader(System.in).readLine()
worker.interrupt()
Thread.sleep(1000) // required!
showGames()
}
} // End of GuessSimple.groovy
Listing 4 Using Object lock:
package net.cox.members.jbetancourt1;
/**
*
*/
class GuessWithLock extends GuessBase{
def lock = new Object() // for thread synchronization
/** create d guesses and store into games array */
def generate(d,games){
this.games = games
createBallSets()
def worker = new Thread(){
void run(){
while(!Thread.currentThread().isInterrupted()){
def w = pickWhite()
def r = pickRed()
games.add((Object)(w+r))
random.setSeed(System.currentTimeMillis());
}
synchronized(lock){
lock.notify()
}
}
}
worker.start()
println "The odds of winning jackpot are: 1 in 146,107,962.00"
println "Enter any key<cr>"
new InputStreamReader(System.in).readLine()
worker.interrupt()
showGames()
}
}
Listing 5 Using Executor:
/**
* File: GuessWithExecutor.groovy
* Author: Josef Betancourt
* Date: 9/1/2007
*
*/
package net.cox.members.jbetancourt1;
import org.apache.commons.collections.*
import java.util.concurrent.*
/**
* Generate powerball plays using Executor.
*/
class GuessWithExecutor extends GuessBase {
/** create d guesses and store into games array */
def generate(d,games){
def dollars = d
try{
def exec = Executors.newSingleThreadExecutor()
def gen = new GameGenerator<Buffer>(dollars,games)
gen.createBallSets()
def future = exec.submit(gen)
println "Press the Enter key on keyboard"
new InputStreamReader(System.in).readLine()
exec.shutdownNow()
waitForTermination(exec,100,10) // 100ms, max of 10 tries
try{
def buffer = future.get()
def exec2 = Executors.newSingleThreadExecutor()
exec2.execute(new GenerateReport(buffer))
exec2.shutdown()
}catch(ExecutionException ignore){
ignore.printStackTrace()
}
}catch(Exception ex){
ex.printStackTrace()
}
}
/**
*
* @param exec the Executor
* @param time how long to wait
* @param unit unit of time
* @param tries how many times to wait
*/
def waitForTermination(exec,time,tries) throws Exception {
def count = 0
while(!exec.awaitTermination(time, TimeUnit.MILLISECONDS)){
if(count++ >= tries){
break
}
}
}
} // end class GuessWithExecutor
/**
* A Callable that generates games until interrupted.
*/
def class GameGenerator extends GuessBase implements Callable {
def GameGenerator(dollars,games){
this.games = games
}
/**
* Create d guesses and store into games array.
* @see java.util.concurrent.Callable#call()
*/
public Object call() throws Exception {
while(!Thread.currentThread().isInterrupted()){
def w = pickWhite()
def r = pickRed()
games.add((Object)(w+r))
random.setSeed(System.currentTimeMillis());
}
return games
}
} // end class GameGenerator
/**
* Class to render the results
*/
def class GenerateReport extends GuessBase implements Runnable {
def GenerateReport(Buffer games){
this.games = games
}
public void run(){
showGames()
}
} // end class GenerateReport
Listing 6 Using Fork/Join:
// to do: approach will be to fork each required game set into its own thread, then join them to get the final result. Crazy, but a way to learn about Fork/Join and have sample code.
CSP
Some experts are touting the benefits of CSP. Below I use the jCSP library to create a network of processes that collaborate to solve the same problem. Note that some of the people involved in jCSP are adding some CSP support to GPar.
Listing 7 Using jCSP:
/**
* File: CspGuess.groovy
* Author: Josef Betancourt
* Date: 9/1/2007
*
*
*/
package net.cox.members.jbetancourt1;
import jcsp.lang.*;
/**
* Generate powerball plays using CSP.
*
* This version of the example uses the JCSP library. Note
* that unlike the other examples, there is an explicit
* declaration of a 'network'.
*
* Required libraries:
* - JCSP ver. 1.0-rc8 CSP library in Java:
* <a href="http://www.cs.ukc.ac.uk/projects/ofa/jcsp"></a><br/>
*
* Uses PAR and ALT methods presented in:
* <a href="http://www.soc.napier.ac.uk/publication/op/getpublication/publicationid/9097759">
* "Groovy Parallel! A Return to the Spirit of occam?"</a>
* by Jon KERRIDGE, Ken BARCLAY, and John SAVAGE
* The School of Computing, Napier University, Edinburgh EH10 5DT in
* Communicating Process Architectures 2005 13
* Jan Broenink, Herman Roebbers, Johan Sunter, Peter Welch, and David Wood (Eds.)
* IOS Press, 2005
*
* Perhaps, a better approach is possible using actors? See for example,
* <a href="http://gpars.codehaus.org/">Groovy Parallel Systems</a>
*
*/
class GuessCSP extends GuessBase {
def generate(dollars,games){
def suspendChannel = new One2OneChannel()
def resultChannel = new One2OneChannel()
// Create the CSP network and run it.
new PAR( [
new GameProcess(suspendChannel,resultChannel,games,dollars),
new UserInputProcess(suspendChannel),
new ReportProcess(resultChannel)]
).run()
}
}
/** process that generates games */
def class GameProcess extends GuessBase implements CSProcess{
def out
def suspend
def dollars
/** Create the process instance */
def GameProcess(done,out,games,dollars){
this.dollars = dollars
this.suspend = done
this.out = out
this.games = games
createBallSets()
}
/** run the process network */
public void run(){
def alternative = new ALT([suspend, new Skipper()])
def STOPPING=0, RUNNING=1
println "Creating ${dollars} game${(dollars>0? "s": "")} "
def suspended = false
while (!suspended){
switch (alternative.priSelect ())
{
case STOPPING:
suspend.read();
if(games.size()>0){
suspended = true;
out.write(games)
}
break;
case RUNNING:
def w = pickWhite()
def r = pickRed()
games.add((Object)(w+r))
random.setSeed(System.currentTimeMillis());
break
}
}
} // end run()
} // end GameProcess process
/** Process that gets the user input */
def class UserInputProcess implements CSProcess{
def suspend
def UserInputProcess(done){
this.suspend = done
}
public void run(){
println "Enter any key<cr>"
new InputStreamReader(System.in).readLine()
suspend.write(new Object())
}
} // end UserInputProcess process
/**
* Process to render the results to console
*/
def class ReportProcess extends GuessBase implements CSProcess{
def input
def ReportProcess(input){
this.input = input
}
public void run(){
games = input.read() // get result via channel only
showGames()
}
} // end of ReportProcess process
/**
* PAR A Groovyish Parallel.
*/
def private class PAR extends Parallel {
PAR(processList){
super( processList.toArray(new CSProcess[0]))
}
}
/**
* ALT A Groovyish Alternative.
*/
def private class ALT extends Alternative {
ALT (guardList) {
super( guardList.toArray(new Guard[0]) )
}
}
// end of CspGuess.groovy
Actor
I started to look into the Actor approach. Below is the start of code to solve the same problem. Even with the few lines of code, attempts to run it give an an exception, and classpath and other easy fixes do not solve it. On my todo list.
Setting up games 5 using "actor"
start
Caught: groovy.lang.MissingMethodException: No signature of method: static groovyx.gpars.actor.Actor.actor() is applicable for argument types: (net.cox.members.jbetancourt1.GuessActor$_generate_closure1) values: [net.cox.members.jbetancourt1.GuessActor$_generate_closure1@1fe571f]
Possible solutions: stop(), wait(), start(), any(), call(java.lang.Object), wait(long)
at net.cox.members.jbetancourt1.GuessActor.generate(GuessActor.groovy:55)
at net.cox.members.jbetancourt1.Main.main(Main.groovy:63)
Listing 8 Using Actors:
package net.cox.members.jbetancourt1
//import static groovyx.gpars.actor.Actors.actor
import groovyx.gpars.actor.*
import groovyx.gpars.actor.Actor.*
// @Grab(group='org.codehaus.gpars', module='gpars', version='0.9')
// @GrabResolver(name='jboss', root='http://repository.jboss.org/maven2/')
/**
*/
class GuessActor extends GuessBase {
def main = new GuessActor()
def dollars
def games
main.generate(dollars, games)
}
def generate(dollars,games){
println "start"
def gen = Actor.actor { index ->
loop {
react {message ->
if (message instanceof String) reply "got it"
else stop()
}
}
}
} // end of class GuessActor
Declarative CSP Using Spring
While experimenting I had an idea that ordinary Java beans could be used in a CSP network with suitable object wrappers or weaving. That is, an ordinary POJO could be wrapped to appear as a Process, just hook up the bean’s entry point for service into the CSP channel end points. This would be similar to the way that ordinary beans can be annotated to behave as Active Objects in the GPars Groovy library.
Below is the Spring Framework configuration file that does this for this software example. The code for the wrapper objects are not included here, too kludgey.
Off topic: The XML language config of Spring is looking long in tooth. Good thing Spring now supports annotations and Java config. Time for a Groovy builder approach too?.
<?xml version="1.0" encoding="UTF-8"?>
<!--
File: applicationContext.xml
Spring Framework based bean definitions for JCSP example program.
Author: Josef Betancourt
Date: 9/19/2007
-->
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:lang="http://www.springframework.org/schema/lang" xmlns:util="http://www.springframework.org/schema/util"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
http://www.springframework.org/schema/lang
http://www.springframework.org/schema/lang/spring-lang-2.0.xsd
http://www.springframework.org/schema/util
http://www.springframework.org/schema/util/spring-util-2.0.xsd">
<!-- ========================================================= -->
<!-- CSP NETWORK -->
<!-- ========================================================= -->
<bean name="network" dependency-check="objects" class="jcsp.lang.Parallel">
<constructor-arg>
<list>
<ref bean="gameProcess" />
<ref bean="reportProcess" />
<ref bean="userInputProcess" />
<ref bean="progressProcess"/>
</list>
</constructor-arg>
</bean>
<!-- ========================================================= -->
<!-- CHANNELS -->
<!-- ========================================================= -->
<bean name="suspendChannel" class="jcsp.lang.One2OneChannel"/>
<bean name="outputChannel" class="jcsp.lang.One2OneChannel"/>
<bean name="progressChannel" class="jcsp.lang.One2OneChannel"/>
<bean name="skipperChannel" class="jcsp.lang.Skip"/>
<!-- ========================================================= -->
<!-- GUARDS -->
<!-- ========================================================= -->
<bean name="alternative" class="jcsp.lang.Alternative" dependency-check="objects">
<constructor-arg>
<list>
<ref bean="suspendChannel" />
<ref bean="skipperChannel" />
</list>
</constructor-arg>
</bean>
<!-- ========================================================= -->
<!-- POJOs -->
<!-- ========================================================= -->
<!-- Creates the game generator, but it is setup programmatically
since it needs the user supplied number of games to play, see GameGenerator.setup(int).
-->
<bean name="gameGenerator" class="net.cox.members.jbetancourt.pb.game.GameGenerator">
<property name="random">
<bean class="java.security.SecureRandom"/></property>
<property name="maxRed" value="42" />
<property name="maxWhite" value="55" />
</bean>
<!-- ========================================================= -->
<!-- PROCESSES -->
<!-- ========================================================= -->
<!-- Receives signal from user to accept current generated game sets.
-->
<bean name="userInputProcess" init-method="init" class="net.cox.members.jbetancourt.pb.process.JCspSingleShotProxy">
<property name="outputChannel" ref="suspendChannel" />
<property name="methodName" value="run"/>
<property name="target">
<bean class="net.cox.members.jbetancourt.pb.game.KeyListener" scope="prototype"/>
</property>
</bean>
<!-- Game generator. Communicates with input and report processes.
-->
<bean name="gameProcess" dependency-check="objects"
class="net.cox.members.jbetancourt1.pb.process.GameProcess">
<property name="alternative" ref="alternative" />
<property name="suspendChannelIn" ref="suspendChannel" />
<property name="reportChannelOut" ref="outputChannel" />
<property name="progressChannelOut" ref="progressChannel"/>
<property name="gameGenerator" ref="gameGenerator"/>
<property name="showInterim" value="false"></property>
</bean>
<!-- Output results to console.
-->
<bean name="reportProcess" dependency-check="objects"
class="net.cox.members.jbetancourt1.pb.process.ReportProcess" scope="prototype">
<property name="input" ref="outputChannel" />
</bean>
<!-- Output interim results to console
-->
<bean name="progressProcess" dependency-check="objects"
class="net.cox.members.jbetancourt1.pb.process.ReportProcess" scope="prototype">
<property name="input" ref="progressChannel" />
</bean>
</beans>
Listing 10 Java main to load the network:
package net.cox.members.jbetancourt1.pb;
import jcsp.lang.Parallel;
import net.cox.members.jbetancourt1.pb.game.GameGenerator;
import org.apache.commons.cli.*;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;
/**
*
* Main program.
*
* @author JBetancourt
*
*/
public class Main {
private static final String TITLE =
"====== Lottery Generator using JCSP and Spring ============";
private static Parallel network;
private static ApplicationContext context;
/**
* Entry point for running the Guess application. Parses
* command line for number of games to play, loads
* wired application using Spring Framework
* based IoC, sets up game array, and then starts
* the network of processes.
* @param args
*/
public static void main(String[] args) {
System.out.println(TITLE);
int dollars = parseCommandLine(args);
if ( dollars == 0 ) {
return;
}
context = new ClassPathXmlApplicationContext(
new String[] { "applicationContext.xml"});
GameGenerator gen = (GameGenerator) context.
getBean("gameGenerator");
gen.setup(dollars);
network = (Parallel) context.getBean("network");
network.run();
}
/**
*/
private static int parseCommandLine(String[] args) {
... ellided ...
}
}
One possible conclusion is that there is no need for a ‘higher’ level concurrency support in Java or that this should evolve on its own before attempts at standardization are attempted.
Abstract
This blog post reiterates the existing critique of the concurrency support in the Java platform and language: that the low level thread and shared memory features should be improved upon with one or more high-level concurrency frameworks. Some of the competing frameworks are listed, and further references are given.
CR Categories: D1.3 [Concurrent Programming]; D.1.5 [Object-oriented Programming]; D.2.2 [Design Tools and Techniques]: Design Tools and Techniques — Software libraries;D3.2 [Language Classifications]: Concurrent, distributed, and parallel languages;D.3.3 [Language Constructs and Features]: Concurrent programming structures;
Keywords and Phrases: concurrency, Java, Actor Model, threads, Component based software engineering, Concurrent object-oriented programming, Distributed systems, Java CSP, CCS, multiprocessor
This is an excerpt from a document I wrote about three years ago. The doc was in a Java Specification Request (JSR) format and from time to time I updated the link section as I came upon interesting references. Maybe someone will find it interesting or it will spark a new idea for a project. In a later post I give some examples of concurrent code using the Groovy language.
Intro
The Java Platform has built-in support for concurrent programming. At the time of Java’s birth this was very big deal. But, is it now time to build upon this by the creation of a requirements document for high-level concurrency support in the Java language? Could this bring Java closer to a Concurrent Object-Oriented Language (COOL)?
Well, since I’m not a concurrency subject matter expert, just a working stiff developer, I will leave that to others, and just put my two cents here. What prompted me even looking into this subject was my experience on a few projects that required concurrency support.
What is needed is a survey or summary of what exactly are the issues, what is available, what their application areas are, and what are the development options. Some of initial tasks I see, in no particular order, are:
State the problem and application scope.
Provide use-cases.
Create project
wiki, forums, etc.
conference
Limit the scope of the effort.
Identify solution categories.
Identify solution selection criteria.
Testing and management requirements.
Measures, such as Performance and scalability.
Tools.
One possible conclusion is that there is no need for a ‘higher’ level concurrency support in Java or that this should evolve on its own before attempts at standardization are attempted.
Problem
One thing has been left out of the OO frenzy is the concept of a process. There are two manifestations of this: Methodology and Concurrency. In the methodological realm, ‘process’, which was part of data-flow analysis, is practically missing from modern analysis (as typified by OOP/UML derived methods). And even now, modular approaches, are still divorced from concurrency concerns. For example, OSGi fortunately brings back modularity to the Java environment. As stated by Kriens:
“Why is modular important? Well, if there is one consistent lesson in all our technology trends over the past 50 years then it is high cohesion and low coupling. Structured programming advocated this; OO forgot it for some time ….” — Peter Kriens, http://forum.springframework.org/showpost.php?p=138459&postcount=2
Yet, even in OSGi, a module is still at the mercy of unconstrained concurrency effects:
“OSGi … does not provide a thread of execution to each bundle. Events are delivered to bundles through certain interfaces, but no guarantees are made about which thread event delivery occurs on. Generally, event callbacks are required to finish quickly and should not make calls back into the OSGi framework to avoid possible deadlock. It is common, therefore, for bundles to start one or more threads in order to get work done.” — Oliver Goldman, “Multithreading, Java, & OSGi”
Welch calls this “unconstrained OO”:
“In unconstrained OO, threads and objects are orthogonal notions — sometimes dangerously competitive. Threads have no internal structure and can cross object boundaries in spaghetti-like trails that relate objects together in a way that has little correspondence to the original OO design. ” – Peter H. Welch in Javadocs for jcsp.lang.CSProcess of the JCSP library
And, others give similar critiques:
All object methods have to be invoked directly (or indirectly) by an external thread of control – they have to be caller-oriented (a somewhat curious property of so-called object oriented systems). — P.H Welsh (“Process Oriented Design for Java: Concurrency for All”)
“In standard OO, if you hold a reference to friend, and you wish to invoke the borrowMoney() method, then the call friend.borrowMoney() is executed in your own thread of control, not in a separate thread (or in friend’s thread), thus breaking all similarity to the way the real world works.” — (Oprean and Pederson, 2008).
“Although the development of parallel languages began around 1972, it did not stop here. Today we have three major communication paradigms: monitors, remote procedures, and message passing. Any one of them would have been a vast improvement over Java’s insecure variant of shared classes. As it is, Java ignores the last twenty-five years of research in parallel languages.”
— JAVA’S INSECURE PARALLELISM by Per Brinch Hansen (1999)
“However, I think Java needs to change. When it was first released, its number one competitor was C++, and Java walked all over it in terms of support for concurrency. Threads in the standard library! Syntactic support for critical blocks! Implicit locks in every object! Now, Java’s number one competitor is C#, which is increasingly starting to look like a functional language. Also languages like Erlang and Haskell have concurrency primitives that walk all over Java’s (I particularly love Haskell’s composable memory transactions). I hope this is something that is addressed in Java 7, but I fear it won’t be…” — Neil Bartlett
A higher level concurrency support in Java will allow a more approachable use of concurrent development. Though the concurrency implementation in Java is a vast improvment over what was available in a popular language at the time, we are now seeing more interests in other approaches as shown in Scala, Erlang, Kilim, Akka, GPars, Haskell, and others.
A standard would then allow, within reason, the sharing of development expertise and common patterns and idioms among the different languages and frameworks. For example, could an Actor in Scala be behaviorially the same as an Actor in plain old Java?
We are also seeing multiple processor cores being used to continue the improvment in thruput since thermal and process limititions have reduced growth. These will also allow true parallism. A high-end actual product, Sparc T3, the Sun Microelectronics version of the chip multi-threaded (CMT) processor, will be capable of forming a quad core SOC (System on Chip) that offers 512 hardware threads. Even on personal desktop systems, we already have quad cores and soon hexacores will be commonplace in the high end, such as the Intel Nehalem-EX. (Note that concurrency and parallelism are not the same thing).
Update: This was written a while back. In the interim, Oracle bought Sun and in the process canceled the Rock high-end CMT based processor design. Yet, Oracle said they will continue to support the upcoming SPARC CMT versions. Interesting discussion is found here. Intel and AMD have not stood still. AMD, for example, is promising a 16 core “Interlagos” chip soon. On the mobile side, ARM and others are also producing multicore systems.
Plain Old Concurrent Component
Instead of just reusing existing terms, I can call this new concurrency support a Concurrent Component. A component is more similar to an OS process, whereas internally it may incorporate light weight threads.
A possible conceptual view of a Component or Plain Old Concurrent Object (POCO) is shown in figure 1 below . Note that this is intentionally reminiscent of a Programmable Logic Device (PLD) block diagram. Each block is an optional ‘concern’ that crosscuts each object that is part of the component. The nested objects can share state and optionally will execute within a fine-grain concurrency kernel. The external control communication is via interfaces that expose the Component API, whereas the actual messaging embeds the application API that the component instantiation provides. This is somewhat of the flavor of Cox’s Software IC concept (ref?). This is also similar to various old Microsoft COM models. In future, it’s even conceivable that each POCO could have its own core assigned in an dynamically created application specific multicore processor implemented in nanoprocessor FPGA.
And, yes the diagram is just my brain’s core dump on the subject. Would require more work to explore this further, and determine what would really make sense.
Concurrent Component
Why isn’t this need met by existing specifications?
Currently, developers can use the concurrency control constructs that are provided in the Java language itself. Many experts consider these too low level for some applications and for use by the average developer (in terms of development effort, correctness, and failure potentials):
Use Erlang-Style Concurrency. The Java concurrency primitives like locks and synchronized have been proven to be too low level and often to hard to use. There are better ways to write concurrent code. Erlang Style concurrency is one of them – in Java there are many ways to achive this in Java – I’ve written about them here. Newer ones are Akka and Actorom. You can also use Join/Fork or the myriad of data structures in java.util.concurrent. —- Stephan Schmidt in Go Ahead: Next Generation Java Programming Style
In Java 5 and above one uses the new java.util.concurrent utilities. These offer a powerful and relatively accessible API such as the Executor and the Fork/Join frameworks. However, though the latter were a significant improvement, their use still require advanced skills and really only offer the means to create application specific or JVM language based concurrency frameworks.
Even with concurrency updates in Java 6 and Java 7, the Java language doesn’t make parallel programming particularly easy. Java threads, synchronized blocks, wait/ notify, and the java.util.concurrent package all have their place, but Java developers pressed to meet the capacity of multi-core systems are turning to techniques pioneered in other languages.
Some languages or libraries, such as Scala (?) and JCSP even use the underlying concurrency support in Java to create the respective ‘active’ object concurrency extensions. See for instance this presentation by Doug Lea, “Engineering Fine-Grained Parallelism Support for Java 7”.
Note that introducing a higher level of abstraction will not necessarily reduce the need for skilled engineering. In fact, it may increase that need until the patterns, methodologies, measurement, and tool sets catch up and support the new framework. Of course, many other concerns are important, such as performance.
Underlying technology or technologies:
There is a rich history in both industrial and academic research into the theory and practices of concurrency. In the fifty plus years of this, surely there must be more that can be used in Java then Monitors, semaphores and other low level tools.
Some examples are Communicating Sequential Processes (CSP), Calculus of communicating systems (CCS), and all the ‘proven’ concurrent support already found in other languages, among them, Occam, Go, Erlang, Haskell, Esterel, Scala, and many others in the academic research community. There is even one observation that the current Monitor implementation in Java itself could more fully embrace the original theoretical Monitor concept.
Of course, as in anything else, there is hype and fashion in software too. Thus, some approaches are touted as fixing deadlocks and other problems, when they don’t.
· Michele Simionato, “Threads, processes and concurrency in Python: some thoughts”, [weblog entry.] The Explorer. 26 Jul 2010. (http://www.artima.com/weblogs/viewpost.jsp?thread=299551). 26 Jul 2010.
· Orlic, Bojan; “SystemCSP : a graphical language for designing concurrent component-based embedded control systems”. (2007) thesis. http://doc.utwente.nl/58009/
· “Gartner Says as the Number of Processors Swells Inside Servers, Organizations May Not Be Able to Use All Processors Thrust on Them”, http://www.gartner.com/it/page.jsp?id=867112
· “Tilera vs. Godzilla”, Louis Savain, http://rebelscience.blogspot.com/2007/08/tilera-vs-godzilla.html
· Irfan Pyarali, Tim Harrison, and Douglas C. Schmidt Thomas D. Jordan; “Proactor: “An Object Behavioral Pattern for Demultiplexing and Dispatching Handlers for Asynchronous Events”, http://www.cs.wustl.edu/~schmidt/PDF/proactor.pdf
· “A Component-Oriented Model for the Design of Safe Multi-threaded Applications”, Reimer Behrends, R. E. K. Stirewalt and L. K. Dillon Dept. of Computer Science and Engineering Michigan State University, accessed on 3/10/2009 at http://www.cse.msu.edu/~stire/Papers/cbse05.pdf
· “Integrating and Extending JCSP”, Peter WELCHa, Neil BROWNa, James MOORES b, Kevin CHALMERS c and Bernhard SPUTH. Communicating Process Architectures 2007. Accessed 2007 at xxxx.
· Yuri Gurevich, Wolfram Schulte, and Charles Wallace, “Investigating Java Concurrency using Abstract State Machines”. In Y. Gurevich, P. Kutter, M. Odersky, and L. Thiele, eds., Abstract State Machines: Theory and Applications, Springer LNCS 1912, 2000, 151-176.
· P. Brinch Hansen, A keynote address on concurrent programming. Keynote address for the IEEE Computer Software & Applications Conference, Chicago, IL, November 1978. Computer 12, 5 (May 1979), 50{56. Copyright °c 1979, Institute of Electrical and Electronics Engineers, Inc., accessed on 3/3/09 at http://brinch-hansen.net/papers/1978c.pdf
· “Concurrency: The Next Generation”, Damian Dimmich, Christian Jacobsen and Matthew Jadud, Computing Laboratory University of Kent Canterbury, CT2 7NZ, accessed on 3/3/09 at http://www.cs.kent.ac.uk/pubs/2006/2522/content.pdf
· “A type safe state abstraction for coordination in Java-like languages”,Ferruccio Damiani, Elena Giachino, Paola Giannini, and Sophia Drossopoulou,Acta Informatica, Volume 45, Numbers 7-8 / December, 2008, http://www.springerlink.com/content/2168q630k2355563/
Books
· The Art of Concurrency: A thread Monkey’s Guide to Writing Parallel Applications, Clay Breshears, O’Reilly Media, May 2009. 978-0-596-52153-0. http://oreilly.com/catalog/9780596521547
· Concurrency: State Models & Java Programs, Jeff Magee and Jeff Kramer
· Java Concurrency in Practice, Brian Goetz
· Concurrent Programming in Java, Doug Lea,
· Java Thread Programming, Paul Hyde,
· Java Threads, Oaks & Wong,
· Multithreaded Programming with Windows NT, Thuan Q. Pham and Pankaj K. Garg