A Groovy powered blog?

Check out the technologies being used at Guillaume Laforge’s new blog.

Check out the technologies being used at Guillaume Laforge new blog. Very impressive and looks great.

Maybe that blog system can be made into a project so that others can use it too?

Update

  1. Seems he is not too keen on Google’s new price structure: Google App Engine’s new pricing model
  2. Not Groovy related, but another post on web hosting strategies using services, like GitHub, S3, etc.: Strategy: Run a Scalable, Available, and Cheap Static Site on S3 or GitHub

Links

Other Links

 

 

 


Updates
– 2011-08-14T19:29 –
I gave a quick look at Gaelyk. At the first test run on local system I got:

WARNING: failed Server@98f352: java.net.BindException: Address already in use: bind
************************************************
Could not open the requested socket: Address already in use: bind
Try overriding --address and/or --port.

Which seems to indicate some kind of command line option switches. So, tried using –port 9080. No good, unknown command option.

Ok, its not one of those things you just flip a switch and your done, you have to pour thru docs from a whole bunch of stuff, RTFM!

Found the answer on the gradle-gae-plugin page. I just added httpPort = 9080 to the gae closure in the build.gradle script. It works!

Hmmm. I like convention over configuration, but the convention is that one will invariably change the port number, thus, that configuration should be explicit.

 

<eop/>

Using replaceAll with Groovy regular expressions

String.replaceAll can take regular expression pattern arguments. Tried it with a Groovy script and had a compile error.

String.replaceAll can take regular expression pattern arguments. Tried it with a Groovy script and had a compile error.

Ooops! Figured it out. In a slashy string, those using ‘/’ as terminators, the last character cannot be a ” since that will escape the terminator character, ‘/’. Thus,

def pattern = ~//

Will not compile. But, this:

def pattern = ~/${''}/

will. It is documented, I just kept missing it.

That is why it is used here:

println /c:abcd/.replaceAll((/${''}/),"/")

Alternatively, if your using Groovy 1.8* you can use the new $//$ slashy string:

println ( /c:abcd/.replaceAll(($//$),"/")  )

or

println ( (/c:abcd/ =~ $//$).replaceAll("/") )

I was reading this blog post on the need to sometimes not use regular expression in Java: “Tip #5 Avoid RegEx When Unnecessary.”

To take the string “c:abcd” and convert it to “c:/a/b/c/d”, just do s.replace(”, ‘/’);. Don’t use regexp. And, I thought, it would be easy to still use regular expressions if Groovy were used, doesn’t the slashy string remove the backslash headache?

Example that gets compile error:

println "new is: " +  /c:abcd/.replaceAll((//),"/")
C:temp>groovy test.groovy
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
C:temptest.groovy: 2: expecting anything but ''n''; got it anyway @ line 2, column 58.
   bcd/.replaceAll((//),"/")
                                 ^
1 error

I also tried using a compiled pattern, pat = ~//, but that did not work either, neither did a few other things. For example, this doesn’t work either:

println  (/c:abcd/ =~ //).replaceAll("/")

The first set of parenthesis should have returned a Matcher object, and then the replaceAll called on it. Instead, we get:
C:temptest.groovy: 1: unexpected char: 0xFFFF @ line 1, column 47.
cd/ =~ //).replaceAll(“/”)
^

So I looked into the Groovy unit tests to see how they test the replaceAll function. Unit tests are sometimes a great way to learn how to use an API. True, some unit tests can be very obscure and complex.

The unit test I found was used the ${”} GString.

So, example that Compiles:

println "new is: " +  /c:abcd/.replaceAll((/${''}/),"/")
C:temp>groovy test.groovy
new is: c:/a/b/c/d

The command line inline script could then be:

C:temp>groovy -e "println "new is: " +  /c:abcd/.replaceAll(/${''}/,"/")"
new is: c:/a/b/c/d

Ok, so it was not easier in Groovy. Maybe a Groovy expert can make it as easy?

Now, why is it so hard?

Specs
Groovy version: 1.8.1
JVM: 1.6.0_25
OS: Windows 7 64bit

Further Reading

 


Ralph Towner — Tale of Saverio

Generate SHA1 hash for a file

I’m starting to learn Git the DVCS or DSCM tool. For each file that Git tracks it identifies the file (the blob) by a generated SHA-1 hash ID of the file contents.

I wanted to check that hash on a file.
First this is how you list the IDs:

C:\work\workspace\SimpleServer>git ls-tree HEAD
...
100644 blob 536d1f72385998faf4e4c61f4385c40ad1138b3e    .gitignore
...

Here is how you get the hash outside of Git:

Using sha1sum.exe

serverscygwinbinsha1sum.exe workworkspaceSimpleServer.gitignore
6062906b77fc71db2960f2e0ec83704a4d3f403e *workworkspaceSimpleServer.gitignore

Using Ant via Groovy
Here is how its done using Groovy and it’s AntBuilder DSL:

new AntBuilder().checksum (
	file: "C:workworkspaceSimpleServer.gitignore",
	algorithm:"SHA-1"
)

This creates a file,.gitignore.SHA-1, with the contents: 6062906b77fc71db2960f2e0ec83704a4d3f403e

Hmmm.
The sha1sum.exe of cygwin and the Ant sha1 both agree, but Git’s value is different.

Update
Ah, the Git hash of the file is of the blob of the file as Git stores it in the Git structure.

Remove duplicate lines using Groovy

A simple use of sort and uniq can remove duplicate lines. But, what if you need to keep the existing line order or only part of the line contains the duplication?

A simple use of sort and uniq can remove duplicate lines. But, what if you need to keep the existing line order or only part of the line contains the duplication?

Traditional command shell scripting
Using the command shell you can run:

cat test.txt | sort | uniq 

Command shell scripting keeping line order
If you need to maintain the text order you can use more powerful scripting language tools like awk.
At this discussion this script is shown:

awk '!x[$0]++'  

Another, more conventional and perhaps less elegant approach using awk is shown here:

awk '{
if ($0 in stored_lines)
   x=1
else
   print
   stored_lines[$0]=1
}' filein > fileout

Line dedupe where subset of line is duplicated
The above approaches are fine when the duplication is per line. When the dupe is based on a subset of the line or multiple lines per record are involved, it gets more complicated. Then you reach for Perl, Python, Ruby and other scriptable languages.

Duplicate Keys in a Properties file
Recently, I had to identify duplicate keys in a Java properties file. Figuring out how to do it with with command line approach would have taken long since I don’t usually use a shell and I’m on Windows (yes I use cygwin).

Note that Creating a Properties object from the file then writing out the unique entries would have not worked since this would lose the comments in the file.

Groovy solution
This is where Groovy really shines; allowing any Java developer to create little helpers without the ponderous “noise”.
The dupe key detector.

def hist = [:]
def result = ""
new BufferedReader(new InputStreamReader(System.in))
     .eachLine(){ s ->

	result = s
	if(s ==~ /^s*[#!].*/){ // comment?
	    println result
	    return  // continue
	}

	if(s ==~ /.*?[=:].*/){   // property?
		// get everything before the '=' or ':'
		def prop = ((s =~ /(.*?)[=:].*/)[0][1]).trim()
		if(hist.containsKey(prop)){
			result = ("#*** DUPE *** $s")
		}else{
			hist[prop] = ''
		}
	}
	
	println result
	
}

Of course, this is not robust and many complications are not addressed that would require more complex parsing per the Properties spec. One problem is that commented out lines in the file are not treated as such. But, this quick solution should work for simple situations.

An alternative approach by extending the Properties class is shown in post: “Java Properties dupe key detect using subclass”.

Example
Property file with duplicate key entries

one=one
two=two
three=three
two=two
one=one
four=four
   #one=one
!  one=one
one:one
one=une=35
five=five
six=six
four=four
seven=seven

Invocation

type test.txt | groovy DedupeLines.groovy

Resulting output. Note that the duplicates are not removed, just marked. This allows testing before they are permanently removed.

one=one
two=two
three=three
#*** DUPE *** two=two
#*** DUPE *** one=one
four=four
   #one=one
!  one=one
#*** DUPE *** one:one
#*** DUPE *** one=une=35
five=five
six=six
#*** DUPE *** four=four
seven=seven

Large files
How about large files that a map or list cannot contain all the lines?
— One possibility is to use a hash function for each line (or subset) and store that instead.
— Another possibility is to use something similar to external sorting, ie., use external files for interim manipulation.
— Use an embedded database.

Tools

Further reading
Search for “remove duplicate lines”

JSON configuration file format

JSON is a data interchange format. Should it also be used as a configuration file format?

JSON is a data interchange format. Should it also be used as a configuration file format, a JSON-CF?

Overview
Had to write yet another properties file for configuration info. Started to think that maybe there are better alternatives. Wondered about JSON for this.

Requirements
What are requirements of a configuration file format?

  • Simple
  • Human readable
  • Cross platform
  • Multi-language support
  • Unicode support

Looks like JSON has all the right qualities.

If all you want to pass around are atomic values or lists or hashes of atomic values, JSON has many of the advantages of XML: it’s straightforwardly usable over the Internet, supports a wide variety of applications, it’s easy to write programs to process JSON, it has few optional features, it’s human-legible and reasonably clear, its design is formal and concise, JSON documents are easy to create, and it uses Unicode.
— Norman Walsh, Deprecating XML

JSON-CF Limitations

  • Instead of angle brackets as in XML, we have quotation marks everywhere.

What does it need?

  • Inline comments, see for example, json-comments
  • Interpolation (property expansion)
  • Namespaces
  • Inheritance
  • Includes
  • Date value
  • Schema
  • Cascading

Example

{
    "_HEADER":{
            "modified":"1 April 2001",
            "dc:author": "John Doe"
    },
    "logger_root":{
            "level":"NOTSET",
            "handlers":"hand01"
     },
    "logger_parser":{
            "level":"DEBUG",
            "handlers":"hand01",
            "propagate":"1",
            "qualname":"compiler.parser"
    },
    "owner":{
             "name":"John Doe",
             "organization":"Acme Widgets Inc."
     },
     "database":{
             "server":"192.0.2.62",     
             "_comment_server":"use IP address in case network name resolution is not working",
             "port":"143",
             "file":"payroll.dat"
      }
}

Programmatic Access using Groovy

Now we can easily read this file in Java. Using Groovy is much easier, of course. Groovy version 1.8 has built-in JSON support, great blog post on this here.

import groovy.json.*;

def result = new JsonSlurper().
                          parseText(new File("config.json").text)

result.each{ section ->
	println "$sectionn"
}

>groovy readConfig.groovy
Resulting in Groovy style data structure, GRON, (look ma, no quotation marks):

logger_parser={qualname=compiler.parser, level=DEBUG, propagate=1, handlers=hand01}

owner={organization=Acme Widgets Inc., name=John Doe}

_HEADER={dc:author=John Doe, modified=1 April 2001}

database={port=143, file=payroll.dat, server=192.0.2.62, _comment_server=use IP address in case network name resolution is not working}

logger_root={level=NOTSET, handlers=hand01}

In Groovy you can access the data with GPath expressions:
println “server: ” + result.database.server

You can also pretty print JSON, for example:
println JsonOutput.prettyPrint(new File(“config.json”).text)

Summary

Raised the question of the use of JSON as a configuration file format.

What I don’t like is the excess quotation marks. YAML is more attractive in this sense. But, the indentation as structure in YAML, similar to Python, may not be wise in configuration files.

Well, what is the answer, should there be a JSON-CF? I don’t know. A very perceptive caution is given by Dare Obasanjo commenting on use of new techniques in general:

So next time you’re evaluating a technology that is being much hyped by the web development blogosphere, take a look to see whether the fundamental assumptions that led to the creation of the technology actually generalize to your use case.

Updates

  • After writing and posting this I searched the topic and, of course, this is not a new question. Search. I updated the reading list below with some useful links.
  • JSON Activity Streams are an example of how JSON is used in new ways.
  • schema.org types and properties as RDFS in the JSON format: schema.rdfs.org/all.json
  • Just learned that node.js uses the NPM package manager which uses a JSON config file format.
  • Jan 7, 2012: Java JSR 353: Java API for JSON Processing

Further Reading
Apache Avro 1.7.7 Specification Schemas
Analysis of JSON use cases compared to XML
json-schema.org
JSON Validation Roundup
A very simple data file metaformat
GRON
HAL – Hypertext Application Language
NPM configuration file format
XML or YAML for configuration files
Using JSON for Language-independent Configuration Files
INI file
JSON+Comments
Comparison of data serialization formats
Data File Formats, in Art of Unix Programming, Eric Steven Raymond.
ConfigParser – Work with configuration files
JSON
Cascading Configuration Pattern
java.util.Properties
RFC 4627
XML-SW,a skunkworks project by Tim Bray. Brings a bunch of XML complex together into one spec.
YAML
ISO 8601
Learning from our Mistakes: The Failure of OpenID, AtomPub and XML on the Web
Groovy Object Notation (GrON) for Data Interchange
Groovy 1.8 Introduces Groovy to JSON
JSON-LD ‘JSON for Linking Data
JSON Schema

 


” Sacred Place ” , R.Towner / P. Fresu, live in Innsbruck , Part 4

Java's HTTP Server for browser-based Groovy app

Example source code illustrating use of the HTTP server included in Java JDK 1.6. via Groovy to present a browser-based UI to a local application.

Code illustrating use of the HTTP server included in Java JDK 1.6. via Groovy to present a browser-based UI to an app.

Result

The code allows this usage:

main.SimpleServer.serve(0){ s, t, p -&gt;
    // handle the request, response here...
    // now shut down,
    s.stopServer()
}

Listing 1, How it’s used.

This will create a web server using the host “localhost” at an unused port p, and then automatically open the default browser at “http://localhost:p/”. The closure will be the application. Neat. Of course, you would only use this behind a firewall, etc.

The above code, as a I later discovered, is very similar to other frameworks. Here is a new one I just learned about:

vertx.createHttpServer().requestHandler { req -&gt;
    def file = req.uri == &quot;/&quot; ? &quot;index.html&quot; : req.uri
    req.response.sendFile &quot;webroot/$file&quot;
}.listen(8080)
See also: vert.io. “Effortless asynchronous application development for the modern web and enterprise”

See also “Java development 2.0: Ultra-lightweight Java web services with Gretty” link, for an approach using Gretty.

Hmmm, just noticed that this is the ‘look’ of a simple node.js example. No way to dupe node.js of course, it is a low level thing, but can streaming event based programming be done with Groovy? Perhaps with GPars. See this discussion node.groovy?.

5/9/2011: Check out blog post on concurrent websockets. Instead of the Java 1.6 embedded server, this is based on Gretty. See also, Gretty/GridGain/REST/Websockets.

A sample session running the example code and using telnet is:

telnet localhost 21224
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
GET /?reply=24 HTTP/1.1

HTTP/1.1 200 OK
Content-length: 15&lt;/pre&gt;
&lt;h1&gt;Wrong!&lt;/h1&gt;
&lt;pre&gt;Connection closed by foreign host.

The Eclipse console contains:

+++++++ Simple Server ++++++++++++++++++
uri: [/?reply=24]
protocol: [HTTP/1.1]
query: [reply=24]
path: [/]
params: [[reply:[24]]]
-------------------------

Stopping server ... stopped!

The browser will show:

The app as seen on browser

Scenario

You have to supply a GUI for running a local Java application. One example could be the setup of a product build. Builds, though usually executed with Ant, Maven, or Gradle, may still require the user to select some target parameters or build type. Why not just use the browser? It can present a more modern interface and with good design, allow more fault tolerant use then that with command line or multiple prompt boxes.

This approach also allows future remote build server use, since the designed browser UI can be reused. The browser is ubiquitous and creating web pages is relatively easy (though can be a hair pulling nightmare sometimes). And, with the use of AJAX and high-level frameworks, like Dojo, JQuery, or React, browser clients can more easily duplicate the usability of dedicated thick client apps. HTML5 (see HTML Cheatsheet) is also helping to obliterate any remaining reasons for using a thick-client app.

An embedded server as used here, is great when the task is ad hoc, short lived, or single user. In the provided example, once the input is received the server shuts down. For more complex or ubiquitous use a standard server or more powerful embedded server should be used.

Embedded Server

For a local application that accesses system resources and uses the browser as the UI, using an embedded server is the simplest approach. Pure Javascript, Applets, and other means are very complex and may run against configuration issues with Browser security settings and so forth. In the Java world, that would mean using the more popular Tomcat or Jetty servers.

However, Java 1.6 now includes a light-weight HTTP Server. So, the configuration and requirements are more tractable. Nothing more to download and programmatic configuration concerns are minor. Note that the Java HTTP server does not offer many features, one augments this with custom code. For example, parsing of the request query is not present.

That the package of this server is com.sun… is problematic. Will it stay around, become part of the javax packages, etc? Should the JDK even have this built in? According to this post by M. MacMahone

… is that the API and implementation are a fully supported, publicly accessible component of Sun’s implementation of Java SE 6. It does mean however, that the packages are not formally part of the Java SE platform, and are therefore not guaranteed to be available on all other (non Sun) implementations of Java SE 6.

Incidentally, the Groovy language has an import system, Grape, that can also make use of Tomcat or Jetty as transparently as using the JDK embedded server. See the Further Reading below for an example using Groovlets.

This code illustrates

(more for my future reference)
The list below was some of the things the code used and the final code listed here may no longer have them.

  • com.sun.net.httpserver API use.
  • Using JQuery in external js files.
  • How to stop the server.
    • With AJAX post
    • Timeout
    • HTTP context
  • Using ScheduledExecutorService to limit runtime.
  • A console ASCII spinner.
  • Groovy GString use.
  • Launching the default browser.
  • Selecting an unused port.
  • Simplistic state machine configuration.
  • Detecting Java version.
  • AJAX using JQuery.
  • Groovy object construction from script.
  • Use of Closure.
  • Basic authentication
  • Quasi Anonymous class use in Groovy
  • access to resources

Of course, not great example of the above, but … Warning, code is not production ready, etc. There is no real exception handling!

How it works.

TODO: Give some idea what all that code does.

The index.html creates a simple form with three buttons (submit, ping, and end), an input field, and a ‘console’ output area.

submit: send the answer to the server which then gives feedback, correct or wrong. The server then deliberately shuts down.
ping: sends an AJAX request to the ping context which just sends back the time, the response is appended to the console.
end: sends an AJAX request to the ‘stop’ context. The server responds with ‘stopping server …’, then shuts down. All buttons are disabled.

Why Groovy

Groovy is a dynamic JVM based language whose most prominent feature is that it extends the Java syntax to be more usable, i.e., less wordy,. From the example, a method that prints the contents of a map can be defined as:

def showInfo(info){info.each{ k,v -> println "$k = $v" }}

Then invoked as:

showInfo(uri:uri,protocol:protocol,query:query,path:path)

The Groovy In Action book is pretty thorough. Chapter 1 makes a good case, and may still be available here.

Why not Groovy? Well, being dynamic can be rough, especially if it impacts the ability of a IDE to offer the features that come from using Java, like completions, etc. The Eclipse plug-in is getting much better, and I read the IntelliJ IDEA groovy support is top-notch. But, the worlds most popular language, JavaScript, is dynamic, and it hasn’t bothered too many people (maybe end users?).

Source

When I first wrote this, I created a complicated “app” with a bunch of class files and so forth. Then I simplified it. Later I said, yuck. Finally I decided that this should just be a simple one method invocation as shown at the beginning of this post. Consequently, a bunch of stuff in this sample code is guarded by conditionals and it works but sure can be simplified.

While coding I ran into a strange classloader issue see this post. There are a few files in this demo

 
The source code can be downloaded at: here

Listing 4. TestServe.groovy (not a unit test, btw)

//import static main.SimpleServer as SS;   // still won't work!!! GROOVY-4386
import com.sun.net.httpserver.HttpExchange;
import main.SimpleServer

main.SimpleServer.serve(0){
	ss, t, params ->

	if(!(params.size())){
		def path = t.getRequestURI().getRawPath()

		if(path.length()>1){
			ss.sendPage(t,path)
		}else if(path =="/") {
			ss.sendPage(t,"/src/index.html")
		}
	}else{
		def answer = params.get("reply")
		reply = (!answer || answer[0] != '"42"') ? "Wrong!" : "Correct!"

		ss.sendString(t,"</pre>
<h1>$reply</h1>
<pre>")
		ss.stopServer()
	}
}

Listing 5 SimpleServer.groovy

/**
 * File: SimpleServer.groovy
 * Date: 20110314T2125-05:00
 * Author: jbetancourt
 */

package main

import java.io.IOException;
import java.io.OutputStream;
import java.net.Authenticator;
import java.nio.channels.*
import java.nio.*
import java.text.SimpleDateFormat
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit as TU;
import java.util.zip.*;
import com.sun.net.httpserver.Authenticator;
import com.sun.net.httpserver.Authenticator.*;
import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpHandler;
import com.sun.net.httpserver.HttpServer;
import com.sun.net.httpserver.BasicAuthenticator;
import java.util.concurrent.ScheduledExecutorService;
import org.codehaus.groovy.reflection.ReflectionUtils

import edu.stanford.ejalbert.BrowserLauncher;

/**
 * Example of using the Java 1.6 HTTP server com.sun.net.httpserver.
 *
 * @see "http://download.oracle.com/javase/6/docs/jre/api/net/httpserver/
 * spec/com/sun/net/httpserver/package-summary.html"
 *
 * Version 0.02
 *
 * @author jbetancourt
 */
class SimpleServer implements HttpHandler{
	static final STOP_WAIT = 2
	static final HOST = "localhost"
	static final HTTP_PROTOCOL = "http://"
	static final splashText = " Embedded Java 1.6 HTTP server example (ver $version) "
	static final company = " 20110314T2125-05:00 jbetancourt"
	static final UTF_8 = "UTF-8"
	static final version = "0.3"
	static final realmName = "my realm"

	static port =0 // if zero, get unused port
	static url
	static int  counter = 0 // for spinner
	static String basedir   // docbase
	static String scriptDir

	HttpServer server
	static SimpleServer simpleServer
	def SRC_INDEX_HTML = "/src/index.html"
	def serverPropFilePath = "server.properties"
	def props = new Properties()
	def browserLauncher
	def ScheduledExecutorService cancelExec
	def authenticate = false
	def Closure handleParams

	/**
	 * The only thing you really need.
	 * @param port   if 0, unused port will be used
	 * @param closure  handles the request,response
	 * @return
	 */
	static serve(port, Closure closure){
		def ss = new SimpleServer()
		ss.port = port
		ss.handleParams = closure
		ss.serverPropFilePath = ""
		ss.exec(new String[0])
		return ss
	}

	/**
	 * Entry point into the demo.
         * @param args
	 */
	static main(args) {
		splash()
		if(!validJavaVersion()){
			return
		}

		simpleServer = new SimpleServer()
		simpleServer.exec(args)

	} // end main

	/**  */
	def exec(args){
		def basedir = new java.io.File(".").getAbsolutePath()
		configure(args)
		createHTTPServer()
		start()
		println("server started. url=$url,basedir=$basedir,scriptDir=$scriptDir,")

		def spb = props.getProperty("progressBar")
		cancelExec = Executors.newScheduledThreadPool(1)
		keepAlive(false,
				TU.MILLISECONDS.convert(1L, TU.MINUTES))
	}

	/**
	 * Handle the given request/response.
	 *
	 * The first response is the input form.  The subsequent
	 * request evaluates the answer if any, and then the server
	 * is stopped.  Any exception will also stop the server.
	 *
	 */
	@Override
	public void handle(HttpExchange t) throws IOException {
		try{
			def uri = t.getRequestURI()
			def query = uri.getRawQuery()
			def path = uri.getRawPath()
			def params = parseQuery(query)
			showInfo(
				uri:uri,protocol:t.getProtocol(),
				query:query,path:path,params:params)			

			if(handleParams){
				handleParams(this, t, params)
			}else{
				if(query == null){
					if(path.length()>1){
						sendPage(t,path)
					}else if(path =="/") {
						sendPage(t,SRC_INDEX_HTML)
					}
				}
			}			

		}catch(Exception ex){
			ex.printStackTrace()
			stopServer()
			throw ex;
		}
	}

	/**  */
	def showInfo(info){
		println "+++++++ Simple Server ++++++++++++++++++"
		info.each{ k,v ->
			println k + (v ? ': [' + v +']' : ": []")
		}
		println "-------------------------"	

	}

	/**  */
	static splash(){
		println("$splashTextnn$company")
	}

	/**  */
	def configure(args){
		try{
			basedir = new java.io.File(".").getAbsolutePath()
			scriptDir = "$basedir/src/"
			def propFileName = (args.length) >0 ? args[0] :serverPropFilePath
			port  = unusedPort(HOST)
			url = "$HTTP_PROTOCOL$HOST:$port/"

			if(propFileName){
				def getResource = {def resource->
					ReflectionUtils.getCallingClass(0).
							getResourceAsStream(resource)
				}

				InputStream context = getResource(propFileName)
				if(context){
					props.load(context)
					context.close()
				}
			}	

		}catch(Throwable ex){
			handleException(ex)
		}finally {
			//System.exit(1)
		}
	}

	/**  */
	def createHTTPServer(){
		server = HttpServer.create(new InetSocketAddress(port),0);

		props.propertyNames().iterator().each{key ->
			if(key.matches(".*Context$")){
				def conf = props.get(key).split(",")
				def className = conf[0]
				def contextPath = conf[1]? conf[1].trim(): "/"

				def obj = createObjectFromScript(className, this)
				def context = server.createContext(contextPath, obj)

				if(conf.length>2){
					def authClassName = conf[2]
					def ac = createObjectFromScript(authClassName,
							conf.length > 3? conf[3] : realmName)

					context.setAuthenticator(ac)
				}

				def iProp = key + ".initialState"
				if(props.containsKey(iProp)){
					obj.currentState = props.getProperty(iProp)
				}

				iProp = key + ".transitions"
				if(props.containsKey(iProp)){
					obj.setTransitions(props.getProperty(iProp))
				}
			}
		}

		def context = server.createContext("/", this)
		if(authenticate){
			context.setAuthenticator(new MyAuthenticator("my realm"))
		}		

		server.createContext("/stop", [
					handle:{
						println("in handler for stop .. t[" + it + "]")
						sendString(it,"stopping server ....")
						stopServer()
					}
				] as HttpHandler);

		server.createContext("/ping", [
					handle:{ ct ->
						println("pinging ...")
						ping(ct);
					}
				] as HttpHandler);

	} // end createHTTPServer

	/**
	 *
	 * @return
	 */
	static boolean validJavaVersion(){
		def flag = true
		def ver = System.getProperty("java.version");
		if(!ver.contains("1.6")  &amp;&amp; !ver.contains("1.7")){
			println("ERROR *** Requires Java 1.6 or above. Detected: $ver");
			flag = false;
		}

		return flag;
	}

	/**
	 * Send the initial query page.
	 *
	 * @param t the request handler
	 * @param filePath the html file
	 * @return nothing
	 */
	static sendPage(HttpExchange t, String filePath) throws IOException {
		OutputStream os = null;
		try{
			def fPath = new File(basedir + filePath)
			def uri = fPath.toURI()
			Map>map = t.getResponseHeaders()

			def binary = false

			if(filePath.endsWith( ".js")){
				map.set("Content-Type", "text/javascript; charset=UTF-8")
			}else if(filePath.endsWith(".gif")){
				map.set("Content-Type", "image/gif;")
				binary = true
			}else if (filePath.endsWith(".jpg")) {
				map.set("Content-Type", "image/jpeg;")
				binary = true
			}		

			println("sending .... " + fPath)
			if(binary){
				byte[] bytes = readFile(fPath.getPath())
				t.getResponseHeaders().set("Content-Encoding", "gzip")
				t.sendResponseHeaders(HttpURLConnection.HTTP_OK,0)
				GZIPOutputStream gos = new GZIPOutputStream(t.getResponseBody())
				gos.write(bytes)
				gos.finish()
				t.close()
			}else{
				def response = fPath.getText()
				t.sendResponseHeaders(HttpURLConnection.HTTP_ACCEPTED, response.length());
				os = t.getResponseBody();
				os.write(response.getBytes());
				t.close()
			}
		}catch(FileNotFoundException ex){
			println(ex.getMessage())
		}catch(Exception ex){
			ex.printStackTrace()
			if(os){
				println("close output stream ...")
				os.close();
			}
		}
	}

	/**
	 *  Read file into buffer.
	 * @param path
	 * @return
	 */
	static byte[] readFile(String path){
		File file = new File(path)

		FileInputStream inStream = new FileInputStream(file)
		FileChannel inChannel = inStream.getChannel();
		def bb = ByteBuffer.allocate(1024*1024)

		while(true){
			int bytesRead = inChannel.read bb
			if(bytesRead == -1){
				break;
			}
		}

		return bb.array()
	}

	/**
	 * Send the resulting score based on response content.
	 * @param t
	 * @param answer
	 * @return
	 */
	static sendString(t, answer ) throws IOException {
		t.sendResponseHeaders(HttpURLConnection.HTTP_OK, answer.length());
		OutputStream os = t.getResponseBody();
		os.write(answer.getBytes());
		os.close();
	}

	/**
	 * Just followed example at:
	 * @see http://download.oracle.com/javase/6/docs/api/java/util
	 * /concurrent/ScheduledExecutorService.html
	 */
	def keepAlive(showProgressBar, Long maxTime){
		def handleBeep = cancelExec.scheduleAtFixedRate(new Runnable(){
					public void run(){
						if(showProgressBar){
							progressBar()
						}
					}

				}, 1, 4, TU.SECONDS);

		cancelExec.schedule(new Runnable(){
					public void run(){
						println("ncancel beeping")
						handleBeep.cancel(true)
						stopServer()
						System.exit(0)
					}

				},4, TU.MINUTES);
	}

	/**
	 *  In shell console, ASCII spinner gives visual feedback of running server.
	 *  Got idea for approach at
	 *  @see http://blogs.msdn.com/b/brada/archive/2005/06/11/428308.aspx
	 *  But, then took out the use of a switch.  As Charles Moore would say,
	 *  never use conditionals when it can be calculated.
	 */
	static def progressBar(){
		print("b${["/","-","","-"][counter++ % 4]}")
	}

	/**  */
	Object createObjectFromScript( String className, Object... args ) throws Exception {
		println "Creating $className"
		def gcl = new GroovyClassLoader(this.class.classLoader)
		def path = "$scriptDir${className.replace('.','/')}.groovy"
		def cl = gcl.parseClass( new File(path))
		def ni = cl.newInstance(args)
		return ni;
	}

	/**
	 * Get an unused port for server and browser url.
	 * If port is non-zero.
	 * BTW, at shell:
	 *   On windows:  netstat -an
	 *   On linux: netstat -an | grep -i listen
	 *
	 * @see http://stackoverflow.com/questions/573361
	 * /how-can-i-detect-a-free-port-on-the-server-by-code-from-client-side
	 *
	 * I tried simpler ways, but they didn't work. Like using 0 as port.
	 *
	 * @param hostname
	 * @return port number
	 */
	static int unusedPort(String hostname) throws IOException {
		if(port){
			return port
		}

		int minPort = 8000
		int range = 0xFFFF - 8000
		while (true) {
			int port = minPort + (int) (range * Math.random());
			try {
				Socket s = new Socket(hostname, port);
				s.close(); // is this wise?
			} catch (ConnectException e) {
				return port;
			} catch (IOException e) {
				if (e.getMessage().contains("refused")){
					return port;
				}
				throw e;
			}
		}
	}

	/**	 	 */
	def ping(t){
		def now = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss").format(new Date())
		sendString(t, "$now")
	}

	/**  */
	String getServerProperty(key){
		return props.getProperty(key)
	}

	/**  */
	def start(){
		server.start();
		launchBrowser(url)
	}

	/**  */
	def launchBrowser(url){
		new BrowserLauncher().openURLinBrowser(url)
	}

	/**  */
	def stopServer(){
		print("nStopping server ... ")
		((HttpServer)server).stop(2)
		print("stopped!")
		System.exit(0)
	}

	/**  */
	static handleException(Exception ex){
		println("ERROR: ${ex.getMessage()}")
		ex.printStackTrace()
		throw ex
	}

	/**
	 * Parse query into list of values array.
	 *
	 * @see http://stackoverflow.com/questions/1667278/parsing-query-strings-in-java
	 * @param query
	 * @return
	 */
	static Map parseQuery(final String query){
		Map params = new HashMap();

		if(!query || query.length() == 0){
			return params
		}

		def key,val

		for (String param : query.split("&amp;")) {
			String[] pair = param.split("=");

			if(pair.length > 0){
				key = URLDecoder.decode(pair[0], UTF_8);
			}

			val=""
			if(pair.length > 1){
				val = URLDecoder.decode(pair[1], UTF_8);
			}

			List values = params.get(key);
			if (values == null) {
				values = new ArrayList();
				params.put(key, values);
			}
			values.add(!val ? "":val );
		}

		return params;
	}

} // end SimpleServer

// the authenticator class.  Should have been just a simple inner class.
class MyAuthenticator extends BasicAuthenticator {
	/**  */
	public MyAuthenticator(String realm){
		super(realm)
	}

	@Override
	public Authenticator.Result authenticate(HttpExchange t){
		return super.authenticate(t)
	}

	@Override
	public boolean checkCredentials(String username, String password){
		//printf("user=%s, pass=%s%n", username, password)
		return true
	}
} // end MyAuthenticator class

Listing 6 server.properties If there is a setting for a property file it will be loaded and used. This is an example:

# SimpleServer configuration
#
# context = FQCN,path[,authenticator FQCN, domain name]*
controllerContext = main.AppContext,/question
controllerContext.initialState = running
# simple state transitions = state:next_state[,state:next_state]*
controllerContext.transitions = init:running,running:end
# Misc
host=localhost
stop_wait = 2
basedir=
scriptdir=
progressBar=true
browserLauncher=main.BrowserLauncher

Listing 7 index.html The ‘app’ UI:

<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.5/jquery.min.js"></script>
	<!--  <script src="/src/jquery.blockUI.js" type="text/javascript"></script> --><script type="text/javascript" src="/src/scripts.js"></script><script type="text/javascript">// <![CDATA[
 		$(document).ready(function(){ 				//$(document).ajaxStart($.blockUI).ajaxStop($.unblockUI); 				pingServer();	 				stopServer(); 		});
// ]]></script></pre>
<div class="box middle lightGrey">
<div class="middle">Embedded HttpServer Demo <span class="tiny">(com.sun.net.httpserver)</span></div>

<hr />

<form id="form1" class="box internal grey" action="/question" method="get" name="form1">
<input id="mode" type="hidden" name="mode" value="mathInput" />
<table width="100%">
<tbody>
<tr>
<td></td>
</tr>
<tr style="margin: 12px;">
<td><span id="lblReply" class="heavy"> What is "4"+"2"? </span></td>
<td><input id="reply" type="text" name="reply" /></td>
<td><input id="submitButton" title="submit" type="submit" value="submit" /></td>
<td><input id="pingButton" type="button" value="ping" /></td>
<td><input id="stopButton" type="button" value="end" /></td>
</tr>
<tr>
<td></td>
</tr>
</tbody>
</table>
</form></div>
<pre></pre>
<pre> 

Listing 8 scripts.js JQuery stuff:

// file: scrips.js
// author: jbetancourt
//
// External JQuery use.
// technique reference:  http://www.latentmotion.com/separating-jquery-functions-into-external-files-without-selectors/
/* <![CDATA[ */ var pingServer; var stopServer; (function($){ 	pingServer = function(){ 		$('#pingButton').click(function(){ 			$.get('/ping', {mode:"ping"}, 				function(data){ 				    $('#target').append(" "+data) 				},"html") 			    .error(function(response,status,xhr){ 					var msg = "Server does not responsd: "; 					$("#target").html(msg + " " + status + 						(xhr.status ? " xhr.status: [" + xhr.status + "] " 							 + "] xhr.statusText: [" + xhr.statusText + "]" 							 : "") 					); 				}); 		}); 	}; })(jQuery); (function($){ 	stopServer = function(){ 		$('#stopButton').click(function(){ 			$("#submitButton").attr('disabled','disabled'); 			$("#pingButton").attr('disabled','disabled'); 			$("#stopButton").attr('disabled','disabled'); 			$('#target').load('/stop', function(response,status,xhr){ 				if(status == "error"){ 					var msg = "Server does not responsd; "; 					$("#target").html(msg + xhr.status + " " + xhr.statusText); 				} 			}); 		}); 	}; })(jQuery); /* ]]> */

Listing 9 stylesheet.css

.heavy{ font-weight:bolder;}
.box{ border:2px solid black;}
.middle{ margin-left:auto;margin-right:auto;width:60%;}
.internal{ margin:2em;background:#F8F8F8 ;}
.horzCenter { margin-left:auto;margin-right:auto;width:50%;}
.grey { background:#F8F8F8;} .lightGrey { background:#F0F0F0;}
.large{ font-size:xx-large;padding-top:4px; padding-bottom:4px}
body{ width:960px }  .tiny{font-size:small;}

Listing 10 example app AppContext.groovy

/**
 * File: AppContext.groovy
 * Date: 20110320T1952-05:00
 * Author: jbetancourt
 */

package main

import java.text.DateFormat;
import java.text.SimpleDateFormat

import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpHandler;

/**
 *  The context that hosts the application.
 *
 * @author jbetancourt
 *
 */
class AppContext  implements HttpHandler{
	def SimpleServer server;
	def static RUNNING = 'running'
	def currentState
	def transitions = [:]

	/**  */
	AppContext(SimpleServer server){
		this.server = server
		currentState = server.getServerProperty("initialState")
		def statesProperty = server.getServerProperty("transitions")
	}

	/**
	 * Handle the given request/response.
	 *
	 * The first response is the input form.  The subsequent
	 * request evaluates the answer if any, and then the server
	 * is stopped.  Any exception will also stop the server.
	 *
	 */
	@Override
	public void handle(HttpExchange t) throws IOException {
		try{
			def uri = t.getRequestURI()
			def final query = uri.getRawQuery()
			def path = uri.getRawPath()
			def params = server.parseQuery(query)
			showInfo([query:query,uri:uri,path:path,currentState:currentState,params:params])

			def mode = params.get("mode")
			if(atState("running")){
				if(mode){
					if(mode[0] == "mathInput"){
						def reply = params.get("reply")
						evaluateAnswer(t,reply)
						transitionNextState()
					}else if (mode[0]=="ping") {
						server.sendString(t,"huh?")
					}
				}
			}

			if(atState("end")){
				server.stopServer()
			}
		}catch(Exception ex){
			ex.printStackTrace()
			server.stopServer()
			throw ex;
		}
	}

	/**
	* And, send response.
	*/
   def evaluateAnswer(t,answer){
	   def reply
	   try{
		   reply = (answer[0] != '"23"') ? "Wrong!" : "Correct!"
	   }catch(Exception ex){
		   reply = "wrong"
	   }

server.sendString(t,"</pre>
<center>
<h1>$reply</h1>
</center>
<pre>")
   }

   /**  */
	def showInfo(info){
		println "++++++++ AppContext +++++++++++++++++"
		info.each{ k,v ->
			println k + (v ? ': [' + v +']' : ": []")
		}
		println "-------------------------"
	}

	/**	 	 */
	def ping(t){
		def now = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss").format(new Date())
		server.sendString(t, "$now")
	}

	/**  */
	def setTransitions(s){
		s.split(",").each{ tran ->
			def kv = tran.split(":")
			transitions.put(kv[0],kv[1])
		}
	}

	/**   */
	def atState(s){
		return currentState == s
	}

	/**   */
	def transitionNextState(){
		print("currentState[$currentState],")
		def ns = transitions[currentState]
		currentState = (ns ? ns : "end")
		println("  next state=[$currentState] ")
	}

} // end AppContext class

Summary

Shown was a simple example of using the JDK 1.6 embedded HTTP server using the Groovy language.

Updates

  • April 15, 2011: Added some beginnings of code to handle image resources.
  • August 4, 2011: Just saw an old post on same subject. “Groovy++ in action: DSL for embedding HttpServer”. Added it to references. That one creates a DSL to use the embedded HTTP server. Nice.
  • Dec 4, 2011: This post has an example of using Gretty via Groovy: Five Cool Things You Can Do With Groovy Scripts
  • Feb 11, 2012: The internal JDK Http server is being used here: “How ION uses Virgo“.

Required Software

• Oracle Java JDK 1.6.0.24
• Groovy 1.8-rc3
• BrowserLauncher2

Dev Software

• Eclipse: Helios Service Release 2
• Windows 7 Pro, 64bit
• Mercurial 1.8.1
• TortiseHG 2.0.2
• MercurialEclipse 1.0.0
• Groovy-Eclipse 2.1.3

Further Reading

API
com.sun.net.httpserver
http://download.oracle.com/javase/6/docs/jre/api/net/httpserver/spec/com/sun/net/httpserver/package-summary.html

Other Servers
Vert.x
Gretty
“Java development 2.0: Ultra-lightweight Java web services with Gretty”
Tomcat

Jetty

HTTP Implementations in other languages
Python
Lib/http/server.py
CherryPy

Misc
* http://www.redtoad.ca/ataylor/2012/02/simple-servlets-in-groovy/
* Groovy++ in action: DSL for embedding HttpServer
* Java non-blocking servers, and what I expect node.js to do if it is to become mature
* Sun’s secret web server
* Using com.sun.net.httpserver
* node.groovy?
* Ratpack
* AsynchronousChannel
* Mp3d project (which uses com.sun.httpserver)
* Groovy Goodness: Groovlets as Lightweight Servlets
* Making a simple web server in Python.
* Embedded HTTP server
* Comparison of web server software
* Groovlets
* Practically Groovy: MVC programming with Groovy templates
* HTTP server API in Sun’s Java SE 6
* Using Sun Java 6 HttpServer to write a functional HTTP test
* Package com.sun.net.httpserver
* Example for Java HTTP Server API and Ruby WEBrick HTTP Server
* Loop back connection leak
* EchoServer.java
* JDK modules source
* Java Documentation 6.0 JDK Modules
* ServerImpl
* D. Ferrin resource access solution
* Simple Java HttpServer / Handler
* post by hiro345
* com.sun.net.httpserver.HttpServer for comet/cometd
* Separating jQuery Functions into External Files (without selectors!)

Easy stream parsing using Groovy, CVS example

You use every combination of options but that dam command won’t give you what you want? Parse the output stream.

You use every combination of options but that dam command won’t give you what you want?

I faced this last week at work. I had to get a list of my commits to CVS. I tried a bunch of stuff and also searched for a solution. None really worked well. An example of an approach is shown here: “how to search cvs comment history“.

Problem
The root problem is that the output of many tools are not always easily reusable. In this situation (and I’m sure in more modern tools like Subversion, Git, or Mercurial) the output resembles (I took out work related info):

=============================================================================
RCS file: /cvs/A...
Working file: Java So..
head: 1.1
branch:
locks: strict
access list:
keyword substitution: kv
total revisions: 4;     selected revisions: 3
description:
----------------------------
revision 1.1
date: 2011/03/  
filename: Produc...tsA
branches:  1....;
file Produ...
----------------------------
revision 1.1.4.1
date: 201....
filename: ProductsA....;
AS.....
----------------------------
revision 1.1.2.1
date: 2011/0
filename: ProductsA....;
ExampleNightMare - ....
=============================================================================

RCS file: /cvs/Am...
Working file: Java S..
head: 1.1
branch:
locks: strict
access list:
keyword substitution: kv
total revisions: 4;     selected revisions: 3
description:
----------------------------
revision 1.1
date: 2011/03/  
filename: Pro...

This output goes on for thousands of lines! Sure if you use a tool often and dug into its idioms or have a guru near by, you could probably get what you want, but …. (of topic, but why don’t Man pages and other docs give examples for every option?).

Options
There is no need to take out the dragon book and start writing a parser (is ‘parser’ the correct term in this context?), or even create a DSL. If your very familiar with real scripting languages like Python, Perl, or even pure shell utilities, this is easy. If your not, on Windows (and don’t use Powershell), or just as another approach, Groovy is easy to use.

Algorthm
The usual pattern I would imagine is to just just read the input and trigger on a start phrase to indicate a block of interest, then the data is captured when the including line is subsequently detected in the input stream. However, in my situation depicted above, I did the opposite, I got the data I needed, but only printed it out when I got a subsequent trigger phrase, the commit comment.

Sure you could generalize or find some tool that does this, but you’d probably spend more time learning the tool or creating a reusable system that only you need or understand.

Example

// file: ParseCvsLog_1.groovy
// Author: jbetancourt

def inside = false
def workingFile

new BufferedReader(new InputStreamReader(System.in)).eachLine(){ s ->
	
	if(s.startsWith("Working file:")){
		inside = true
		workingFile = s.split("Working file:")[1] // got what I want?
	}
  
	// this indicates that it is.
	def found = s ==~ /.*ExampleNightMare.*/
	if(found){
		println(workingFile)   // send to next pipe
		inside = false
	}  
	
}

Probably not a good example of idiomatic Groovy code, but easy to follow. A Groovy expert could probably do it on one line (I don’t like those smarty one-liners; one week later, you don’t know what you did.).

Usage
This is used as (all one line):

cvs inscrutable bunch of gibberish that doesn't answer question | groovy ParseCvsLog_1.groovy &gt; myChanges.txt

Conclusion
Nothing new in this post, of course. The value of any scripting approach is that it is infinitely adaptable. And, when the scripting language is easy to use, the results could even be reusable. Perl, Python, and Ruby, for example, have great facilities for sharing of snippets and modular code solutions. Groovy and other JVM based languages like Scala are beginning to add this capability to Java environments.

Updates

  • 20110323T1906-5: Cleaned up the sample code a little; don’t want to give the wrong impression.
  • 20110402T1702-5: While looking thru the book “Groovy In Action” noticed that section 13.5.3 Inspecting version control, deals with this subject.

Further Reading

Groovy-Eclipse, compile and run are using different classloaders?

I just manually deleted the compiled Groovy classes, then ran the app. Works. Now I rerun still works. Now add a simple blank line to one of the sources and run. It died. Classloaders are different for other Groovy classes.

In an Eclipse Groovy project, the order of compilation is causing Groovy classloader conflicts.

I just manually deleted the compiled Groovy classes, then ran the app. Works. Now I rerun still works. Now add a simple blank line to one of the sources and run. It died. Manually delete the output class files, works again. 🙁

Strange I don’t have “rebuild automatically” turned on. Also strange, in the project properties, for builders, only the Java builder is listed.

The console output when I get the error:

Caught: groovy.lang.MissingMethodException: No signature of method: static main.SimpleServerSupport.keepAlive() is applicable for argument types: (main.QuestionServer, java.lang.Long) values: [main.QuestionServer@b32ed4, 60000]
Possible solutions: keepAlive(main.QuestionServer, java.lang.Long)

The following classes appear as argument class and as parameter class, but are defined by different class loader:
main.QuestionServer (defined by ‘groovy.lang.GroovyClassLoader$InnerLoader@16921fd’ and ‘org.codehaus.groovy.tools.RootLoader@42e816’)

If one of the method suggestions matches the method you wanted to call,
then check your class loader setup.
at main.QuestionServer.main(QuestionServer.groovy:72)

Haven’t solved this yet. I can’t find any reference to something like it. Maybe I just have a config issue?

I now remember that the plugin adds hooks into the Java compiler. So is the compiler used different when you run the app with “run as Groovy script” then when Java builder executes, that is what the Classloader names would suggest? But, that makes no sense, since at run time, the classloader hierarchy has nothing to do with the compile, or does it, since Groovy precompiles at runtime?

Yes, I uninstalled the Groovy feature and reinstalled.
Running the same program (script) at the command line is fine.

Todo: Post some code that exhibits the issue. The current code is still at the “throw at wall and see what sticks” stage.

Updates

  • 20110328T2010-5: Solved it. I moved a method from a support class, into the main SimpleServer class. This method was using an Executor.newScheduledThreadPool(1) to start a thread to limit the server’s lifetime, it also did a little console spinner to show that there is something running. I don’t see how that is related to classloaders.

Environment
Windows 7 64bit Professional
Eclipse Helios Service Release 2
Groovy-Eclipse Feature 2.1.3.xx-20110317-1200-e36

Java Dev Using Embedded Groovy Console in Eclipse

Embed a script engine and have new ways of analyzing and developing code.

In development, simple but powerful tools to get at the essence of a code source in order to understand, test, and extend it is essential. This is even more important in Test Driven Development (TDD). Eclipse’s Java Debugger is for most situations, powerful enough. Eclipse has an Expressions View available in the debug perspective to execute snippets of code. The Java Development Toolkit (JDT) also has a Scrapbook facility that allows the creation, storage, and running of experimental code. And, of course, all the other features of the debugger are excellent.

However, when you need it, it’s possible to embed a script engine and have new ways of analyzing and developing code. In listing 1, an app shows the use of the ConsoleWaiter class. When the code executes the waiter.run() at line 43, it opens the Groovy Console which allows the use Groovy shell scripting in a GUI frame, see figure 1. When the console is closed the app continues executing.

Listing 1
/*
* File: ExampleApp2.java
* @author jbetancourt
* Date: 20101213T1718-5
*
*/
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

/**
*
* @author jbetancourt
*
*/
public class ExampleApp2 {
   static public String greeting = "Hello world!";
   private static final List<String> nodes;

   static {
        nodes = new
            ArrayList<String>(
                  Arrays.asList("one","two"));
   }

   private String title = "Example 2";

   public String getTitle(){
      return title;
   }

   /**
   * @param args command line args
   */
   public static void main(String[] args) {
      ExampleApp2 app = new ExampleApp2();

      ConsoleWaiter waiter = new ConsoleWaiter(app);

      waiter.setVar("greet", greeting);
      waiter.setVar("nodes", nodes);
      waiter.setVar("title", app.getTitle());
      waiter.run();
      System.out.println("Done!");
   }

}
screen capture of Groovy console
Console screen capture, click to view
Another screen capture, click to view

This is awesome. In one project I had to examine the contents of a Properties object. Did it have an “email” value? I was stumped when using the Eclipse debugger, it did not show all entries in the Map, at the end was “…”. Sure, I could use the Expressions window, but with the Console I could not only do a get(key), but iterate using Groovy style closures and much more.

The magic that enables this is the ConsoleWaiter.groovy class shown below in listing 2 that was written by John Green. Since a Groovy script is a Java class underneath, in Eclipse you can call Groovy from Java easily (some compiler magic).

Listing 2
/**
 * File:  ConsoleWaiter.groovy
 */

import groovy.lang.Binding;
import groovy.ui.Console;

/**
 * Provides a wrapper for the console.
 *
 * Based on source by John Green
 * Adapted from:  http://www.oehive.org/files/ConsoleWaiter.groovy
 * Released under the Eclipse Public License
 * http://www.eclipse.org/legal/epl-v10.html
 *
 * I added methods to allow use from Java.
 *
 * The run() method launches the console and causes this thread
 * to sleep until the console's window is closed.
 * Allows easy interaction with the objects alive at a given
 * point in an application's execution, like in a debugger
 * session.
 *
 * Example 1:
<pre> * new ConsoleWaiter().run()
 *</pre>
 *

 * Example 2:
<pre> * def waiter = new ConsoleWaiter()
 * waiter.console.setVariable("node", node)
 * waiter.run()
 *</pre>
 */
class ConsoleWaiter {
 Console console
 Object source
 boolean done = false;

 /** */
 public ConsoleWaiter(Console inConsole){
    this.console = inConsole
 }

 /** */
 public ConsoleWaiter(Object source){
    console =
    new Console(getClass().classLoader,
    new Binding())
    this.source = source
    console.setVariable("source", source)
 }

 /** */
 public void setVar(String key, Object value){
    console.setVariable(key, value)
 }

 /** 	 */
 public void setVar(String key, List values){
    console.setVariable(key, values)
 }

 /** 	 */
 public void setVar(String key, Object[] values){
    console.setVariable(key, values)
 }

 /** 	 */
 public void run() {
    console.run()
    // I'm a little surprised that this exit() can be private.
    console.frame.windowClosing = this.&amp;exit
    console.frame.windowClosed = this.&amp;exit
    while (!done) {
       sleep 1000
    }
 }

 /** 	 */
 public boolean isDone(){
    return done;
 }

 /** 	 */
 public void exit(EventObject evt = null) {
    done = true
 }

 /** 	 */
 public Console getConsole(){
    return console;
 }
}

Eclipse Integration

The disadvantage of this approach is that you have to put extraneous code inside the tests or target class. Not only is this tedious and slow, what if code is accidentally deployed like this? A better approach is to just set a breakpoint in the code, and then have the ability to open a script console at that breakpoint, in the Java Stack Frame, that has access to the execution context and classpath. Is there an Eclipse add-in that does this? If not, there should be.

Conclusion

Shown was a simple example of embedding a Groovy console in Java code to allow scripting. Of course, this is not a new idea. It was even mentioned in an older JVM scripting language, Beanshell. Note that it is possible to instead of using a GUI console, to use the Groovy shell, InteractiveShell class. In the reading list below this approach is taken to allow remote scripting of a server hosted application.

Updates

  • Oct 10, 2011:
    Interesting tool that could be relevant: YouDebug.
  • March 20, 2012: Not exactly same scenario, but the concept of an embedded script console is found in many products. Jenkins CI Server has one and it uses Groovy. Jenkins Script Console

Further reading

Use Groovy to find missing lines in a file

Compare two files, print missing lines from one file. Also show how to test a Groovy script.

In a project at work I had to compare two files and generate a list of missing lines so that a spreadsheet could be created. Sounds easy and probably is. Example, file A has 1,2,4. File B has 1,2. So File B is missing 4.

So, if args[] has the file paths, a simple approach is:

MissingLines.groovy script

// MissingLines.groovy

def firstList = new File(args[0]).readLines()

new File(args[1]).readLines().each {
    if(!firstList.contains(it)){
        println(it)
    }
}

That was easy! 🙂 Testing it is not so easy. 🙁

Alternative approach using Map

// MissingLines.groovy  using Map

def inFirst = [:]

new File(args[0]).eachLine{
	inFirst.put(it, null)
}

new File(args[1]).eachLine {
	if(!inFirst.containsKey(it)){
		println(it)
	}
}

Still easy …

The script above compares two files and prints the lines that appear in the first but not in the second. It uses a simple string compare. To find out whats missing in each file a better recourse is to use a real file diffing tool. This script reads one of the files into memory, so with very large files may balk. How to test a Groovy script?

From a Computer Science point of view this script is suboptimal of course, all of those calls to contains() must be bad. But, this was a script to solve a single task, so a throwaway.

For some reasons using the available tools I could not do it. I tried different file comparison tools, even KDiff3, diff, some editors, and simple command line utilities. Maybe I was having a bad hair day. It’s not that they won’t show the missing lines, they gave no way to reuse that info. Hint to tool builders, allow the user to re-purpose the result.

Yes, I know this can be done in one line on Linux using the incredible Linux utilities and shell, or a Perl script so tiny it can be etched using atoms. And, since I’m running on Windows, can also be done using Cygwin. I searched and saw how. I’m sure I could have figured it out too, using sort, diff, and all that.

So this post is on how to write a simple script that does this using the Groovy JVM based language and how to test it with JUnit. The testing part for a learning exercise; I really used the compare script. But, why is it so hard to do with common tools, oh well. Groovy offers an easy to use syntax and for Java developers, a comfortable super set of standard Java.

Testing it

Below is a test of this script. Note that this is just a JUnit 3 test. JUnit is embedded in Groovy (that is cool). As I wrote above, this simple script does not require a test class. But, it gave me the opportunity to figure out how to test a Groovy script using JUnit. This illustrates:

  • how to test a simple script
  • how to provide command line args
  • how to capture and test the console output.
  • use of the default Eclipse test invocation project location.
Test data
a.txt             b.txt 
-----             ----- 
Alfa              India
Bravo             Juliet
Charlie           Kilo
Delta             Lima
Echo              Mike
Foxtrot           November
Golf              Oscar
Hotel             Papa
Romeo             Quebec
Sierra            
Tango             
Uniform           
Victor            
Whiskey           
Xray              
Yankee            
Zulu              

MissingLinesTest.groovy class

// MissingLinesTest.groovy
import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import groovy.util.GroovyTestCase;
import groovy.lang.GroovyShell;

/**
 * Unit Test the MissingLines.groovy script.
 * Also illustrates techniques
 * 1.  Capturing console out.
 * 2.  Accessing test data via Eclipse's project folder.
 * 3.  Supplying command line args to script.
 * 
 * The project structure is:
 * .
 * +---.hg
 * +---.settings
 * +---bin
 * +---docs 
 * +---data
 * ---src
 * 
 */
class MissingLinesTest extends GroovyTestCase {
	def basedir = new java.io.File(".").getAbsolutePath()
	def originalOut // console output
	def ByteArrayOutputStream captured // 
	def script = new GroovyShell() //

	@Override
	protected void setUp() throws Exception {
		originalOut = System.out
		captured = new ByteArrayOutputStream()
		System.setOut(new PrintStream(captured)) // can fail
	}
	
	/** first test */
	public final void testAB(){		
		callTest("dataa.txt","datab.txt", 
		["India","Juliet","Kilo","Lima","Mike","November","Oscar","Papa","Quebec"])
	}
	
	/** Second test */
	public final void testBA(){		
		callTest("datab.txt","dataa.txt", 
			["Alfa","Bravo","Charlie","Delta","Echo","Foxtrot",
			"Golf","Hotel","Romeo","Sierra","Tango","Uniform","Victor","Whiskey","Xray","Yankee","Zulu"])
	}
        
	/**  */
	public final void testNothingFellOut(){
		// todo: combining missing from each should give total lines from both files.
	}
	
	/** avoids repeating test code */
	def callTest(String file1Path, String file2Path, List expected){
		script.run(new File(buildPath("srcMissingLines.groovy")), [buildPath(file1Path), buildPath(file2Path)] )
		def list = captured.toString().readLines()
		def flag = list.containsAll(expected)
		assertTrue("Did not find missing strings",flag)
	}	
	
	
	private String buildPath(name){
		return basedir + "" + name		
	}
	
	@Override
	protected void tearDown() throws Exception {
		System.out = originalOut
	}

	
}

Compare the size and complexity of the test class with the target test script. Could it have been smaller too?

Further reading