Why a Repository for Java Dev?

Benefits of a Repository Manager: Part I. Excellent article on why a Repository Manager is crucial for software development process. Not using a Repository Manager is an anti-pattern.

Excellent article on why a Repository Manager is crucial for software development process.  Makes the case that not using a Repository Manager is the cause of many anti-patterns. It could be that the Repository is the next essential besides the VCS in development best practices.

The article is using Maven as a case in point and also it is selling a product (nothing wrong with that) so perhaps one could be a little wary. However there are other dependency managers like Ivy which are used by other Build systems like Gradle available.

I have seen places that do not use a Software Configuration Managment (SCM) Version Control System (VCS).  And, then there are places that use a VCS incorrectly; as this article points out the VCS becomes an ad hoc file store for everything.   I remember one place where our partner gave us access to their SCM to download a project source, and we got everything!  They had application executables, utilities, documents, binaries and other things like their Office apps and other tool chains, which had nothing to do with the project we wanted the source of.  Someone must have accidentally  imported their whole PC into CVS, yikes.

Enter the Repository which, I believe, first became “popular” with the introduction of Maven.  When I tried to introduce use of an internal Repository into a former company I got push back:  “It’s very easy to just put one’s jars and dependent binaries into version control” or  “Who needs that Repository play toy stuff!”  Oh well. In that situation, it was probably the best decision, there is initial complexity in adopting any tool that aims to reduce complexity.

Is just using Mavan or Gradle with an internal Repository as proxy to external ones enough Repository Management (which Sonatype calls stage one: Proxying Remote Repositories) or does one have to use a full blown Repository Manager subsystem? Why does using the internal Repository for one’s own output destination require a Repository Manager (which Sonatype calls ‘stage two’)? The Maven site has this to say:

Aside from the benefits of mediating access to remote repositories, a repository manager also provides something essential to full adoption of Maven. Unless you expect every member of your organization to download and build every single internal project, you will want to provide a mechanism for developers and departments to share both SNAPSHOT and releases for internal project artifacts. A Maven repository manager provides your organization with such a deployment target. Once you install a Maven repository manager, you can start using Maven to deploy snapshots and releases to a custom repository managed by the repository manager. Over time, this central deployment point for internal projects becomes the fabric for collaboration between different development teams. — Repository Management with Maven Repository Managers.

If you don’t think this is important you probably have not been on a project where disasters like xxx.jar was sent to a customer and we don’t know what version it is and who made it. You know, using version numbers as part of binary files would defeat the purpose of using a VCS no?

On a side note: Why did the Java development community develop its own repository system when there were plenty out there such as the application-level package systems used by the Linux community?

Links

Maven Repository Managers for the Enterprise

Why Do I Need a Repository Manager?: link

Maven Repository Manager Feature Matrix: link

Archiva

Artifactory

Nexus

Gradle:  http://www.gradle.org/

Ivy:  http://ant.apache.org/ivy/

Manage dependencies with Ivy

Maven:  http://maven.apache.org/

Ant:  http://ant.apache.org/

Continuous Integration:  http://en.wikipedia.org/wiki/Continuous_integration


Concurrent Code Using Groovy

In a prior post “Java Plain Old Concurrent Object” I wrote about a need for a higher level concurrency support in Java. Here I give the program code that I used to experiment with concurrency and learn a little more.

 
Summary

Presented are a few ways of coding a concurrent program using Java threading and also using a Communicating Sequential Processes (CSP) library.  They are written in the Groovy language. Used as an example is a simple lottery number generation application.

[intro],  [problem],  [Listings],  [simple thread],  [object lock],  
[executor],  [Fork/Join],  [jCSP],  [Actor],  [Declaritive CSP using Spring],  [Links],   [bottom of page]

Intro

In a prior blog post (Java Plain Old Concurrent Object) I wrote about a possible need for a higher level concurrency support in Java.   Here I give the program code that I used in September 2007 to learn more then the rudimentary Java concurrency concepts.

I managed to use CSP and the usual Java concurrent library support, and recently was about to try Actors using the new GPars library.  It was a good way to get a feel for each approach.  Note that it was not enough to become a concurrency expert; that was not a goal; still studying Goetz’s book Java Concurrency In Practice.

My first use of multitasking code was in C++ code I did in 1992:

… technical hurdles with the product, so I put it aside until I could get back to it. One thing that I was proud of was learning a little bit of Fuzzy Logic and using it as the controller. I even wrote a graphical simulator in the C++ language; threading was fun stuff. Watching the fuzzy sets behave like analog surfaces or neural EEG waves gave me the idea for the biomimicry aspects. — An Adaptive Controller Using Fuzzy Memory

Problem

For code example I used lottery number play generator.  Powerball is a U.S. multi-state lottery game.  A Powerball play is composed of 5 white balls in range 1 – 55 and one red ball in the range of 1 – 42.  As you can imagine the odds are extremely high that you could guess the Jackpot set (1 in 195,249,054).  The jackpot gets very large and that is a great enticement to forgo any sanity and “you can’t win if you don’t play” you know. 

If you enjoy the frills, ambiance, and clientele, gambling is a great way to throw away money.   For more on Lottery see: Lottery Links
back to top

Source Code

To simulate real random number generation, the program will continuously generate plays, but only produce the requested numbers of generated sets when the user clicks the enter key.  I’m assuming a user waiting a certain amount of time adds a random element to the PRNG being used.  Does it?  I don’t think so, but that’s another topic.

A few points:

  • The problem could have been solved without using concurrent approach.
  • The code could be optimized and made more Groovy.
  • I used the Eclipse IDE and the Groovy Eclipse Plugin.
  • Example code use at your own risk.  No correctness or quality guaranteed.

A Thread monitor view using VisualVM doesn’t show much, just six threads with four daemon ones.

We use the following batch file, which simply invokes the Main.groovy script.

@rem File: run.cmd
@rem Created 28 Dec 2008 J. Betancourt
call setEnv.cmd
groovy ..\src\main\groovy\net\coxmembers\jbetancourt1\Main.groovy %*

The main script driver is shown in listing 1.  Two arguments are specified, how many dollars to lose and what type of concurrency approach to use.  Executing it with no arguments would give:

c:Users\jbetancourt\Documents\projects\dev\ConcurrentGroovy-1bin>run
usage: usage -t type -d dollars [-h]
 -d,--dollars    How many dollars to lose
 -h,--help       Usage information
 -t,--type       Type of threading

Options for '-t' are:
 "simple"    ----> "Using simple Threads"
 "lock"      ----> "Using lock"
 "executor"  ----> "Using Futures"
 "csp"       ----> "Using JCSP"
 "actor"     ----> "Using Actors"
Example: run -d 5 -t simple

And, here is a sample run:

c:Users\jbetancourt\Documents\projects\dev\ConcurrentGroovy-1bin>run -d 5 -t simple
Setting up games 5 using "simple"
The odds of winning jackpot are: 1 in 146,107,962.00
Enter return key

Number of games generated are: 5
 1:   7 11 14 23 34 PB:  9
 2:  21 22 27 28 55 PB: 21
 3:   2  9 30 37 46 PB: 30
 4:   7 49 50 51 53 PB: 30
 5:   9 18 24 40 42 PB: 32
Good luck!

Success?

Not really.   A funny thing about this code, if you invoke it and quickly hit the return key, it doesn’t work correctly.  Concurrent code can be hard to craft.
Example:

>run -d 5 -t executor
Setting up games 5 using "executor"
Press the Enter key on keyboard

Number of games generated are: 1
 1:  18 20 22 27 41 PB: 25
Good luck!

Listing 1 Main driver class:

package net.cox.members.jbetancourt1;

import org.apache.commons.collections.buffer.CircularFifoBuffer
import org.apache.commons.collections.Buffer
import java.security.SecureRandom
import org.apache.commons.collections.BufferUtils

/**
 * Generate Powerball plays.
 * Required libraries:
 * - Commons CLI
 * - Commons collection
 *
 * A Powerball play is composed of 5 white balls in range 1 - 55
 * and one red ball in the range of 1 - 42
 * This script gets how many games to play, then
 * continually generates this number of games
 * and adds them into a circular buffer.  This is
 * repeated until the user hits a key, which
 * interrupts the generation thread and dumps the results.
 *
 * A circular buffer is used since the results are
 * only needed when the user hits a key,
 * thereby making it "random".  In actuality,
 * since we are using a psuedo-random
 * generator, this is still not true random.
 * There is no way to improve the odds of
 * picking the correct numbers for a lottery game.
 */
 class Main{
 def apps = [
    "simple": [new GuessSimple(),"Using simple Threads"],
    "lock":   [new GuessWithLock(),"Using lock"],
    "executor": [new GuessWithExecutor(),"Using Futures"],
    "csp": [new GuessCSP(),"Using JCSP"],
    "actor" : [new GuessActor(),"Using Actors"]
 ]

 /** Main entry point */
 static main(args){
    def main = new Main()

    def options = main.parseArguments(args)
    if((options != null ) && options.d && options.t){
        println "Setting up game" + (options.d>0? "s": "") +
        " ${options.d} using "${options.t}""
        def dollars = Integer.parseInt(options.d)
        def games = BufferUtils.synchronizedBuffer(
           new CircularFifoBuffer(dollars))

           def threadType = ""
           if(options.t){
               threadType = options.t
           }

           def guess = main.apps[threadType][0]
                guess.generate(dollars,games)
    }

    System.exit(0)
 } // end main

 /****************************************************************************
 * Using CLI builder, parse the command line.
 */
 def OptionAccessor parseArguments(String[] args){
    def cli = new CliBuilder(usage: 'usage -t type -d dollars [-h]')
    cli.h(longOpt: 'help', 'Usage information')
    cli.t(longOpt: 'type', 'Type of threading',args:1)
    cli.d(longOpt: 'dollars','How many dollars to lose',args:1,required:false)
    def options = cli.parse(args)

    if(options == null || options.getOptions().size() == 0 || options.h){
         cli.usage()
         println "nOptions for '-t' are: "
                    apps.each{
         println("t"${it.key}"".
              padRight(12) + " ----> "${it.value[1]}"")
    }
    println "nExample: run -d 5 -t simple"
    return options
  }

  return options
 }
} // end class Main

Listing 2 Base class:

<pre>package net.cox.members.jbetancourt1;

import org.apache.commons.collections.buffer.CircularFifoBuffer
import org.apache.commons.collections.Buffer
import java.security.SecureRandom
import org.apache.commons.collections.BufferUtils

/**
 *
 */
class GuessBase{
 def whiteBalls = [0]
 def redBalls = [0]
 def random = new SecureRandom()
 //each element is an array of 0-4 white ball values and 5 is the red ball value.
 def games //= BufferUtils.synchronizedBuffer(new CircularFifoBuffer(dollars))

 def createBallSets(){
      def i = 0
      for(j in 1..55){
        whiteBalls[i++] = j
      }
      i = 0
      for(j in 1..42){
         redBalls[i++] = j
      }
 }

 /** Pick the set of white balls */
 def pickWhite(){
     def array = []
     array.addAll(whiteBalls)
     Collections.shuffle(array,random)
     return array[0..4]
 }

 /** swap two elements in a list */
 def swapElements(list,i,j){
      def temp = list[i]
      list[i] = list[j]
      list[j] = temp
 }

 /**  Pick the single red 'powerball'. */
 def pickRed(){
      int offset = random.nextInt(redBalls.size())
      return redBalls[offset]
 }

 def showGames(){
     def gamesSorted = games.sort(){a,b -> a[5] <=> b[5]}
     def rownum = 1
     println "Number of games generated are: ${games.size()}"
     for (loop in gamesSorted){
       print ((rownum++ + ": ").padLeft(8))
       for(x in (  loop[0..4]).sort()  ){
         print ((x.toString()).padLeft(3))
       }
       println " PB:" + ((loop[5]).toString()).padLeft(3)
     }
     println "Good luck!"
 }

} // end class GuessBase

Listing 3 Using simple Java Thread class:

package net.cox.members.jbetancourt1;

/**
 * Example that uses a simple 'inline' thread.
 */
class GuessSimple extends GuessBase{

 /** create d guesses and store into games array */
 def synchronized generate(d,games){
   this.games = games
   createBallSets()

   def worker = new Thread();
   worker.setDaemon(true);

   worker.start{ // Groovy allows closure here.
     while(!Thread.currentThread().isInterrupted()){
       def w = pickWhite()
       def r = pickRed()
       games.add((Object)(w+r))
       random.setSeed(System.currentTimeMillis());
     }
   }

  println "The odds of winning jackpot are: 1 in 146,107,962.00nEnter return key"
  new InputStreamReader(System.in).readLine()

  worker.interrupt()
  Thread.sleep(1000) // required!
  showGames()
 }
} // End of GuessSimple.groovy

Listing 4 Using Object lock:


package net.cox.members.jbetancourt1;

/**
 *
 */
class GuessWithLock extends GuessBase{
  def lock = new Object() // for thread synchronization
  /** create d guesses and store into games array */
  def generate(d,games){
  this.games = games
  createBallSets()

  def worker = new Thread(){
     void run(){
       while(!Thread.currentThread().isInterrupted()){
         def w = pickWhite()
         def r = pickRed()
         games.add((Object)(w+r))
         random.setSeed(System.currentTimeMillis());
       }
       synchronized(lock){
          lock.notify()
       }
     }
 }

 worker.start()

 println "The odds of winning jackpot are: 1 in 146,107,962.00"
 println "Enter any key<cr>"
 new InputStreamReader(System.in).readLine()
 worker.interrupt()
 showGames()
 }
}

Listing 5 Using Executor:

/**
* File: GuessWithExecutor.groovy
* Author: Josef Betancourt
* Date:  9/1/2007
*
*/
package net.cox.members.jbetancourt1;

import org.apache.commons.collections.*
import java.util.concurrent.*

/**
 * Generate powerball plays using Executor.
 */
 class GuessWithExecutor extends GuessBase {

 /** create d guesses and store into games array */
 def generate(d,games){
       def dollars = d
       try{
         def exec = Executors.newSingleThreadExecutor()
         def gen = new GameGenerator<Buffer>(dollars,games)
         gen.createBallSets()
         def future = exec.submit(gen)
         println "Press the Enter key on keyboard"
         new InputStreamReader(System.in).readLine()
         exec.shutdownNow()
         waitForTermination(exec,100,10) // 100ms, max of 10 tries

         try{
           def buffer = future.get()
           def exec2 = Executors.newSingleThreadExecutor()
           exec2.execute(new GenerateReport(buffer))
           exec2.shutdown()
         }catch(ExecutionException ignore){
            ignore.printStackTrace()
         }
       }catch(Exception ex){
           ex.printStackTrace()
       }
 }

 /**
 *
 * @param exec the Executor
 * @param time how long to wait
 * @param unit unit of time
 * @param tries how many times to wait
 */
 def waitForTermination(exec,time,tries) throws Exception {
    def count = 0
    while(!exec.awaitTermination(time, TimeUnit.MILLISECONDS)){
        if(count++ >= tries){
           break
        }
    }
 }

 } // end class GuessWithExecutor

 /**
 * A Callable that generates games until interrupted.
 */
 def class GameGenerator extends GuessBase implements Callable {
       def GameGenerator(dollars,games){
                this.games = games
       }

      /**
       * Create d guesses and store into games array.
       * @see java.util.concurrent.Callable#call()
       */
     public Object call() throws Exception {
         while(!Thread.currentThread().isInterrupted()){
             def w = pickWhite()
            def r = pickRed()
            games.add((Object)(w+r))
            random.setSeed(System.currentTimeMillis());
         }
         return games
 }

 }  // end class GameGenerator

 /**
 * Class to render the results
 */
 def class GenerateReport extends GuessBase implements Runnable {
      def GenerateReport(Buffer games){
              this.games = games
      }

      public void run(){
         showGames()
     }
} // end class GenerateReport

Listing 6 Using Fork/Join:


// to do:  approach will be to fork each required game set into its own thread, then join them to get the final result.  Crazy, but a way to learn about Fork/Join and have sample code.

CSP

Some experts are touting the benefits of CSP. Below I use the jCSP library to create a network of processes that collaborate to solve the same problem. Note that some of the people involved in jCSP are adding some CSP support to GPar.

Listing 7 Using jCSP:

/**
* File: CspGuess.groovy
* Author: Josef Betancourt
* Date:  9/1/2007
*
*
*/

package net.cox.members.jbetancourt1;

import jcsp.lang.*;

/**
 * Generate powerball plays using CSP.
 *
 * This version of the example uses the JCSP library.  Note
 * that unlike the other examples, there is an explicit
 * declaration of a 'network'.
 *
 * Required libraries:
 * - JCSP ver. 1.0-rc8 CSP library in Java:
 * <a href="http://www.cs.ukc.ac.uk/projects/ofa/jcsp"></a><br/>
 *
 * Uses PAR and ALT methods presented in:
 * <a href="http://www.soc.napier.ac.uk/publication/op/getpublication/publicationid/9097759">
 * "Groovy Parallel!  A Return to the Spirit of occam?"</a>
 * by Jon KERRIDGE, Ken BARCLAY, and John SAVAGE
 * The School of Computing, Napier University, Edinburgh EH10 5DT in
 * Communicating Process Architectures 2005 13
 * Jan Broenink, Herman Roebbers, Johan Sunter, Peter Welch, and David Wood (Eds.)
 * IOS Press, 2005
 *
 * Perhaps, a better approach is possible using actors?  See for example,
 * <a href="http://gpars.codehaus.org/">Groovy Parallel Systems</a>
 *
 */
class GuessCSP extends GuessBase {
 def generate(dollars,games){
       def suspendChannel = new One2OneChannel()
       def resultChannel = new One2OneChannel()

       // Create the CSP network and run it.
       new PAR(  [
            new GameProcess(suspendChannel,resultChannel,games,dollars),
            new UserInputProcess(suspendChannel),
            new ReportProcess(resultChannel)]
       ).run()
 }

}

/** process that generates games */
def class GameProcess extends GuessBase implements CSProcess{
    def out
    def suspend
    def dollars

    /**  Create the process instance */
    def GameProcess(done,out,games,dollars){
       this.dollars = dollars
       this.suspend = done
       this.out = out
       this.games = games
       createBallSets()
    }

    /** run the process network */
    public void run(){
       def alternative = new ALT([suspend, new Skipper()])
       def STOPPING=0, RUNNING=1
       println "Creating  ${dollars} game${(dollars>0? "s": "")}  "
       def suspended = false
       while (!suspended){
          switch (alternative.priSelect ())
             {
               case STOPPING:
                 suspend.read();
                 if(games.size()>0){
                      suspended = true;
                      out.write(games)
                 }
                 break;
               case RUNNING:
                 def w = pickWhite()
                 def r = pickRed()
                 games.add((Object)(w+r))
                 random.setSeed(System.currentTimeMillis());
                 break
             }
      }
 } // end run()

} // end GameProcess process

/** Process that gets the user input */
def class UserInputProcess implements CSProcess{
     def suspend

     def UserInputProcess(done){
            this.suspend = done
     }

     public void run(){
              println "Enter any key<cr>"
              new InputStreamReader(System.in).readLine()
              suspend.write(new Object())
     }
} // end UserInputProcess process

/**
 * Process to render the results to console
 */
def class ReportProcess extends GuessBase implements CSProcess{
 def input

 def ReportProcess(input){
     this.input = input
 }

 public void run(){
     games = input.read() // get result via channel only
     showGames()
 }
} // end of ReportProcess process

/**
 * PAR A Groovyish Parallel.
 */
def private class PAR extends Parallel {
       PAR(processList){
            super( processList.toArray(new CSProcess[0]))
       }
}

/**
 * ALT A Groovyish Alternative.
 */
def private class ALT extends Alternative {
       ALT (guardList) {
              super( guardList.toArray(new Guard[0]) )
       }
}
// end of CspGuess.groovy

Actor

I started to look into the Actor approach. Below is the start of code to solve the same problem. Even with the few lines of code, attempts to run it give an an exception, and classpath and other easy fixes do not solve it. On my todo list.

Setting up games 5 using "actor"
start
Caught: groovy.lang.MissingMethodException: No signature of method: static groovyx.gpars.actor.Actor.actor() is applicable for argument types: (net.cox.members.jbetancourt1.GuessActor$_generate_closure1) values: [net.cox.members.jbetancourt1.GuessActor$_generate_closure1@1fe571f]
Possible solutions: stop(), wait(), start(), any(), call(java.lang.Object), wait(long)
	at net.cox.members.jbetancourt1.GuessActor.generate(GuessActor.groovy:55)
	at net.cox.members.jbetancourt1.Main.main(Main.groovy:63)

Listing 8 Using Actors:

package net.cox.members.jbetancourt1
//import static groovyx.gpars.actor.Actors.actor
import groovyx.gpars.actor.*
import groovyx.gpars.actor.Actor.*
// @Grab(group='org.codehaus.gpars', module='gpars', version='0.9')
// @GrabResolver(name='jboss', root='http://repository.jboss.org/maven2/')

/**
 */
class GuessActor extends GuessBase  {
	def main = new GuessActor()
	        def dollars
	        def games
	        main.generate(dollars, games)
        }

	def generate(dollars,games){
		println "start"

		def gen = Actor.actor { index ->
			loop {
				react {message ->
					if (message instanceof String) reply "got it"
					else stop()
				}
			}
		}

} // end of class GuessActor

Declarative CSP Using Spring

While experimenting I had an idea that ordinary Java beans could be used in a CSP network with suitable object wrappers or weaving.  That is, an ordinary POJO could be wrapped to appear as a Process, just hook up the bean’s entry point for service into the CSP channel end points.  This would be similar to the way that ordinary beans can be annotated to behave as Active Objects in the GPars Groovy library.

Below is the Spring Framework configuration file that does this for this software example.  The code for the wrapper objects are not included here, too kludgey.

Off topic: The XML language config of Spring is looking long in tooth. Good thing Spring now supports annotations and Java config. Time for a Groovy builder approach too?.

Update:

Listing 9 bean configuration file:

<?xml version="1.0" encoding="UTF-8"?>
<!--
 File: applicationContext.xml
 Spring Framework based bean definitions for JCSP example program.
 Author: Josef Betancourt
 Date:  9/19/2007
-->
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xmlns:lang="http://www.springframework.org/schema/lang"  xmlns:util="http://www.springframework.org/schema/util"
 xsi:schemaLocation="http://www.springframework.org/schema/beans
     http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
     http://www.springframework.org/schema/lang
     http://www.springframework.org/schema/lang/spring-lang-2.0.xsd
     http://www.springframework.org/schema/util
     http://www.springframework.org/schema/util/spring-util-2.0.xsd">

 <!-- ========================================================= -->
 <!-- CSP NETWORK -->
 <!-- ========================================================= -->
 <bean name="network" dependency-check="objects" class="jcsp.lang.Parallel">
     <constructor-arg>
         <list>
              <ref bean="gameProcess" />
              <ref bean="reportProcess" />
              <ref bean="userInputProcess" />
              <ref bean="progressProcess"/>
         </list>
     </constructor-arg>
 </bean>

 <!-- ========================================================= -->
 <!-- CHANNELS -->
 <!-- ========================================================= -->
 <bean name="suspendChannel" class="jcsp.lang.One2OneChannel"/>
 <bean name="outputChannel" class="jcsp.lang.One2OneChannel"/>
 <bean name="progressChannel" class="jcsp.lang.One2OneChannel"/>
 <bean name="skipperChannel" class="jcsp.lang.Skip"/>

 <!-- ========================================================= -->
 <!-- GUARDS  -->
 <!-- ========================================================= -->
 <bean name="alternative" class="jcsp.lang.Alternative" dependency-check="objects">
        <constructor-arg>
            <list>
               <ref bean="suspendChannel" />
               <ref bean="skipperChannel" />
            </list>
        </constructor-arg>
 </bean>

 <!-- ========================================================= -->
 <!-- POJOs -->
 <!-- ========================================================= -->
 <!--  Creates the game generator, but it is setup programmatically
 since it needs the user supplied number of games to play, see GameGenerator.setup(int).
 -->
 <bean name="gameGenerator" class="net.cox.members.jbetancourt.pb.game.GameGenerator">
      <property name="random">
         <bean class="java.security.SecureRandom"/></property>
         <property name="maxRed" value="42" />
         <property name="maxWhite" value="55" />
 </bean>

 <!-- ========================================================= -->
 <!-- PROCESSES -->
 <!-- ========================================================= -->
 <!-- Receives signal from user to accept current generated game sets.
 -->
 <bean name="userInputProcess" init-method="init" class="net.cox.members.jbetancourt.pb.process.JCspSingleShotProxy">
    <property name="outputChannel" ref="suspendChannel" />
    <property name="methodName" value="run"/>
    <property name="target">
            <bean class="net.cox.members.jbetancourt.pb.game.KeyListener" scope="prototype"/>
    </property>
 </bean>

 <!--  Game generator.  Communicates with input and report processes.
 -->
 <bean name="gameProcess" dependency-check="objects"
            class="net.cox.members.jbetancourt1.pb.process.GameProcess">
     <property name="alternative" ref="alternative" />
     <property name="suspendChannelIn" ref="suspendChannel" />
     <property name="reportChannelOut" ref="outputChannel" />
     <property name="progressChannelOut" ref="progressChannel"/>
     <property name="gameGenerator" ref="gameGenerator"/>
     <property name="showInterim" value="false"></property>
 </bean>

 <!-- Output results to console.
 -->
 <bean name="reportProcess" dependency-check="objects"
         class="net.cox.members.jbetancourt1.pb.process.ReportProcess" scope="prototype">
       <property name="input" ref="outputChannel" />
 </bean>

 <!-- Output interim results to console
 -->
 <bean name="progressProcess" dependency-check="objects"
    class="net.cox.members.jbetancourt1.pb.process.ReportProcess" scope="prototype">
      <property name="input" ref="progressChannel" />
 </bean>

</beans>

Listing 10 Java main to load the network:

package net.cox.members.jbetancourt1.pb;

import jcsp.lang.Parallel;
import net.cox.members.jbetancourt1.pb.game.GameGenerator;

import org.apache.commons.cli.*;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;

/**
 * 
 * Main program.
 * 
 * @author JBetancourt
 * 
 */
public class Main {
    private static final String TITLE = 
    "====== Lottery Generator using JCSP and Spring ============";
    private static Parallel network;
    private static ApplicationContext context;

    /**
     * Entry point for running the Guess application. Parses 
     * command line for number of games to play, loads 
     * wired application using Spring Framework
     * based IoC, sets up game array, and then starts 
     * the network of processes. 
     * @param args
     */
    public static void main(String[] args) {
        System.out.println(TITLE);
        int dollars = parseCommandLine(args);
        if ( dollars == 0 ) {
            return;
        }

        context = new ClassPathXmlApplicationContext(
            new String[] { "applicationContext.xml"});
        
        GameGenerator gen = (GameGenerator) context.
        getBean("gameGenerator");
        gen.setup(dollars);

        network = (Parallel) context.getBean("network");
        network.run();
    }

    /**
     */
    private static int parseCommandLine(String[] args) {
        ... ellided ...
    }

}

Further Reading

Lottery Links

Transform XML with Groovy and XMLTask

Presented is an example of Groovy AntBuilder to invoke XMLTask to transform an XML file and test using XMLUnit.

(originally created 15 July 2007)

Abstract

Presented is an example using Groovy’s AntBuilder to invoke XMLTask to transform an XML file.  Also shown is how to unit test using the XMLUnit framework.

Jump to script
Jump to testing

Background

This is my third program used for learning Groovy. How it came about? I had to do an XML transform, change a flag value in an element. After looking at very cryptic sed, grep, awk, bash approaches, I decided that a naive Java program would be good enough to get this out the door ASAP, a simple state machine to traverse the file. This is a Java shop, so if I get run over by a truck, anyone could maintain it. So, I coded a Java class that performed the string replace. Not so unusual, of course, plenty of applications do this, like parse RSS feeds and so forth.

XML transform

The problem with a programmatic string replace of XML is that it is not semantically coherent. One is changing a tree structured data structure using a flat text based approach. Sure it works now, but changes to the data structure may break it. Plus, XML aware tools would provide better coding and testing. Thus, though we shipped my simple transformer I still was thinking about this; maybe next maintenace cycle I could replace it with something more robust. Should I have used XSLT or some other XML based approach, SAX, DOM, StAX, XQuery, JDOM, XOM? Forgot to mention that other preexisting scripts were already manipulating the XML files as text, so transforming and creating XML output that changes the ‘text’ layout could have broken existing processes.

After thinking about it I finally felt that XMLTask would offer the most direct approach. Essentially, this is just a single line, which in Ant would be:

<replace path="//UNIT[@name='widgetB']/VALUE/@value" withText="${newValue}"/>

Software used

  • Groovy 1.5
  • XMLTask 1.15.1
  • XMLUnit 1.1
  • JUnit 3.8.1
  • Ant 1.7
  • Java 1.5
  • GroovyIDE plugin for Eclipse 1.0.1

Requirements

Some requirements that illustrate why the Groovy AntBuilder was chosen:

  • Not change the XML text file except for the specific target node.  (don’t remember if this was true.  7/21/10)
  • Easy to write.
    • Uses mainstream language (Java, but…)
    • Compact, using scripting flavor
    • Plenty of docs on the language
    • Easy to test and debug
  • Command line driven;
  • Cross platform. Used on Windows and Linux, so no Bash or PowerShell scripting
  • No cygwin, just because it is not on each machine
  • Easy to maintain. That ruled out a one-line Perl script or monstrous Bash script with sed, awk, here documents, etc.
  • Reusable
    • Can be copied and used for other tasks. (Note, don’t worry about extensibility).
  • Performance.  Not a concern in this case.

Replace Script

As shown in listing 2, this is amazingly small. Sure there are other frameworks and libraries that are even more powerful, but this is within an existing framework, Ant, so its available as part of larger processes.

Not shown here is the hours I wasted trying to get XMLCatalog support to work so that the DOCTYPE could be handled properly. I’m sure if other XML technologies such as namespaces or Entities were being used, that would have also caused aggravations. Fortunaely, in this case, using XMLTask’s ‘entity’ element got around the custom ‘classpath’ path being used here. I left this in this example in case someone has the same issue. I saw a bunch of forum pleas for help with this.

Listing 2 script

/**
 * Example of using Groovy AntBuilder to invoke XMLTask via Ant API.
 * @author Josef Betancourt
 * @date 20071205T23:14
 */

 def SOURCE='data/storage.xml'
 def DEST='target/storage-new.xml'
 def XPATH='//UNIT[@name='widgetB']/VALUE/@value'
 def VALUE='Y'
 def SYSTEM = 'classpath://some/path/to/dtd/narly.dtd'
 def REMOTE = SYSTEM

 def ant = new AntBuilder()
 ant.sequential{
        path(id: "path") {
            fileset(dir: 'lib') {
                           include(name: "**/*.jar")
                  }
        }

        ant.taskdef(name:'xmltask',classname:
            'com.oopsconsultancy.xmltask.ant.XmlTask',
            classpathref: 'path')

        ant.xmltask(source:SOURCE,dest:DEST,
               expandEntityReferences:false,
               report:false,system:SYSTEM){
                 // don't use DTD spec'd in DOCTYPE
                 entity(remote:REMOTE,local:'')
                 replace(path:XPATH,withText:VALUE)
        }
 }
// end Replace.groovy script

How would have normal Ant have looked like? Not bad. In this case, the Ant script is just as small. The only advantage the Groovy approach would have, other then the avoidance of pointy brackets, is the potential to allow more programmatic manipulations. Of course, I had problems getting XML Catalogs to work in Ant too. Here is my plea to the open source movement: if your not going to document it well, don’t bother. Minimally, there should be examples for all the use cases.

Listing 3 conventional Ant use

<taskdef name="xmltask" classname="com.oopsconsultancy.xmltask.ant.XmlTask"/>

<target name="transform" depends="init">
 <xmltask source="${inputFile}" dest="${outputFile}"
    expandEntityReferences="false" report="false"
      system="classpath://some/path/to/dtd/narly.dtd">
    <entity remote="classpath://some/path/to/dtd/narly.dtd"
         local=""/>
    <replace path="//UNIT[@name='widgetB']/VALUE/@value"
         withText="${newValue}"/>
 </xmltask>
</target>
Using command Line arguments

Instead of hard coding the values, you can get them from the command line with something like the following, which uses Apache Common’s CLI.

Listing 4 command line arg parsing

 // === command line options handling
def cli = new CliBuilder()
cli.s(longOpt: 'source',"source file path",args:1,required:true)
cli.d(longOpt: 'dest',"destination file path",args:1,required:true)
cli.x(longOpt: 'xpath',"xpath expression",args:1,required:true)
cli.v(longOpt: 'value',"replacement string",args:1,required:true)

def options = cli.parse(args)
if(options.s){SOURCE = options.getOptionValue('s')}
if(options.d){ DEST = options.getOptionValue('d')}
if(options.x){ XPATH = options.getOptionValue('x')}
if(options.v){ VALUE = options.getOptionValue('v')}
//=== end command line options handling

Unit Testing

Ok the transform works. How do you know? Eyeballing the resulting XML files? If there are structural changes made to the XML file, will it still work?

Eyeballing the files is not reliable and cannot be automated. One way of testing the changes, is to just use tools such as ‘diff’, and testing the output. I tried that, worked great, until the actual QA testing. There were end-of-line differences in the files depending where you ran the transform and the initial source file. So that would have then required dos2unix or unix2dos to be part of the pipeline. Perhaps there is a switch to diff command to get by this, but I did not find it.

For testing I used JUnit and XMLUnit. I just subclassed Groovy’s JUnit Test subclass called GroovyTestCase.

The test data file is similar to the production data, but much smaller, of course.

Writing the tests was harder then writing the actual script. Fortunately XMLUnit has a very easy to use API.

As usual there were complications. Again, the DOCTYPE was killing the test and I could not get the XMLCatalog support working. My hack was to preprocess the source and output files and filter the DOCTYPE. Notice how small this method is. Straight Java would be pretty wordy.

/** read file, stripping doctype string */
 String filterDocType(path, docString) throws Exception{
           def input = new File(path);
           def writer = new StringWriter()
           input.filterLine(writer){
             it.replaceAll(DOCSTRING,{Object[]s -> ""})
           }
           return writer
 }

The code below is showing the use of command line arguments, whereas the Replace.groovy code was not using them. I left them in here since my original code was using args, this testing shows how to create a command line arg array and invoke a Groovy script.

Listing 5 Replace Unit Test

/**
 * Unit testing the example of using Groovy AntBuilder to invoke XMLTask via Ant API.
 * @author Josef Betancourt
 * @date 20071205T23:14
 *
 * @see http://www.bytemycode.com/snippets/snippet/475/
 * @see http://www.oopsconsultancy.com/software/xmltask
 * @see http://groovy.codehaus.org/Using+Ant+from+Groovy
 *
 */

import org.custommonkey.xmlunit.*

/**
Run the Replace script and make sure only one change is made.
The file paths are relative to TEST_ROOT_PATH passed in
with -DTEST_ROOT_PATH=xxxxx
*/
class ReplaceTest extends GroovyTestCase {
 def DOCSTRING =
 '<!DOCTYPE STORE SYSTEM "classpath://some/path/to/dtd/narly.dtd">'
 def main
 def root
 def lineSeparator = System.getProperty("line.separator")
 def source='data/storage.xml'
 def dest='target/storage-new.xml'
 def args

 void setUp() throws Exception {
         super.setUp()
         main = new Replace()
         root = System.getProperty("TEST_ROOT_PATH")+
                            "/GroovyXMLTaskXMLUnit/"
         XMLUnit.setIgnoreWhitespace(true)
         args = ['-s', source,  '-d', dest, '-x',
             '//UNIT[@name='widgetB']/VALUE/@value', '-v', 'Y']
             as String[]
 }

 void testReplaceScript() throws Exception {
          main.main(args);
          def input =root + source;
          def output = root + +dest;
          validateSingleChange(input, output, "N","Y");
 }

 /** Ensure only one change in XML at XPath */
 void validateSingleChange(final inFile, final outFile,
           value, newValue) throws Exception {
         def diff =
              new DetailedDiff(
                      new Diff(filterDocType(inFile,DOCSTRING),
                               filterDocType(outFile,DOCSTRING)
              )
         )

         def list = diff.getAllDifferences();
         if(list.size()==1){
                  for (dif in list) {
                   def v1 = dif.getControlNodeDetail().getValue();
                   def v2 = dif.getTestNodeDetail().getValue();
                   assertTrue("Failed to change '${value}'
                     to '${newValue}' 
                     using XPath:
                     '${dif.getControlNodeDetail()
                      .getXpathLocation()}'  
                     control value is: $v1 test value is: $v2",
                     (v1.equals(value) &amp;&amp; v2.equals(newValue)));
         }
       }else{
           fail("Expected 1 change, but had ${list.size()} changes.
                diff is: " + diff)
       }
 }

 /** Not really a unit test, but provides confidence in XMLUnit tool use */
 void testNoReplacement() throws Exception {
         main.main(args);
         def input =root + source;
         assertTrue("Should have been the same",
                new DetailedDiff(
                      new Diff(filterDocType(input,DOCSTRING),
                       filterDocType(input,DOCSTRING)
                )
         ).getAllDifferences().size()==0
      );
 }

 /** Another non-unit test, but provides confidence in XMLUnit tool use */
 void testTooManyChanges() throws Exception {
         main.main();
         def t = '(<UNIT name=".*?">)'
         def input =filterDocType(root+source,DOCSTRING)
         def output = input.replaceAll(t,{Object[]it ->
                        it[1]+'<extra>1</extra>'})
         def diff =new DetailedDiff(new Diff(input,output))

         assertTrue("Should have been more then one change",
            diff.getAllDifferences().size()>1)
 }

 /** read file, stripping doctype line */
 String filterDocType(path, docString) throws Exception{
           def input = new File(path);
           def writer = new StringWriter()
           input.filterLine(writer){!(it =~ docString)}
           return writer
 }
}

When run and no failures:


C:homeprojectsdevGroovyXMLTaskXMLUnit&gt;replacetest.cmd ... Time: 2.578
OK (3 tests)

Links

Groovy
XMLTask
XMLUnit
AntBuilder
Ant

Groovy Object Notation (GrON) for Data Interchange

Foregoing the use of JSON as a data interchange when Groovy language applications must interact internally or with other Groovy applications would be, well, groovy.

Summary

Foregoing the use of JSON as a data interchange when Groovy language applications must interact internally or with other Groovy applications would be, well, groovy.

Introduction

JavaScript Object Notation (JSON) is a language-independent data interchange format based on a subset of the JavaScript (ECMA-262 3rd Edition) language. Many languages and libraries now support marshal to and from JSON using external libraries or extensions. This complicates applications since they must rely on more subsystems and there may be a performance penalty to parse or generate an external object notation.

If an application must only interact within a specific language or environment, such as the Java Virtual Machine (JVM), perhaps using the host language’s data structures and syntax will be a simpler approach. Since Groovy (a compiled dynamic language) has built-in script evaluation capabilities, high-level builders (for Domain Specific Language (DSL) creation) , and meta-programming capabilities, it should be possible to parse, create, transmit, or store data structures using the native Groovy data interchange format (GDIF), i.e., based on the native Groovy data structures.

Syntax example

Below is an example JSON data payload.

JSON (JavaScript) syntax:

{"menu": {
  "id": "file",
  "value": "File",
  "popup": {
    "menuitem": [
      {"value": "New", "onclick": "CreateNewDoc()"},
      {"value": "Open", "onclick": "OpenDoc()"},
      {"value": "Close", "onclick": "CloseDoc()"}
    ]
  }
}}

Below is the same data payload; this time using Groovy syntax. Note that there are not too many differences, the most striking is that maps are created using brackets instead of braces. It looks simpler too.

Groovy syntax:

[menu: [
	id: "file",
	value: "File",
	popup: [
	menuitem : [
	 [ value: "New", onclick: "CreateNewDoc()" ],
	 [ value: "Open", onclick: "OpenDoc()" ],
	 [ value: "Close", onclick: "CloseDoc()" ]
     ]
  ]
]]

Code Example

/**
 * File: GrON.groovy
 * Example class to show use of Groovy data interchange format.
 * This is just to show use of Groovy data structure.
 * Actual use of "evaluate()" can introduce a security risk.
 * @sample
 * @author Josef Betancourt
 * @run    groovy GrON.groovy
 *
 * Code below is sample only and is on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
 * or implied.
 * =================================================
*/
 */
class GrON {
    static def message =
    '''[menu:[id:"file", value:"File",
     popup:[menuitem:[[value:"New", onclick:"CreateNewDoc()"],
     [value:"Open", onclick:"OpenDoc()"], [value:"Close",
     onclick:"CloseDoc()"]]]]]'''

    /** script entry point   */
    static main(args) {
       def gron = new GrON()
       // dynamically create object using a String.
       def payload = gron.slurp(this, message)

        // manually create the same POGO.
        def obj = [menu:
	    [  id: "file",
	       value: "File",
                 popup: [
                   menuitem : [
                   [ value: "New", onclick: "CreateNewDoc()" ],
                   [ value: "Open", onclick: "OpenDoc()" ],
                   [ value: "Close", onclick: "CloseDoc()" ]
          	]
           ]
         ]]

         // they should have the same String representation.
         assert(gron.burp(payload) == obj.toString())
    }

/**
 *
 * @param object context
 * @param data payload
 * @return data object
 */
def slurp(object, data){
	def code = "{->${data}}"  // a closure
	def received = new GroovyShell().evaluate(code)
	if(object){
		received.delegate=object
	}
	return received()
}

/**
 *
 * @param data the payload
 * @return data object
 */
def slurp(data){
     def code = "{->${data}}"
     def received = new GroovyShell().evaluate(code)
     return received()
}

/**
 * @param an object
 * @return it's string rep
 */
def burp(data){
     return data ? data.toString() : ""
}

} // end class GrON

Possible IANA Considerations

MIME media type: application/gron.

Type name: application

Subtype name: gron

Encoding considerations: 8bit if UTF-8; binary if UTF-16 or UTF-32

Additional information:

Magic number(s): n/a

File extension: gron.

Macintosh file type code(s): TEXT

API

To be determined.

Security

Would GrON be a security hole? Yes if it is implemented using a simple evaluation of the payload as if it were a script. The example shown above used evaluate() as an example of ingestion of a data object. Incidently, this is the same naive approach used in some JavaScript applications and general not recommended.

For real world use, some kind of parser and generator for object graphs would be needed. The advantage would accrue if the underlying language parser could be reused for this.

Now this begs the question, if Groovy must now support a data parser, why not just use JSON with the existing libraries, like JSON-lib? [May 7, 2014: the built in JSON support].

Is using the Java security system an alternative as one commenter mentioned?

Notes

The idea for GrON was formulated about a year ago. Delayed posting it since I wanted to create direct support for it. However, the task required more time and expertise then I have available at this time.

I was debating what to call it, if anything. Another name I considered was Groovy Data Interchange Format (GDIF), but I decided to follow the JSON name format by just changing the “J” to “G” and the “S” to “r” (emphasizing that Groovy is more then a Scripting language, its an extension of Java).

Updates

10Sept2011: See also this post: “JSON Configuration file format“.

9Feb2011: Looks like Groovy will get built in support for JSON: GEP 7 – JSON Support

I found (May 18, 2010, 11:53 PM) that I’m not the first to suggest this approach. See Groovy Interchange Format? by DeniseH.

Recently (Oct 3, 2010) found this blog post:
Groovy Object Notation ? GrON?

Mar 22, 2011: Groovy 1.8 will have JSON support built in.

Further Reading


Creative Commons License
Groovy Object Notation (GrON) for Data Interchange
by Josef Betancourt is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 Unported License.
Based on a work at wp.me.

Switching between versions of tools

How does one manage the use of multiple versions of a software tool, such as a programming language or framework?

How does one manage the use of multiple versions of a software tool, such as a programming language or framework?

*inux

In a blog post the author, Jeff, shows a way to switch between versions of his development tools, using the Grails framework as an example.   In Linux he creates symlink to the specific version.  He further uses aliases in the ~/.profile to allow quick changes to these symlinks.  Nice post.

Windows 7

Many, I’m sure, have been using that approach, symlinks to the current version.  For example, on Windows 7, I use directory junctions.  Below is a snippet of my java folder.  Notice that all the Groovy versions are in the GroovyVersions subfolder and the file “groovy” will point to one of these.

c:java>dir
Directory of c:java
04/07/2010  05:55 PM    <JUNCTION>     groovy


04/07/2010  05:52 PM    <DIR>          GroovyVersions

The built-in command to create a junction in Windows 7 is:

c:java>mklink /?
Creates a symbolic link.
MKLINK [[/D] | [/H] | [/J]] Link Target

/D      Creates a directory symbolic link.  Default is a file
symbolic link.
/H      Creates a hard link instead of a symbolic link.
/J      Creates a Directory Junction.
Link    specifies the new symbolic link name.
Target  specifies the path (relative or absolute) that the new link
refers to.

So, to create the shown link one would execute:
mklink /J groovy c:javaGroovyVersionsgroovy-1.7.2
or should that be:
mklink /D groovy c:javaGroovyVersionsgroovy-1.7.2
(todo: review the differences between a junction and a softlink)

Changing versions

Changing to a particular would now require delete of the current junction and then creation of a new one to point to the desired version.  However, this is clunky.  Jeff used shell aliases which could just as easily be done in Windows using batch files or PowerShell commands.

Advantages

As the original blog post mentioned, this allows easier control over the environment.  Thus, GROOVY_HOME environment variable would not have to change.  It would remain, GROOVY_HOME=C:javagroovy

Updates

There are already systems to handle versioning of software in an OS. One approach that works on Linux is GVM.

Links

GVM
NTFS Hard Links, Directory Junctions, and Windows Shortcuts

NFTF symbolic link
NTFS junction point
Symbolic Links on MSDN
Using a Virtual Drive in a Project with Multiple Branches

Replace file name suffix using Groovy scripting

Simple high-level filename utility in Groovy using AntBuilder. Contrasts with one-line shell script examples.

This could be written in one line in most scripting languages, including Groovy too. Gets messy and unreadable in my opinion. Plus, its rather complex. For example, I tried this using PowerShell, just to see whats involved, but it didn’t work:

dir c:temp*.THM | foreach {move-item -destination "c:temp"
 -path c:temp$_ $_.Name.replace(".THM",".JPG")}

Sure, I could figure it out or search the web for solution. The way to really do it is just using ‘dir, SED, then xargs, perhaps. And, would be the ideal solution. On Windows, cygwin would supply all the tools.

I think the script below is at a “higher level” and so, maintainable.

/**
 * File: ReplaceSuffix.groovy
 * Replace filename suffixes.
 * Scenario, digital camera creates file.MPG but also file.THM.
 * THM files are JPG, so having the JPG extension is desired.
 * Sample command shell run in Win7 on Sony camera output files.
 * groovy srcReplaceSuffix c:temp THM JPG
 *
 * Code below is sample only and is on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * ========================================================================
 */

if(args.length < 3){
 println("ERROR! requires <path> <extension> <new_extension>")
 return
}

new AntBuilder().
        move( todir:"${args[0]}", 
		  includeemptydirs:"false"){			
			fileset(dir:"${args[0]}"){}
			regexpmapper(
				casesensitive:"false",
				from:"^(.*).${args[1]}$$",
				to:"1.${args[2]}"
            ){}
}
// end file ReplaceSuffix.groovy

In Ant’s XML syntax this would look something like the following, plus there would be many other complications:

<?xml version="1.0" encoding="UTF-8"?>
<project name="ReplaceSuffix" default="run" basedir=".">
  <!-- set global properties for this build -->
  <target name="init">
    <!-- Create the time stamp -->
    <tstamp/>
    <!-- Create the build directory structure used by compile -->
    <mkdir dir="build"/>
  </target>

  <target name="run" depends="init">  
	  <copy todir="build" includeemptydirs="false">
	    <fileset dir="/temp">
	      <exclude name="**/*.bak"/>
	    </fileset>
	    <regexpmapper casesensitive="false" from="^(.*).THM$$" to="1.JPG"/>
	  </copy>  
  </target>
</project>

Links

Use Groovy to create quick change dir scripts

How to use Groovy Templates, AntBuilder, and RegExp to create quick change directory command line batch scripts.


index: abstract, background, implementation, to do, extensions, conclusion, Further Reading

Abstract

Templating in the Groovy language is used to create scripts that provide a quick change directory utility at the command line based on a database of key to path assignments.

Background

On a prior job, I was often using the command line shell on Windows, Linux, and Unix.  One thing that I found inefficient was changing directory locations, the “cd” command.  When one needs to change to deeply nested paths this can get tedious.  Sure, tab completion (available on Windows, *nix) helps and one could also create aliases or use other techniques.  I guess this is enough of an issue that there are many utilities to make “cd” use “better”.

Recently, again I needed to do a lot of change directories in a command shell.  This time I looked at those projects that offer tools to do this.  I tried one and it didn’t work on Windows 7.  It ran, created a database of paths, but perhaps it does not handle file junctions in Windows, it could not locate a simple path.  It was time to revisit my previous solution to this.

At another prior job, on Windows I created a simple batch file, go.cmd, that accepted one argument that mapped to a directory path.  The script simply did a cd to that path.  The advantage with this method is that any string could be used for the path, whereas many tools do a search for a matching path. Thus, I could set foo=very/long/path and then just execute ‘go foo’. I wouldn’t have to find “path” or be presented with a list of partial matches.

I could just reuse that approach.  One drawback though, was that one had to edit a batch file to add or remove a new path target.  Not good.  If a script is working, changing it would require full use of a software development lifecycle. Yes, scripts should be treated with the same care any code gets. See for example, “Testing Matters, even with shell scripts”.

So, I wanted to do the same thing:  A utility that would take a key string, find the corresponding path in a database, and perform a cd.  This utility had to work on different OS’s and it had to use a high-level language.  Java would be adequate (had reasons for not using Perl, Python, PowerShell, etc), however there are issues with attempting to change directory from a JVM instance.  But, we could use “source code generation”, i.e. generate the required batch files.  This is a standard practice, even used as part of install systems.  Further, to make this easy, we could use the Groovy dynamic programming language that runs on the JVM.

Implementation

The Simple Approach

Temporary batch creation and use

If we store the desired key to path mapping in a Java Properties file. We can look up the key specified then generate a batch file that does the required CD. However, this process must itself be driven by a batch file.

Example run:

go.cmd sync
c:\batch\sync>

File go.cmd the driver script:

@echo off
set OLDDIR=%CD%
call groovy Go.groovy %*
call goto.cmd
del %OLDDIR%goto.cmd

File: goMappings.txt, the key to path mappings:

	java : c:\java
	projects : c:\Users\jbetancourt\Documents\projects
	sync : c:\batch\sync

Now the Groovy script is simply:

class Go {
	static main(args) {
		if(args.length != 1){
			return // TODO: better handling
		}

		def db = new Properties()
		db.load(new FileReader(new File("goMappings.txt")))
		def f = new File("Goto.cmd")
		f.write("cd " + db.get(args[0]))
	}
}

One neat we could add here is that we could use a regexp to get the closest match, and if there are multiple give the user the option to pick a specific path. We could even search the stored paths in addition to the tag names.

A Complex Approach

Using a command pattern

As shown in listing one (oops, some lines are cut off in this view), the solution is simple. Download source

Update 10/1/2012: Made a version that uses external file for list of target folders to change to.

Listing 1

[expand title=”Listing 1, cd batch creator”]

/**
 * GoGenerator.groovy
 * Copyright  2010 Josef Betancourt 20100312-17:47
 *
 *  Licensed under the Apache License, Version 2.0 (the "License");
 *  you may not use this file except in compliance with the License.
 *  You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 *  Unless required by applicable law or agreed to in writing, software
 *  distributed under the License is distributed on an "AS IS" BASIS,
 *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 *  See the License for the specific language governing permissions and
 *  limitations under the License.
 *
 */
package batch

/*@Grapes([
@Grab(group='ch.qos.logback', module='logback-core', version='0.9.18'),
@Grab(group='ch.qos.logback', module='logback-classic', version='0.9.18'),
@Grab(group='org.slf4j', module='slf4j-api', version='1.5.10')])

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
*/

/**
 * Command line utility class to generate an OS specific
 * quick change directory script file.
 *
 * Syntax: GoGenerator [&lt;file name&gt;]
 * If target file name is not specified, output goes to std out.
 * Example: groovy GoGenerator.groovy
 * Example: groovy GoGenerator.groovy go.cmd
 *
 * Other utilities of this type are:
 *   Quick Change Directory: http://www.stevemiller.net/qcd/
 *   WCD:  http://www.xs4all.nl/~waterlan/
 *   List of directory changers:  http://www.xs4all.nl/~waterlan/other.htm
 *
 * Currently only Windows builder is implemented below.
 * Tested on Groovy 1.7.0, java version "1.6.0_18", Windows 7
 *
 * On Ubuntu:
 * /development$ sudo groovy -cp ./slf4j-api-1.5.11.jar:
 *    ./slf4j-simple-1.5.11.jar:./ant.jar:./ant-launcher.jar:.
 *    GoGenerator.groovy go.sh
 *
 * @author jbetancourt
 */
class GoGenerator {
	// Map of the location targets for the script utility.
	def static winDb = [
		"java" : "c:java",
		"work" : "Usersjbetancourtwork",
		"projects" : "c:UsersjbetancourtDocumentsprojects",
		"concurrent" : "c:UsersjbetancourtDocumentsprojectsdevConcurrentGroovy-1",
		"sync" : "c:batchsync"
	]

	def static linuxDb = [:]

	/**
	 * main entry point
	 * @param args
	 */
	static main(args) {
		def fileName
		if(args.length &gt; 0){
			fileName = args[0]
		}

		def builder = createBuilder(fileName)
		builder.generate()
	}

	/**
	 * factory method to create an OS specific go builder.
	 *
	 * @param db map of folders
	 * @param fileName target file name or null
	 * @return the builder object
	 */
	static createBuilder(fileName){
		def ant = new AntBuilder()
		ant.condition(property:"winOS"){
			os(family:"windows")
		}
		ant.condition(property:"linuxOS"){
			os(family:"unix")
		}

		def builder
		// get first property entry whose key ends in "OS"
		def entry = ant.project.getProperties().find{ it.key ==~ /.*OS$/}

		switch(entry ? entry.key : "xyz"){
		  case 'winOS':
			builder = new WinGoBuilder(winDb, fileName)
			break
		  case 'linuxOS':
			builder = new LinuxGoBuilder(linuxDb, fileName)
			break
		  case 'xyz' :
			builder = new ErrorGoBuilder()
			break
		}

		return builder
	}

} // end class GoGenerator

/**
 * Creates go scripts.
 * Superclass of OS specific builders
 *
 * @author jbetancourt
 */
private class GoBuilder {
    def script = '''echo TODO!''' // template
    def db  // stores the data binding
    def fileName  // the target script file to create

    /**
     * Create the target script file or if
     * no filename send to stdout
     */
    def generate(){
		def engine = new groovy.text.SimpleTemplateEngine()
		def template = engine.createTemplate(script)
		def result = template.make(["db":db])
		if(fileName){
			File file = new File(fileName)
			file.write(result.toString())
		}else{
			println result
		}
    }
} // end class GoBuilder

/**
 * Creates an error message output builder.
 *
 * @author jbetancourt
 */
private class ErrorGoBuilder extends GoBuilder {
    def os = System.getProperty("os.name")
} // end class ErrorGoBuilder

/**
 * Creates a Windows specific go script.
 *
 * @author jbetancourt
 *
 */
private class WinGoBuilder extends GoBuilder  {
	    def winScript =
'''
@ECHO OFF
rem Go.bat
rem DO NOT EDIT THIS FILE.  It is generated using GoGenerator.groovy script.
rem purpose:   aid rapid directory setting at command line
rem example:  go full
rem if "%OS%"=="Windows_NT" setlocal
if "%1."=="." goto ERROR
&lt;% db.each { %&gt;
if NOT %1==$it.key goto ${it.key.toUpperCase()}-END
&lt;% if(it.value ==~ /^S:.*/) {
print(it.value.substring(0,1) + ":")
}%&gt;
cd $it.value
goto FINISH
: ${it.key.toUpperCase()}-END
&lt;% } %&gt;
:ERROR
echo GO.BAT: Huh?  [%1] Syntax is: go keyword
echo Didn't locate keyword: [%1]
echo Targets are
&lt;% db.each { %&gt;
echo &lt;% print("t" + it.key + "=" + it.value) %&gt;
echo -------------------------------------
&lt;% } %&gt;
:FINISH
rem if "%OS%"=="Windows_NT" endlocal
'''
	    /**
	     * Constructor
	     */
	    WinGoBuilder(db, fileName){
	    	this.db = db
	    	this.fileName = fileName
	    	script = winScript
	    }

} // end class WinGoBuilder

/**
 * Creates a Linux specific go script builder
 *
 * @author jbetancourt
 */
private class LinuxGoBuilder  extends GoBuilder {
	def linuxScript = ''''''

    /** Constructor */
    LinuxGoBuilder(fileName){
    	this.fileName = fileName
    }

} // end class LinuxGoBuilder

// end of source GoGenerator.groovy

[/expand]

The code illustrates the use of two features of Groovy: Templates and the AntBuilder.  The AntBuilder allows easy reuse of Ant tasks.  The Ant Condition task which has an OS detection method is reused here.  Why reinvent OS detection code?  The code,

 

def ant = new AntBuilder()
ant.condition(property:"winOS"){
	os(family:"windows")
}

def builder
def entry = ant.project.getProperties().find{ it.key ==~ /.*OS$/}

switch(entry.key){
 case 'winOS':
	builder = new WinGoBuilder(db, fileName)
	break
}

return builder

creates an Ant task that sets a new Ant property, “winOS”, if the OS family is windows.  Later, if this property is found, using a regular expression, a Windows specific builder is created passing in the map of keyword to folder pairs.  Notice that a “switch” in Groovy can use more then just ints as in Java.

The builder is more of a builder pattern, not the more powerful Groovy Builder concept.  The builder creates a data binding and using the stored Groovy Template, creates the target script text which is then stored in a target file or printed to console if no file was specified.  A map is used to store the keyword to directory path pairs.  An alternative was to store this data in an external properties file, of course.  For the intended purpose, that was not important.

The template format is similar to many other template languages such JSP or ASP.  Of course, most systems allow templating; even Ant has low level support for token replacement using filterchains. Within the template, Groovy code is embedded to iterate over the db map:

<% db.each { %>
if NOT %1==$it.key goto ${it.key.toUpperCase()}-END
<% if(it.value ==~ /^S:.*/) {
print(it.value.substring(0,1) + ":")
}%>
cd $it.value
goto FINISH
: ${it.key.toUpperCase()}-END
<% } %>

This section of code ultimately produces something like this, for Windows:
if NOT %1==java goto JAVA-END
c:
cd c:java
goto FINISH
: JAVA-END

Not expert level batch programming, but effective.

To Do

  • Since this code was written using the Universal Admin Development Model, it is not Unit test friendly. Rewrite so that unit tests can be created.
  • The DB map being used is at the class level. This means it is shared by all builders. Not very useful since paths on each OS not only would have different formats but would not be the same destinations.
  • I still have not figured out how to include backslashes in a template. nor \ work. Good thing I didn’t need them yet.
  • The switch should be within a conditional. Using a trick to avoid a match with switch(entry ? entry.key : “xyz”) still forces the switch to do a lookup.

Extensions

One is tempted to embellish a simple utility, like adding a GUI for managing the resources. I resisted that. However, on Linux I also used two other scripts that I created. These allowed me to traverse a dir hierarchy, either up or down. For example, to go up the tree 4 levels, cdu 4. To go down, one indicates the destination folder, cdd drivers. They were implemented using Linux tools like “find”. I wonder if they could be added to the go generator? That would be complex using DOS batch files. A better approach is to generate Powershell scripts.

Conclusion


Shown was an approach for the generation of batch file scripts using Groovy language. The implementation illustrated the use of Groovy Templates, AntBuilder, Regexp, and powerful switch statement.
This was a simple exercise in Groovy scripting and on the whole was successful. The code has some issues, such as embedded data, and the OS detection and builder creation could be improved.

 

Updates

July 25, 2011: Did Groovy 1.8* change and now requires an external Ant in the classpath? I had to change my quick change batch file and my Groovy script did not work until I included ant.jar and ant-Launcher.jar.

Top

Further Reading

Minar, Igor, “Testing matters, even with shell scripts”, link

Groovy, link

“Builder Pattern”, link

A. Glover, S. Davis, “Practically Groovy: MVC programming with Groovy templates”, 15 Feb 2005, accessed on 14 Mar 2010 at link

PowerShell Team, “The Admin Development Model and Send-Snippet”, [Weblog entry.] Windows PowerShell Blog. 1 Jan 2007. (link). 14 Mar 2010.

“Using Ant from Groovy”, accessed on 14 Mar 2010 at link

“Groovy Templates”, accessed on 14 Mar 2010 at link

PowerShell, link

“Code generation done right”, link

Why use an AOP language when other languages can now do Aspects?

Interesting research, JOT: Journal of Object Technology – How AspectJ is Used: An Analysis of Eleven AspectJ Programs: Sven Apel, on how an AOP language, specifically AspectJ, is used in a few sample projects. Was it useful, required? The jury is out.

Interesting research,  JOT: Journal of Object Technology – How AspectJ is Used: An Analysis of Eleven AspectJ Programs: Sven Apel, on how an AOP language, specifically AspectJ, is used in a few sample projects.  Was it useful, required?  The jury is out.

Interestingly the author correctly mentions that some languages now have rudimentary support for handling of basic crosscutting concerns.   And, since the analyzed projects did not really need advanced AOP techniques,

“numbers indicate that languages that provide only basic crosscutting mechanisms are appropriate to implement a large extent of the analyzed programs.”  

Note that the study does not apply to container based AOP as used in frameworks such as JBossAOP and SpringAOP.  Several Dynamic Languages use a Metaobject Protocol.

I used AspectJ for some utility and diagnostic help.  The language is fine and the concepts are approachable.  It’s the configuration, tools, and environment hassles that put a damper on things.   That’s why frameworks and tool support like that found in Eclipse are critical.

Updates
9/11/2010:
Adding Logging around all of the Methods of a Class with Groovy“, Ted Naleid. http://naleid.com/blog/2010/09/11/adding-logging-around-all-of-the-methods-of-a-class-with-groovy/

Further Reading

JOT: Journal of Object Technology – How AspectJ is Used: An Analysis of Eleven AspectJ Programs, Sven Apel

McClean, John, “Painless AOP with Groovy“, http://www.infoq.com/articles/aop-with-groovy

Meta-object Protocol, http://en.wikipedia.org/wiki/Meta-object_protocol

Dynamic Programming Language, http://en.wikipedia.org/wiki/Dynamic_language

AspectJ, http://www.eclipse.org/aspectj/

AJDT, http://www.eclipse.org/ajdt/