Created a web app to generate a report from the version control repository, Apache Subversion™. Similar approach is possible targeting a different repository, like Git.
TL;DR
Someone said a process we follow could not be automated. I took that as a challenge and created a proof of concept (POC) tool.
The final GUI using ReactJS is relatively complex: five data tables that hide/show/expand/collapse. Searches on those tables, sorting, navigation links, help page, Ajax requests to access Subversion repo data, export to CSV, report generation, and client side error handling. It started as just a GUI to a report, but since it was easy, added more features: Zawinski’s law.
To top it off, the app had to automatically invoke the default workflow or no one would use it.
Result?
1. It is a complex disaster that works. And works surprisingly fast. Using ReactJS and Flux made it into a fast elegant (?) disaster that works … kind of.
2. The app served as an example of a SPA in the our dev group. But, mostly to try out the ReactiveJS approach.
3. My gut feel is that there are some fundamental problems in the client side MV* approach which leads to control flow spaghetti (a future blog post).
Since the time I wrote that app I have noticed a mild push back on React that point out the hidden complexities. There are now new ideas and frameworks, like Redux or Cycle.js. Most recently, to tackle the action-coordination, there is much digital ink written on Redux Sagas, for example: “Managing Side Effects In React + Redux Using Sagas“.
Note, though there are critiques of the ReactJS approach or implementation, this does not imply that React was not a real breakthrough in front end development.
Report generation
Creating simple reports from a version control repository can be accomplished with command line tools or querying XML output from SVN log commands. In this case generating the criteria to generate the report was the hard part. Details are not relevant here: this web app would remove a lot of manual bookkeeping tasks that our group currently must follow due to a lot of branch merging and use of reports for error tracking, verification, and traceability. Yup, long ago legacy Standard Operating Procedures (SOP) of an IT shop.
Architecture Server
A simple Java web app was created and deployed to a Tomcat server. A Java Servlet was used at the server to receive and send JSON data to the browser based client. This server communicates with the version control repository server.
Client
The browser is the client container with ReactJS as the View layer and Flux (implemented in the McFly library) as the framework client implementation. Dojo was used as the JavaScript tools library. Dojo supplied the Promise and other cross-browser capabilities. Why Dojo? That is already in use here. If we were using JQuery, that is what I would use.
Local application service
Performance note: Since the repo query and processing occurs at the server, multiple developers accessing the service would have a performance impact. A future effort is to deploy this as an runnable Jar application (Spring Boot?) that starts an embedded app server, like Tomcat or Jetty, at the developer’s workstation. The browser would still be used as the client.
Repository Query
Some options to generate SVN reports:
1. Use a high level library to access SVN information.
2. Export SVN info to a database, SQL or NoSQL.
3. Use an OS or commercial SVN report generator.
4. Use command line XML output option to create a navigational document object model (DOM)
5. Use SVN command line to capture log output, and apply a pipeline of Linux utilities.
This was a ‘skunkworks’ project to determine if some automation of a manual process could be done and most importantly, if doable, would the resulting tool be used? The first option, seemed easiest, and was chosen. The repo was accessed with the SvnKit Java library. (For Java access to a Git repo, JGit is available).
The process approach was to generate and traverse a Log collection. A simple rule engine was executed (just a series of nested conditionals) to determine what to add from the associated Revision objects.
This seemed like a workable idea until a requirement was requested after the POC was completed: instead of listing a particular source file once per report, show multiple times per each developer who made a commit to it. An easy change if this were implemented as an SVN log query sent to a pipe of scripts. However, with the design this required going into the nuts and bolts of the “rule engine” to add support for filtering, and further changes to the model.
Yup, a POC solution can be a big ball of mud, and unfortunately can be around a long time. Incidentally, this happened with Jenkins CI; where I …
Very recently a flaw in the design will force a revisit of the algorithm again. Instead of making the ‘rule engine’ more powerful, an alternative approach is to start from a Diff collection. The diff result would be used to navigate the Log collection. A similar approach is shown here: http://www.wandisco.com/svnforum/forum/opensource-subversion-forums/general-setup-and-troubleshooting/6238-svn-log-without-mergeinfo-changes?p=36934#post36934
But, already a problem was found with diff output. There is no command line or Java library support for pruning of deleted folders. For example, if a/b/c is a hierarchy, and you delete b, c is also deleted. Now if you generate a diff, the output would contain delete entries for: a/b and a/b/c. What was needed was just a/b. Simple, you say. Sure, but this information is a OOP object graph, so can be complicated.
Perhaps revisit the alternative approaches, like export to database? Not sure if this really would have simplified things, but instead just change where the complexity was located. Is the complexity of a software solution a constant?
Other systems take this export approach. One system I saw years ago, exports the version control history (it was CVS) into an external SQL database and then used queries to provide required views.
Client Single-Page Application
What to use as the browser client technology? From past experience, I did not want go down the path of using event handlers all over the place and complex imperative DOM updates.
Anyway, React seemed interesting and had a shorter learning curve. I looked at Angular, but it seemed to be the epitome of embedding the developer into the product (future blog post on the application development self-deception).
A few ReactJS components were created:
BranchSelect
CommentLines
ControlPanel
DiffTable
ErrorPanel
ExclusionRow
ExclusionTable
FilesRow
FilesTable
ManifestRow
ManifestTable
ProgramInfo
ProjectPanel
RevisionRow
RevisionTable
ViewController
Lessons Learned
This project progressed very quickly. React seemed very easy. But, that was only temporary. Until you understand a library or a paradigm, start with a smaller application. Really understand it. Of course, these too can fool you. For example, when this app first loads, I had to invoke the most likely use-case. There was a endless challenge of chicken/egg model flow disasters. Solved it, but can’t understand how I did it. Somehow I tamed the React flow callbacks. Or this is just a lull and will blow up in as a result of an unforeseen user interaction.
Next SPA project?
My next app will probably use Redux as the Flux framework. Or may leave Reactjs and go directly with Cycle.js which is looking very good in terms of ‘principles’ or conceptual flow and is truly Reactive, based on a ReactiveX library: RxJS.
When you create a Promise you can chain asynchronous behavior by using the then(success,fail) method. The success and fail are callback functions invoked appropriately when the promise is resolved or rejected. Note that since ‘then(…)’ returns a promise, this technique is called promise chaining: p.then(…).then(….).
What if in complex application you already have promises objects and want to create promise chains linking them to each other? How would you do this with ‘then’ methods? Amazingly, in most examples of using Promises on the web this simple scenario is never presented directly. Granted it doesn’t *seem* to be relevant.
Here is the source of a simple demo based on one of the Dojo Deferred examples. It is ‘normal’ promise chaining. I’m using this as the basis for the investigation presented here. The output displayed on page should be: completed with value: 75
The actual chaining ‘stitching’ is setupTrans().then(function(x){return action()}).then(outputValue);
<!DOCTYPE html>
<html >
<head>
<script src="http://ajax.googleapis.com/ajax/libs/dojo/1.10.0/dojo/dojo.js" data-dojo-config="async:false"></script>
<script>
require(["dojo/when", "dojo/Deferred", "dojo/dom", "dojo/on", "dojo/domReady!"],function(when, Deferred, dom, on){
var balance = 100;
function asyncProcess(x){
var deferred = new Deferred();
setTimeout(function(){
if(x === 'plus' || x === 'minus'){
x === 'plus' ? (balance += 25) : (balance -= 25);
}
deferred.resolve(balance);
}, 1000);
return deferred.promise;
}
function setupTrans(){
return asyncProcess('z');
}
function debitAccount(){
return asyncProcess('minus');
}
function creditAccount(){
return asyncProcess('plus');
}
function outputValue(value){
dom.byId("output").innerHTML += "<br/>completed with value: " + value;
}
on(dom.byId("startButton"), "click", function(){
var debit = true;
var action = debit ? debitAccount : creditAccount;
setupTrans()
.then(function(x){return action()})
.then(outputValue);
});
});
</script>
</head>
<body class="claro">
<h1>Output:</h1>
<div id="output"></div><p />
<button type="button" id="startButton">Start</button>
</body>
</html>
What is ugly is the inline function. Luckily I found the “Flattening Promise Chains” article where the author shows how to get rid of the function and simplify. However that article didn’t answer my question, how can you simply and directly chain promise objects. What the article showed is that you have to use the correct structure or design. Well, of course, but …
Answer
I thought this would be difficult. The Dojo documentation page never mentions that instead of the first ‘then’ argument being onFulfilled handler function it could be an actual promise object. But this is mentioned in specs such as the Promises/A+ specification. Or does it? The ‘Promise resolution procedure’ seems to mix compile time view with runtime, or I’m reading it incorrectly. Q documentation is very good on this. Will have to research further.
To directly chain separate promise chains the above code sample is modified as follows by just using an existing promise as the argument to another chains then(…) method. However, this can only be done using a promise returning function.
on(dom.byId("startButton"), "click", function(){
var debit = true;
var action = debit ? debitAccount : creditAccount;
setupTrans()
.then(action)
.then(outputValue);
});
Chaining using actual promise objects inside the then(f,r) is made easy using the approach shown next, using “scalars”.
Chaining values into promise chains
While looking at this subject I thought about a situation where we want to directly insert a known value into the chain so that the next ‘link’ will use this as the prior handler’s return value.
We have a chain, and I want to put the great Answer in there. This won’t work: var p1 = promiseReturningFunction().then(…).then(’42’).then(….).
Sure you can wrap ’42’ in an inline function then(function(){return ’42’}), but that is yucky. How about a function that does that:
/** return a function that returns the value */
function now(value){
return function(){return value;};
}
Now you can do: var p1 = async().then(…).then(now(’42’)).then(….);
Here is the prior source code with the changes. This is just some of the script changes. The output on screen is:
completed with value: 75
completed with value: Goodbye
/* ... */
function outputValue(value){
dom.byId("output").innerHTML += "<br/>completed with value: " + value;
}
function now(valueOrPromise){
return function(){return when(value);};
}
on(dom.byId("startButton"), "click", function(){
var debit = true;
var action = debit ? debitAccount : creditAccount;
var promise1 = setupTrans()
.then(debit ? debitAccount : creditAccount)
.then(outputValue)
.then(function(){
console.log("now do another promise chain...");
});
promise1.then((now('Goodbye'))).then(outputValue);
});
Using promise object directly in then(f,r) by using now(x)
With the Promises implementation in Dojo, you can’t use promiseObject.then(anotherPromiseObject). And you can’t use promiseObject.then(when(anotherPromiseObject)). At least, I haven’t figured how to do it.
With the now(x) function shown above, this is easy. The code below will output:
completed with value: 75
completed with value: 100
completed with value: undefined
completed with value: finished both chains
/** return function that returns the value
or apply Dojo's when(...) to return a Promise
*/
function now(valueOrPromise){
return function(){return when(value);};
}
on(dom.byId("startButton"), "click", function(){
var debit = true;
var action = debit ? debitAccount : creditAccount;
var promise1 = setupTrans()
.then(action)
.then(outputValue)
.then(function(){
console.log("now do another promise chain...");
});
debit = false;
action = debit ? debitAccount : creditAccount;
var promise2 = setupTrans()
.then(action)
.then(outputValue);
promise1
.then(now(promise2)).then(outputValue)
.then(now("finished both chains"))
.then(outputValue);
});
});
Groovy language has Promises too GPars brings Promises to the Groovy language.
Synopsis
Reducing the quantity of information to what is essential is very important in maintaining complex software. Modern development environments have debuggers and other tools to assist in this effort: conditional breakpoints, watch expressions, logging, stop on change techniques, and so forth.
As a supplementary technique an approach to debugging is shown which uses the dynamic differences in a running system to output a reduced level of information. A simple programmatic implementation is shown in a JavaScript scenario, but the technique is applicable using other languages and automated techniques.
Context
Recently I was debugging a legacy JavaScript program. I used the outstanding FireBug plug-in in the FireFox browser. By the way, FireBug is truly indispensible, such a wealth of features and it is FOSS. Now combine that with Selenium which gives you the ability automate the browser and you have a powerful Integrated Development Environment (IDE). All that could be lacking is time and skill.
Unfortunately, I was not finding the problem. Later at home I got an idea. What if I could simply log what changed? That way I could more clearly see how the program behaves. All programs are just obfuscated state machines. Through a use-case or scenario, state variables are being manipulated and non-state data is being changed. Correlating these two with the events that drive the system, is a prime aspect of debugging.
Forces
• You want to log what is happening in a system, but you don’t want to be inundated with non-useful data.
• You or someone else want to be able to quickly resume this investigation in another situation or project.
• By saving ‘interests’, a Built-in-self-test aid can be implemented.
• During development hooks can be put in place to provide differential logging.
• You want to capture the live system not a system which is controlled by a debugger.
• You cannot install new systems, tools, or frameworks to help.
• Breakpoints will break as the system is maintained.
Solution
My solution was to dynamically log the differences. I’m sure this is not new. However, it probably is not explicitly discussed. The usual approach is to log as much as possible, but since this is prohibitive, instead the logging is limited, level filters, and contexts are used to control the output. Breakpoints and watches while usually enough to do the job are in the long term brittle since the target program may change during the dev and debug process. Thus, dynamic difference reporting would be a better approach in this situation.
Consequences
Todo.
Implementation
There are a few implementation choices.
API
One is to declare the data of interest. Then the logging system will track these and only output anything when there is a change in any of the data. With this approach the logger must be triggered either manually or automated. Manually is very easy; at an event change or at enter and exit points of a module or function, the differential logger is invoked.
Automated
To avoid changing the source code (it may not even be available), AOP could be used to implement the above. For example, an Around advise can be used to differentially trace the changes in methods.
Framework Plugin
If the system under test is running as part of a debug or is instrumented, it may already have a facility to capture change events. For example, FireBug, the FireFox plugin has this and the FireBug plug-in, FireDiff, implements a change monitor and visualization output. There may be or should be a comparable plug-in for other systems like the Eclipse IDE.
Code Example
To illustrate this concept it is used to debug a web page’s DOM changes via JavaScript. The page has a button that when clicked will randomly change two input field’s content. Notice that when the change involves just an ordering of two adjacent characters in a field, it is not readily apparent. The Differential logging makes it explicit. The log output is just the result object converted to JSON. An example: {“event”:”run”,”record”:{“foo2”:[“”,”accd”],”foo4″:[“”,”ollrr”]}}. A real implementation would do otherwise.
To specify which DOM nodes to monitor, a JavaScript object is used:
{"foo2":{"get":"v"},"foo4":{"get":"v"}};
The “get” field is used to specify how to get the value from the browser DOM. Another method would be “innerHTML” access. A better approach is to detect the appropriate ‘getter’ via using the node type.
Listing 1, implementation
/**
*
* Differ constructor.
*
* @param data which nodes to record
* @param formatter how to show results
* @returns
*/
function Differ(data, formatter){
this.data = data;
this.formatter = formatter;
if(!formatter){
this.formatter =this.format;
}
};
/**
* Data and methods of class.
*
*/
Differ.prototype = {
MAXSIZE : 15,
/**
* Detect changes and save latest.
*
* @param event Where was diff created
* @returns result object
*/
diff : function(event){
var results = {};
if(!data){
return results;
}
if(event){
results = {"event":event};
}
results["record"] = {};
var record = results["record"];
for(var key in data){
var ref = document.getElementById(key);
var domValue = "";
switch (data[key].get) {
case "v":
domValue = ref ? this.shorten(ref.value) : "";
break;
}
var prior = data[key].value ? data[key].value : "";
if((domValue != prior)){
record[key] = [prior, domValue];
data[key].value = domValue;
}
} // end for
return results;
},
/**
* Convert results to message.
*
* @param results object
* @returns JSON
*/
toMessage : function(results){
var obj = results;
if(!results){
obj ={"error":"results is undefined or null"};
}
return this.format(JSON.stringify(obj));
},
/**
* Get size of an object
* @param obj
* @returns {Number}
*/
size : function(obj){
var len = 0;
for(var key in obj){
if(obj.hasOwnProperty(key)){
len++;
}
}
return len;
},
/**
* Limit size of value.
*
* @param value
* @returns
*/
shorten : function(value){
return (value.length > this.MAXSIZE) ?
value.substring(0,this.MAXSIZE-1):value;
},
/**
* Make message human readable.
*
* @param message
*/
format : function(str){
return str + '<br/>';
}
};
Listing 2, Demo script
var data = {
"foo2" : {
"get" : "v"
},
"foo4" : {
"get" : "v"
}
};
var differ = new Differ(data);
differ.diff(); // prime values
function runIt() {
var ref2 = document.getElementById('foo2');
var ref4 = document.getElementById('foo4');
if(Math.floor(Math.random()*101) > 60){
swapInputs(ref2,ref4);
}else{
scramble(ref2,ref4);
}
var msg = differ.toMessage(differ.diff("run"));
document.getElementById('output').innerHTML += msg;
document.getElementById('output').innerHTML +=
Array(60).join("-") + "<br/>";
}
function scramble(ref2, ref4){
var ch = ref2.value.charAt(1);
var r2v = ref2.value;
ref2.value = replaceChar(r2v, 2, ch);
ref4.value = replaceChar(ref4.value, 2, ref4.value.
charAt(1));
}
function swapInputs(ref2,ref4){
var temp = ref2.value;
ref2.value = ref4.value;
ref4.value = temp;
}
function replaceChar(str, ndx, v) {
return str.substr(0, ndx) + v +
str.substr(ndx + v.length);
}
Jenkins is a joy to use. Well, it is not perfect, what is? Like, I need to pass the user’s name that invoked a build via Jenkins to the target DOS script (yea, Windows) that eventually invokes the legacy Ant scripts.
Finally got a Jenkins server installed. Had a host of system issues, like communicating to our source code repo.
Jenkins is a joy to use. Well, it is not perfect, what is? Like, I need to pass the user’s name that invoked a build via Jenkins to the target DOS script (yea, Windows) that eventually invokes the legacy Ant scripts. A quick Google search shows that this is asked in various ways, but no answers. For example, here or here. Hmmmm.
Anyway, now comes a trial use, to see if it is what we really need and can we manage it to do what we will want. With 400 plugins, I don’t see how it could lack. Plus, I’m sure I can use the Groovy plugin to cobble something up. Jenkins even includes a Groovy Console. Finally, there is a road map for possible migration of legacy Ant scripts to Gradle using the Gradle Plugin.
I take back my past snarky comment. Jenkins is not just a pretty face on Cron.
Off Topic
Was watching the Easyb intro video. BDD is interesting. Definitely “should” is a better then “test”. With so many great tools why are products still bug ridden?
Enterprise Level CI Server?
I was told that QuickBuild is better for corporate use.
Wikis and the home link
BTW, is there some Wiki law that says a wiki shall never ever have a link to the parent project? If you get tossed into a wiki by following a link, invariably you will click in agony at links that should go to the real home. Instead, you have to edit the URL in the address bar. Since I never curse, I can’t write “wtf”.
Now cygwin is awesome, bringing some of the best tools from *nix. However, with Powershell, Microsoft finally transformed the Windows command line into a powerful and usable administration resource, and unlike *nix scripting, based on Objects, not text.
In this blog entry, Windows Tricks For Whacking Lingering Java Processes …, the author resorts to the cygwin linux shell to automate an admin need. Now cygwin is awesome, bringing some of the best tools from *nix. However, with Powershell, Microsoft finally transformed the Windows command line into a powerful and usable administration resource.
Below a line from the bash script the author used to find each process given a process name and then loop thru each process ID and terminate it. Note how the script requires a good understanding of bash scripting and in addition multiple Linux utilities ps, grep, and sed. The most important line in the script is:
found=` ps -aW | grep $procText | sed -e's/^s*([0-9][0-9]*).*/1/' `
Here is how this works.
ps -aW: show all processes and windows too.
grep $procText: only use lines that contain the command line string
sed -e: run stream editor inline
s///: substitute
^s*: all beginning white space
([0-9][0-9]*): find two or more digits, remember these.
.*: any characters.
/1/: replace with the found digits
Very complex!!!! But, to *nix users this is nothing. Its even fun to do. With a Linux command line you can rule the world. Note that in the above script line, it’s just text processing.
Powershell has a different approach. In Powershell one works with objects (in the OOP sense). Thus, instead of transforming everything to text, one manipulates properties of objects. One pipes object instead of text.
To do:
Show how to do this in Powershell. Unfortunately, wrote the above a long time ago and lost what little Powershell mojo I was developing. Any help?
Update
3/18/12: The powershell approach will probably be expanded from something like this, which just lists the processes by name and process ID:
Example source code illustrating use of the HTTP server included in Java JDK 1.6. via Groovy to present a browser-based UI to a local application.
Code illustrating use of the HTTP server included in Java JDK 1.6. via Groovy to present a browser-based UI to an app.
Result
The code allows this usage:
main.SimpleServer.serve(0){ s, t, p ->
// handle the request, response here...
// now shut down,
s.stopServer()
}
Listing 1, How it’s used.
This will create a web server using the host “localhost” at an unused port p, and then automatically open the default browser at “http://localhost:p/”. The closure will be the application. Neat. Of course, you would only use this behind a firewall, etc.
The above code, as a I later discovered, is very similar to other frameworks. Here is a new one I just learned about:
See also: vert.io. “Effortless asynchronous application development for the modern web and enterprise”
See also “Java development 2.0: Ultra-lightweight Java web services with Gretty” link, for an approach using Gretty.
Hmmm, just noticed that this is the ‘look’ of a simple node.js example. No way to dupe node.js of course, it is a low level thing, but can streaming event based programming be done with Groovy? Perhaps with GPars. See this discussion node.groovy?.
A sample session running the example code and using telnet is:
telnet localhost 21224
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
GET /?reply=24 HTTP/1.1
HTTP/1.1 200 OK
Content-length: 15</pre>
<h1>Wrong!</h1>
<pre>Connection closed by foreign host.
The Eclipse console contains:
+++++++ Simple Server ++++++++++++++++++
uri: [/?reply=24]
protocol: [HTTP/1.1]
query: [reply=24]
path: [/]
params: [[reply:[24]]]
-------------------------
Stopping server ... stopped!
The browser will show:
The app as seen on browser
Scenario
You have to supply a GUI for running a local Java application. One example could be the setup of a product build. Builds, though usually executed with Ant, Maven, or Gradle, may still require the user to select some target parameters or build type. Why not just use the browser? It can present a more modern interface and with good design, allow more fault tolerant use then that with command line or multiple prompt boxes.
This approach also allows future remote build server use, since the designed browser UI can be reused. The browser is ubiquitous and creating web pages is relatively easy (though can be a hair pulling nightmare sometimes). And, with the use of AJAX and high-level frameworks, like Dojo, JQuery, or React, browser clients can more easily duplicate the usability of dedicated thick client apps. HTML5 (see HTML Cheatsheet) is also helping to obliterate any remaining reasons for using a thick-client app.
An embedded server as used here, is great when the task is ad hoc, short lived, or single user. In the provided example, once the input is received the server shuts down. For more complex or ubiquitous use a standard server or more powerful embedded server should be used.
Embedded Server
For a local application that accesses system resources and uses the browser as the UI, using an embedded server is the simplest approach. Pure Javascript, Applets, and other means are very complex and may run against configuration issues with Browser security settings and so forth. In the Java world, that would mean using the more popular Tomcat or Jetty servers.
However, Java 1.6 now includes a light-weight HTTP Server. So, the configuration and requirements are more tractable. Nothing more to download and programmatic configuration concerns are minor. Note that the Java HTTP server does not offer many features, one augments this with custom code. For example, parsing of the request query is not present.
That the package of this server is com.sun… is problematic. Will it stay around, become part of the javax packages, etc? Should the JDK even have this built in? According to this post by M. MacMahone
… is that the API and implementation are a fully supported, publicly accessible component of Sun’s implementation of Java SE 6. It does mean however, that the packages are not formally part of the Java SE platform, and are therefore not guaranteed to be available on all other (non Sun) implementations of Java SE 6.
Incidentally, the Groovy language has an import system, Grape, that can also make use of Tomcat or Jetty as transparently as using the JDK embedded server. See the Further Reading below for an example using Groovlets.
This code illustrates
(more for my future reference)
The list below was some of the things the code used and the final code listed here may no longer have them.
com.sun.net.httpserver API use.
Using JQuery in external js files.
How to stop the server.
With AJAX post
Timeout
HTTP context
Using ScheduledExecutorService to limit runtime.
A console ASCII spinner.
Groovy GString use.
Launching the default browser.
Selecting an unused port.
Simplistic state machine configuration.
Detecting Java version.
AJAX using JQuery.
Groovy object construction from script.
Use of Closure.
Basic authentication
Quasi Anonymous class use in Groovy
access to resources
Of course, not great example of the above, but … Warning, code is not production ready, etc. There is no real exception handling!
How it works.
TODO: Give some idea what all that code does.
The index.html creates a simple form with three buttons (submit, ping, and end), an input field, and a ‘console’ output area.
– submit: send the answer to the server which then gives feedback, correct or wrong. The server then deliberately shuts down.
– ping: sends an AJAX request to the ping context which just sends back the time, the response is appended to the console.
– end: sends an AJAX request to the ‘stop’ context. The server responds with ‘stopping server …’, then shuts down. All buttons are disabled.
Why Groovy
Groovy is a dynamic JVM based language whose most prominent feature is that it extends the Java syntax to be more usable, i.e., less wordy,. From the example, a method that prints the contents of a map can be defined as:
The Groovy In Action book is pretty thorough. Chapter 1 makes a good case, and may still be available here.
Why not Groovy? Well, being dynamic can be rough, especially if it impacts the ability of a IDE to offer the features that come from using Java, like completions, etc. The Eclipse plug-in is getting much better, and I read the IntelliJ IDEA groovy support is top-notch. But, the worlds most popular language, JavaScript, is dynamic, and it hasn’t bothered too many people (maybe end users?).
Source
When I first wrote this, I created a complicated “app” with a bunch of class files and so forth. Then I simplified it. Later I said, yuck. Finally I decided that this should just be a simple one method invocation as shown at the beginning of this post. Consequently, a bunch of stuff in this sample code is guarded by conditionals and it works but sure can be simplified.
While coding I ran into a strange classloader issue see this post. There are a few files in this demo
SimpleServer.groovy: provides the ‘facade’ to HTTPServer. Handles the “/” context.
AppContext.groovy: An example of an “app” that gets mapped to a context path.
/**
* File: AppContext.groovy
* Date: 20110320T1952-05:00
* Author: jbetancourt
*/
package main
import java.text.DateFormat;
import java.text.SimpleDateFormat
import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpHandler;
/**
* The context that hosts the application.
*
* @author jbetancourt
*
*/
class AppContext implements HttpHandler{
def SimpleServer server;
def static RUNNING = 'running'
def currentState
def transitions = [:]
/** */
AppContext(SimpleServer server){
this.server = server
currentState = server.getServerProperty("initialState")
def statesProperty = server.getServerProperty("transitions")
}
/**
* Handle the given request/response.
*
* The first response is the input form. The subsequent
* request evaluates the answer if any, and then the server
* is stopped. Any exception will also stop the server.
*
*/
@Override
public void handle(HttpExchange t) throws IOException {
try{
def uri = t.getRequestURI()
def final query = uri.getRawQuery()
def path = uri.getRawPath()
def params = server.parseQuery(query)
showInfo([query:query,uri:uri,path:path,currentState:currentState,params:params])
def mode = params.get("mode")
if(atState("running")){
if(mode){
if(mode[0] == "mathInput"){
def reply = params.get("reply")
evaluateAnswer(t,reply)
transitionNextState()
}else if (mode[0]=="ping") {
server.sendString(t,"huh?")
}
}
}
if(atState("end")){
server.stopServer()
}
}catch(Exception ex){
ex.printStackTrace()
server.stopServer()
throw ex;
}
}
/**
* And, send response.
*/
def evaluateAnswer(t,answer){
def reply
try{
reply = (answer[0] != '"23"') ? "Wrong!" : "Correct!"
}catch(Exception ex){
reply = "wrong"
}
server.sendString(t,"</pre>
<center>
<h1>$reply</h1>
</center>
<pre>")
}
/** */
def showInfo(info){
println "++++++++ AppContext +++++++++++++++++"
info.each{ k,v ->
println k + (v ? ': [' + v +']' : ": []")
}
println "-------------------------"
}
/** */
def ping(t){
def now = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss").format(new Date())
server.sendString(t, "$now")
}
/** */
def setTransitions(s){
s.split(",").each{ tran ->
def kv = tran.split(":")
transitions.put(kv[0],kv[1])
}
}
/** */
def atState(s){
return currentState == s
}
/** */
def transitionNextState(){
print("currentState[$currentState],")
def ns = transitions[currentState]
currentState = (ns ? ns : "end")
println(" next state=[$currentState] ")
}
} // end AppContext class
Summary
Shown was a simple example of using the JDK 1.6 embedded HTTP server using the Groovy language.
Updates
April 15, 2011: Added some beginnings of code to handle image resources.
August 4, 2011: Just saw an old post on same subject. “Groovy++ in action: DSL for embedding HttpServer”. Added it to references. That one creates a DSL to use the embedded HTTP server. Nice.
Embed a script engine and have new ways of analyzing and developing code.
In development, simple but powerful tools to get at the essence of a code source in order to understand, test, and extend it is essential. This is even more important in Test Driven Development (TDD). Eclipse’s Java Debugger is for most situations, powerful enough. Eclipse has an Expressions View available in the debug perspective to execute snippets of code. The Java Development Toolkit (JDT) also has a Scrapbook facility that allows the creation, storage, and running of experimental code. And, of course, all the other features of the debugger are excellent.
However, when you need it, it’s possible to embed a script engine and have new ways of analyzing and developing code. In listing 1, an app shows the use of the ConsoleWaiter class. When the code executes the waiter.run() at line 43, it opens the Groovy Console which allows the use Groovy shell scripting in a GUI frame, see figure 1. When the console is closed the app continues executing.
Listing 1
/*
* File: ExampleApp2.java
* @author jbetancourt
* Date: 20101213T1718-5
*
*/
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
/**
*
* @author jbetancourt
*
*/
public class ExampleApp2 {
static public String greeting = "Hello world!";
private static final List<String> nodes;
static {
nodes = new
ArrayList<String>(
Arrays.asList("one","two"));
}
private String title = "Example 2";
public String getTitle(){
return title;
}
/**
* @param args command line args
*/
public static void main(String[] args) {
ExampleApp2 app = new ExampleApp2();
ConsoleWaiter waiter = new ConsoleWaiter(app);
waiter.setVar("greet", greeting);
waiter.setVar("nodes", nodes);
waiter.setVar("title", app.getTitle());
waiter.run();
System.out.println("Done!");
}
}
Console screen capture, click to viewAnother screen capture, click to view
This is awesome. In one project I had to examine the contents of a Properties object. Did it have an “email” value? I was stumped when using the Eclipse debugger, it did not show all entries in the Map, at the end was “…”. Sure, I could use the Expressions window, but with the Console I could not only do a get(key), but iterate using Groovy style closures and much more.
The magic that enables this is the ConsoleWaiter.groovy class shown below in listing 2 that was written by John Green. Since a Groovy script is a Java class underneath, in Eclipse you can call Groovy from Java easily (some compiler magic).
Listing 2
/**
* File: ConsoleWaiter.groovy
*/
import groovy.lang.Binding;
import groovy.ui.Console;
/**
* Provides a wrapper for the console.
*
* Based on source by John Green
* Adapted from: http://www.oehive.org/files/ConsoleWaiter.groovy
* Released under the Eclipse Public License
* http://www.eclipse.org/legal/epl-v10.html
*
* I added methods to allow use from Java.
*
* The run() method launches the console and causes this thread
* to sleep until the console's window is closed.
* Allows easy interaction with the objects alive at a given
* point in an application's execution, like in a debugger
* session.
*
* Example 1:
<pre> * new ConsoleWaiter().run()
*</pre>
*
* Example 2:
<pre> * def waiter = new ConsoleWaiter()
* waiter.console.setVariable("node", node)
* waiter.run()
*</pre>
*/
class ConsoleWaiter {
Console console
Object source
boolean done = false;
/** */
public ConsoleWaiter(Console inConsole){
this.console = inConsole
}
/** */
public ConsoleWaiter(Object source){
console =
new Console(getClass().classLoader,
new Binding())
this.source = source
console.setVariable("source", source)
}
/** */
public void setVar(String key, Object value){
console.setVariable(key, value)
}
/** */
public void setVar(String key, List values){
console.setVariable(key, values)
}
/** */
public void setVar(String key, Object[] values){
console.setVariable(key, values)
}
/** */
public void run() {
console.run()
// I'm a little surprised that this exit() can be private.
console.frame.windowClosing = this.&exit
console.frame.windowClosed = this.&exit
while (!done) {
sleep 1000
}
}
/** */
public boolean isDone(){
return done;
}
/** */
public void exit(EventObject evt = null) {
done = true
}
/** */
public Console getConsole(){
return console;
}
}
Eclipse Integration
The disadvantage of this approach is that you have to put extraneous code inside the tests or target class. Not only is this tedious and slow, what if code is accidentally deployed like this? A better approach is to just set a breakpoint in the code, and then have the ability to open a script console at that breakpoint, in the Java Stack Frame, that has access to the execution context and classpath. Is there an Eclipse add-in that does this? If not, there should be.
Conclusion
Shown was a simple example of embedding a Groovy console in Java code to allow scripting. Of course, this is not a new idea. It was even mentioned in an older JVM scripting language, Beanshell. Note that it is possible to instead of using a GUI console, to use the Groovy shell, InteractiveShell class. In the reading list below this approach is taken to allow remote scripting of a server hosted application.
Updates
Oct 10, 2011:
Interesting tool that could be relevant: YouDebug.
March 20, 2012: Not exactly same scenario, but the concept of an embedded script console is found in many products. Jenkins CI Server has one and it uses Groovy. Jenkins Script Console
Foregoing the use of JSON as a data interchange when Groovy language applications must interact internally or with other Groovy applications would be, well, groovy.
Summary
Foregoing the use of JSON as a data interchange when Groovy language applications must interact internally or with other Groovy applications would be, well, groovy.
Introduction
JavaScript Object Notation (JSON) is a language-independent data interchange format based on a subset of the JavaScript (ECMA-262 3rd Edition) language. Many languages and libraries now support marshal to and from JSON using external libraries or extensions. This complicates applications since they must rely on more subsystems and there may be a performance penalty to parse or generate an external object notation.
If an application must only interact within a specific language or environment, such as the Java Virtual Machine (JVM), perhaps using the host language’s data structures and syntax will be a simpler approach. Since Groovy (a compiled dynamic language) has built-in script evaluation capabilities, high-level builders (for Domain Specific Language (DSL) creation) , and meta-programming capabilities, it should be possible to parse, create, transmit, or store data structures using the native Groovy data interchange format (GDIF), i.e., based on the native Groovy data structures.
Below is the same data payload; this time using Groovy syntax. Note that there are not too many differences, the most striking is that maps are created using brackets instead of braces. It looks simpler too.
/**
* File: GrON.groovy
* Example class to show use of Groovy data interchange format.
* This is just to show use of Groovy data structure.
* Actual use of "evaluate()" can introduce a security risk.
* @sample
* @author Josef Betancourt
* @run groovy GrON.groovy
*
* Code below is sample only and is on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
* or implied.
* =================================================
*/
*/
class GrON {
static def message =
'''[menu:[id:"file", value:"File",
popup:[menuitem:[[value:"New", onclick:"CreateNewDoc()"],
[value:"Open", onclick:"OpenDoc()"], [value:"Close",
onclick:"CloseDoc()"]]]]]'''
/** script entry point */
static main(args) {
def gron = new GrON()
// dynamically create object using a String.
def payload = gron.slurp(this, message)
// manually create the same POGO.
def obj = [menu:
[ id: "file",
value: "File",
popup: [
menuitem : [
[ value: "New", onclick: "CreateNewDoc()" ],
[ value: "Open", onclick: "OpenDoc()" ],
[ value: "Close", onclick: "CloseDoc()" ]
]
]
]]
// they should have the same String representation.
assert(gron.burp(payload) == obj.toString())
}
/**
*
* @param object context
* @param data payload
* @return data object
*/
def slurp(object, data){
def code = "{->${data}}" // a closure
def received = new GroovyShell().evaluate(code)
if(object){
received.delegate=object
}
return received()
}
/**
*
* @param data the payload
* @return data object
*/
def slurp(data){
def code = "{->${data}}"
def received = new GroovyShell().evaluate(code)
return received()
}
/**
* @param an object
* @return it's string rep
*/
def burp(data){
return data ? data.toString() : ""
}
} // end class GrON
Possible IANA Considerations
MIME media type: application/gron.
Type name: application
Subtype name: gron
Encoding considerations: 8bit if UTF-8; binary if UTF-16 or UTF-32
Additional information:
Magic number(s): n/a
File extension: gron.
Macintosh file type code(s): TEXT
API
To be determined.
Security
Would GrON be a security hole? Yes if it is implemented using a simple evaluation of the payload as if it were a script. The example shown above used evaluate() as an example of ingestion of a data object. Incidently, this is the same naive approach used in some JavaScript applications and general not recommended.
For real world use, some kind of parser and generator for object graphs would be needed. The advantage would accrue if the underlying language parser could be reused for this.
Now this begs the question, if Groovy must now support a data parser, why not just use JSON with the existing libraries, like JSON-lib? [May 7, 2014: the built in JSON support].
Is using the Java security system an alternative as one commenter mentioned?
Notes
The idea for GrON was formulated about a year ago. Delayed posting it since I wanted to create direct support for it. However, the task required more time and expertise then I have available at this time.
I was debating what to call it, if anything. Another name I considered was Groovy Data Interchange Format (GDIF), but I decided to follow the JSON name format by just changing the “J” to “G” and the “S” to “r” (emphasizing that Groovy is more then a Scripting language, its an extension of Java).
How does one manage the use of multiple versions of a software tool, such as a programming language or framework?
How does one manage the use of multiple versions of a software tool, such as a programming language or framework?
*inux
In a blog post the author, Jeff, shows a way to switch between versions of his development tools, using the Grails framework as an example. In Linux he creates symlink to the specific version. He further uses aliases in the ~/.profile to allow quick changes to these symlinks. Nice post.
Windows 7
Many, I’m sure, have been using that approach, symlinks to the current version. For example, on Windows 7, I use directory junctions. Below is a snippet of my java folder. Notice that all the Groovy versions are in the GroovyVersions subfolder and the file “groovy” will point to one of these.
c:java>dir
Directory of c:java 04/07/2010 05:55 PM <JUNCTION> groovy
04/07/2010 05:52 PM <DIR> GroovyVersions
The built-in command to create a junction in Windows 7 is:
c:java>mklink /?
Creates a symbolic link. MKLINK [[/D] | [/H] | [/J]] Link Target
/D Creates a directory symbolic link. Default is a file
symbolic link.
/H Creates a hard link instead of a symbolic link.
/J Creates a Directory Junction.
Link specifies the new symbolic link name.
Target specifies the path (relative or absolute) that the new link
refers to.
So, to create the shown link one would execute:
mklink /J groovy c:javaGroovyVersionsgroovy-1.7.2
or should that be:
mklink /D groovy c:javaGroovyVersionsgroovy-1.7.2
(todo: review the differences between a junction and a softlink)
Changing versions
Changing to a particular would now require delete of the current junction and then creation of a new one to point to the desired version. However, this is clunky. Jeff used shell aliases which could just as easily be done in Windows using batch files or PowerShell commands.
Advantages
As the original blog post mentioned, this allows easier control over the environment. Thus, GROOVY_HOME environment variable would not have to change. It would remain, GROOVY_HOME=C:javagroovy
Updates
There are already systems to handle versioning of software in an OS. One approach that works on Linux is GVM.
In my blog post “How to do Ad Hoc Version Control” I suggested Mercurial as very good DVCS to use. However, Git is good enough I suppose, having not used it yet.
In my blog post “How to do Ad Hoc Version Control” I suggested Mercurial as the DVCS to use. However, I read that Git is a very popular and powerful alternative to Mercurial. Over at the VIM Tips Blog, this post, shows how easy it is to create a local folder repository using Git. Very similar to Mercurial’s steps.
If you want to get some version control on a simple local project (i.e., it doesn’t have a big remote repository or anything), then you can simply use git init to create your own standalone local repository. For example, if you’re working on some design concepts for a new application, then you could do something like the following:
mkdir design_concepts
git init
Now you can add files, commit, branch, and so on, just like a “real” remote Git repository. If you want to push and pull, you’ll need to set up a remote repository.
git remote add
git pull master
Which is better Git or Mercurial?
For ad hoc use, it probably doesn’t matter. What matters is how easy it is to install, learn, and create repositories on demand. For large systems with many version control requirements, one must evaluate each alternative carefully. I’m sure advance use will be ‘difficult’ in any VCS system.
A nice recent blog post by C. Beust, “Git for the nervous developer”, provides a nice perspective on using Git, though perhaps not entirely relevant to the stated Ad Hoc usage scenarios. That post has a nice list of Git references.
What is an optimal ad hoc workflow?
In the original article I gave context for an ad hoc situation. For example, fixing some server configuration files. In that use-case, Git-Flow would be overkill. Perhaps even GitHub Flow too.
A simpler approach is having two “standard” branches: master and dev. The ‘master’ branch is the live or solution branch. The ‘dev’ branch is where you will make any changes or experiments. Rarely would you create a new branch off of dev. If the ad hoc situation is that complicated, you have other issues! In a “lone developer” workflow however, branching off of dev would be more common. Another ingredient is a separate local Git repo to serve as a backup.
The GitHub Flow is still very applicable, of course, and may be better then the above. The GitHub Flow reminds me of “The Flying Fish Approach” that was documented in the cvsbook.
Once one of these attempts on the dev branch (or github flow feature branch) produces a positive result (the dev branch is live while it is current branch), you merge it into the master branch and then tag the master branch. Each success is tagged, that way when the success turns out to be bogus or has unintended consequences, you go back to the last ‘good’ tag. If you don’t use tags, you can just use the commit SHA1 id. Remember the master is always the final live environment of the ad hoc situation.
Now how to use Git to do the above workflow? I don’t know. Still learning Git. 🙂
