Statechart with memory state and gates


Abstract

There are some things a FSM cannot handle[5].  The author attempts addresses this by adding memory-based States connected to logic “gate” junction pseudo states.  This enable Statecharts to handle more complex applications while still remaining a transition network at its core.

Josef Betancourt, November 27, 2018

1. Background

Hierarchical Finite State Machines (HFSM), especially the Statechart (UML state machine), are a useful tool in increasingly complex applications like Graphical User Interfaces.  They allow the logical behavior flow requirements to be expressed in a declarative model.

The state of the art, Statecharts, are defined in diagrammatic terms by David Harel as:
“statecharts = state-diagrams + depth + orthogonality + broadcast-communication”. [1]

In real world use, various frameworks and libraries such as the SCXML, provide additional constraints, and Executable Content[4] to that declarative state transition network.  These other feature are actually executing an imperative flow of control.  So it could be argued that the state graph does not really specify the system in a declarative model.

Why isn’t the state graph enough?  While StateCharts do limit the combinatorial state explosion that are a drawback of traditional FSM, they export some of the complexity to a side channel or dimension of the Statechart run-to-completion behavior.  These are ‘extended state machines’[2].

2. Example

A GUI form presents a list of fields that must be completed or activated by the user.  There is also a Done button that the user clicks to signal that the form is ready to be submitted.

Requirement:  All fields must be activated by the user.  When Done is clicked, an error message is shown if all field have not been activated, else the form is submitted.  This is a common ‘validation’ scenario in form processing. The fields, for example, could be checkboxes. The Done button invokes a validation process to determine if the form is ready for submission.

A naive Statechart of this scenario would make each field and the Done button states.  Each field could even be a concurrent state (in an orthogonal region) of the form. But, when the Done button is clicked and the form must transition to a Validating state, how is the state of all the buttons used to determine the transition to error or submit?  How is the ‘all fields were checked’ captured in a Statechart?

A programmatic solution is just to transition to a validation state that determines if all fields were activated. In a Statechart this complexity can be handled by an invoked program/script that evaluates a transition guard to a target state: error or submission.  This guard uses the extended state to allow this determination.


Note:  

This is just a simple example.  That it could be done with only present Statecharts is not the issue.  It is probably not a perfect example for this topic. 

The UML statechart standard, for example, has a richer set of states for this purpose.  Pseudo States such as: join, fork, junction, choice, and exitPoint.

David Khourshid, the developer of XState, responded to this example at https://github.com/davidkpiano/xstate/issues/261#issuecomment-442451581

“Have you seen the semantics of <final> states (as in SCXML)? https://xstate.js.org/docs/guides/final.html
We can model “all fields must be valid” by setting 'valid' as the “final” state for each orthogonal node and listening for the done.state.* event.”

And then gave a code example.


3. Synchronous AND transitions

Petri Net approach

The example scenario could be trivially solved using a simple Petri Net model.   Each form field is a ‘place’ and when it is activated, a ‘mark’ or token is placed there.  All the places connect to a ‘Transition’ that fires when all the input places have a mark.  This fired transition puts a mark into the Valid place.

State Memory

Borrowing from Petri Nets, we can use marks by adding memory to a state.  This memory allows the synchronous use of prior state transitions. History states are already one type of memory used in Statechart standard.  By generalizing the use of memory, more complex systems can be modeled.

Why ‘synchronous’? A machine can have one current state.  In a Statechart, nested states form an XOR. To handle complexity, Statecharts also provide an AND in the form of orthogonal substates.  These are called ‘parallel’ in the XState implementation [6]. But, they could loosely be termed asynchronous, since their behavior could be independent of the superstate container.

Definition

State memory is a numeric limit value attribute used as a modulus.  The current value dynamically changes. The limits are:

n=-1: latch,
n=0: no memory (default),
n=1: toggle,
n=>1: roller

The values allow the state to be repeatably reused in operational in reactive state changes.

4. Operation

To provide a transition via an AND, a state must have a non-zero memory limit.  If the machine re-transitions to the same state, the memory value determines when the memory is cleared.  The preset memory size value is the increment modular rollover. Whenever the state is entered, the current value is increment.  When the value reaches the memory size, the current value is set to zero.

A size of zero, turns off memory.  A negative size, causes a state transition to a state to always have a memory of that occurrence, no incrementing.  A value of one clears the memory on each transition, so acts as a toggle. When size is greater than one, subsequent transitions to the state will begin to increment the memory until the value of the memory size setting.  

When all the memory states have a current value the AND fires, and the machine transitions to the target state of the AND.  Unlike a Petri Net, a fired AND also resets all the input state’s current memory to zero. In practice an implementation would make this behavior also configurable per each AND relation.

States with memory are ‘normal’ states in a Statechart.  Their ‘memory’ aspect only affects behavior when used with an AND pseudostate.

5. Symbol in visual diagram

The simplest realization in visual form is a Petri Net solid bar |.  A bar is also used in UML activity diagrams to indicate a fork/join.  Since the bar already has so many connotations, an actual military style AND circuit symbol  or a European style & rectangle could be used.  The simplest thing that would work is a solid bar:  |.

6. Example solution using Statechart memory

With the aforementioned approach the Petri Net equivalent solution is shown below.   Note that this diagram is not showing how to formally add memory and gates to the standard Statechart visual notation. 

7. Generalization to Gates

Adding memory to a state and its use to create an AND pseudostate leads to the possibility of other types of pseudostates to tackle fine-grained complexity and details.  Since these are more like logic ‘gates’, calling these “gates” seems appropriate.

With the addition of OR (||) and NOT  (o), many other gates could be created, such as NAND, NOR, and so forth.  This is similar to the many types of gates and their combinations that capture hardware logic and specified using Hardware Description Languages (HDL)[3].

8. Conclusion

The addition of state memory that connect to gates enables creation of more complex and detailed Statechart or HFSM models.  More work is required to evaluate this approach and modify or abandon.

In a way this effort bridges the gap from high level software back to hardware. It may be possible to use Computer Aided Design (CAD) technologies in this realm, especially if extendable Statecharts could be used as component building blocks.

References

  1. Statecharts – A Visual Formalism for Complex Systems by David Harel
  2. Extended States: https://en.wikipedia.org/wiki/UML_state_machine#Extended_states
  3. Hardware description language: https://en.wikipedia.org/wiki/Hardware_description_language
  4. Executable Content: https://www.w3.org/TR/scxml/#executable
  5. “What exactly can finite-state machines not do?”: https://cs.stackexchange.com/questions/14093/what-exactly-can-finite-state-machines-not-do
  6. Parallel State Nodes: https://xstate.js.org/docs/guides/parallel.html

Links

 

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Is REST tired? GraphQL a stimulant?

Based on my quick review, it seems a company should add GraphQL to their tech stack. It’s not an either/or situation.

REST
Representational State Transfer (REST) is the architecture underlying the WWW. It scales. The REST concept is also be used to create RESTful services as an alternative to older technologies such as Remote Procedure Calls (RPC) and SOAP. REST architecture scales.

REST has some limitations and critiques. For example, few REST APIs are truly “RESTful”. The level of REST is captured in the Richardson Maturity Model (RMM). The highest level requires use of HATEOAS, see “Designing a A True REST State Machine”.

“A very strong argument could be made that if most APIs are REST-ish instead of REST-ful, and assuming that most of the conventions that we’re actually using boil down to making URLs consistent and basic CRUD, then just maybe REST really isn’t buying us all that much.” — @brandur

One claim is that REST has issues since it is misapplied:
“…“REST is intended for long-lived network-based applications that span multiple organizations” according to its inventor. This is not a requirement for APIs that serve a client app built within the same organization.” — https://reactjs.org/blog/2015/05/01/graphql-introduction.html

GraphQL
GraphQL was created to fill a hole in modern web API. GraphQL unlike common REST queries, describes the ‘shape’ of data to retrieve from a single end-point. The structure of that data at the end point is a black box. The host knows how to fill in the shape to create the response. Kind of like SQL (in a very rough kind of comparison): In SQL the data has a shape, the Relational Model, and the single end-point queries, declaratively describe what to get from that shape. In REST when you ask for a cup you get the kitchen sink and all the cabinets.

“GraphQL is a declarative data fetching specification and query language for APIs. It was created by Facebook back in 2012 to power their mobile applications. It is meant to provide a common interface between the client and the server for data fetching and manipulations. GraphQL was open sourced by Facebook in 2015.” — GraphQL vs REST

The single endpoint is a critical distinction. To get distributed, consolidated, or nested data a REST endpoint could, of course, invoke an integration service on the backend, or use techniques such as API Chaining. in GraphQL the “integration” exists semantically on the client query. The declarative query just describes the result and the server provides it. And since the shape determines the data desired at the single endpoint, the over/under fetching of REST is avoided. The query is ‘resolved’ on the server which may invoke of existing multiple SQL, REST services, MQ, or other services. This affords natural growth of the API. In contrast a REST API grows via more endpoints and/or addition of query parameters (which could morph to disguised RPC).

Image on www.graphql.com

“GraphQL was our opportunity to rethink mobile app data-fetching from the perspective of product designers and developers. It moved the focus of development to the client apps, where designers and developers spend their time and attention.” — https://code.facebook.com/posts/1691455094417024/graphql-a-data-query-language/

In “How GraphQL Replaces Redux” after the team started to use GraphQL they had three ‘Startling Realizations’:

  1. Most of our state management code was concerned with merging and mutating data from discrete REST resources into the right shape for our UI (reducers, selectors, actions etc.).
  2. A lot of our most complex state management was trying to manage the asynchronous nature of getting all that data in the right order for a specific route (sagas, middleware, thunks etc.).
  3. Practically everything else, UI state, worked great with plain old React state.

And, then they deleted a lot of code. A similar experience shown here: “Reducing our Redux code with React Apollo”, https://dev-blog.apollodata.com/reducing-our-redux-code-with-react-apollo-5091b9de9c2a

Summary

  • GraphQL is not a replacement, both can be used; even “GraphQL over REST”
  • GraphQL allows less chatty communication
  • GraphQL is not perfect
  • For simple API GraphQL will only add complexity.
  • Has been proven in many production applications and sites.

Critiques of GraphQL
New technology hype hardly ever gives the true efforts and failed approaches that it took to make something work. GraphQL is no exception.

1. “GraphQL Seems Like We Do Not Want to Do the Hard Work of API Design”, https://dzone.com/articles/graphql-seems-like-we-do-not-want-to-do-the-hard-w
2. “Just Because Github Has a GraphQL API Doesn’t Mean You Should Too”, https://www.programmableweb.com/news/just-because-github-has-graphql-api-doesnt-mean-you-should-too/analysis/2016/09/21
a. API chaining is fix for chatty REST?
3. “Are there any disadvantages to GraphQL?”, https://stackoverflow.com/questions/40689858/are-there-any-disadvantages-to-graphql?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa
4. “Semantics and complexity of GraphQL”, https://blog.acolyer.org/2018/05/21/semantics-and-complexity-of-graphql/ Provides an algorithm for determining response size of a potential query.

Learning GraphQL
1. Introduction to GraphQL: http://graphql.org/learn/
2. “A Front End Developer’s Guide to GraphQL”, https://css-tricks.com/front-end-developers-guide-graphql/
3. “The Fullstack Tutorial for GraphQL”, https://www.howtographql.com/
4. Graphql.com tutorials: https://www.graphql.com/tutorials/
5. “Five Common Problems in GraphQL Apps (And How to Fix Them)”, https://medium.freecodecamp.org/five-common-problems-in-graphql-apps-and-how-to-fix-them-ac74d37a293c
6. “GraphQL Deep Dive: The Cost of Flexibility”, https://edgecoders.com/graphql-deep-dive-the-cost-of-flexibility-ee50f131a83d
7. GibHub explorer using GraphiQL: https://developer.github.com/v4/explorer/
8. “Thinking in Graphs”,
9. “Dan Schafer, GraphQL at Facebook”, https://www.youtube.com/watch?v=Ey3BtXDO2UM

Implementations
1. Prisma: https://twitter.com/GraphQL
2. Apollo client: https://www.apollographql.com/client/
3. Relay: https://facebook.github.io/relay/
4. GraphQL.js: http://graphql.org/graphql-js/
5. Apollo Server: https://www.apollographql.com/server
6.

YouTube Videos
*TODO*

Further Reading
1. “Is GraphQL Moving Toward Ubiquity?”, https://nordicapis.com/is-graphql-moving-toward-ubiquity/
2. “Top 5 Reasons to Use GraphQL”, https://blog.graph.cool/top-5-reasons-to-use-graphql-b60cfa683511
3. “GraphQL over REST at Reactathon 2018”, https://www.slideshare.net/sashko1/graphql-over-rest-at-reactathon-2018
4. “What are advantages and disadvantages of GraphQL, SOAP and REST?”, https://www.quora.com/What-are-advantages-and-disadvantages-of-GraphQL-SOAP-and-REST
5. “…And GraphQL for all?”, https://apihandyman.io/and-graphql-for-all-a-few-things-to-think-about-before-blindly-dumping-rest-for-graphql/
6. “REST versus GraphQL”, https://blog.pusher.com/rest-versus-graphql/
7. “Simplify relational logic with GraphQL”, https://scaphold.io/community/blog/querying-relational-data-with-graphql/
8. Who is using GraphQL? http://graphql.org/users/
9. “GraphQL is SQL for Knowledge, not Data”, https://react-etc.net/entry/graphql-is-sql-for-knowledge-not-data
10. “GraphQL vs REST comparison – Is This Solution from Facebook Worth Taking the Risk?”, https://www.netguru.co/blog/grapghql-vs-rest
11. “Sharing data in a Microservices Architecture using GraphQL”, https://labs.getninjas.com.br/sharing-data-in-a-microservices-architecture-using-graphql-97db59357602
12. StarBucks use of GraphQL: https://twitter.com/davidbrunelle/status/961103308957147136
13. “Richardson Maturity Model”, https://martinfowler.com/articles/richardsonMaturityModel.html
14. “2 Fast 2 Furious: migrating Medium’s codebase without slowing down”; https://medium.engineering/2-fast-2-furious-migrating-mediums-codebase-without-slowing-down-84b1e33d81f4
15. “Cartoon Guide to Facebook’s relay”, https://code-cartoons.com/a-cartoon-intro-to-facebook-s-relay-part-1-3ec1a127bca5
16. “Thinking in GraphQL”, https://facebook.github.io/relay/docs/en/thinking-in-graphql.html
17. The GitHub GraphQL API: https://githubengineering.com/the-github-graphql-api/
18. You don’t need a fancy framework to use GraphQL with React (All you need is a good naming strategy) : https://edgecoders.com/you-dont-need-a-fancy-framework-to-use-graphql-with-react-b47b436626fb#.hu577b1ln
19. “How GraphQL Replaces Redux”, https://hackernoon.com/how-graphql-replaces-redux-3fff8289221d
20. “Reducing our Redux code with React Apollo”, https://dev-blog.apollodata.com/reducing-our-redux-code-with-react-apollo-5091b9de9c2a
21. The GitHub GraphQL API: https://githubengineering.com/the-github-graphql-api/
22. https://twitter.com/GraphQL
23. “The Evolution of The New York Times Tech Stack”, https://stackshare.io/posts/evolution-of-new-york-times-tech-stack
24. “Designing a A True REST State Machine”, https://nordicapis.com/designing-a-true-rest-state-machine/
25. https://twitter.com/GraphQL
26. “The Evolution of The New York Times Tech Stack”, https://stackshare.io/posts/evolution-of-new-york-times-tech-stack
27. “Here’s my follow-up to REST is the new SOAP: let’s talk about the Original REST”, https://medium.freecodecamp.org/follow-up-to-rest-is-the-new-soap-the-origins-of-rest-21c59d243438
28. “GraphQL vs REST: Overview”, https://philsturgeon.uk/api/2017/01/24/graphql-vs-rest-overview/
29. “Reflections on REST — FSE’17 Keynote”, https://www.youtube.com/watch?v=6oFAmQUM8ws&feature=youtu.be
30. “A Journey Through React Apollo – Peggy Rayzis aka @peggyrayzis at @ReactEurope 2018”, https://www.youtube.com/watch?list=PLCC436JpVnK3xH_ArpIjdkYDGwWNkVa73&time_continue=1&v=fCXYA3lZTbo
31. “Dan Schafer – GraphQL: Client-Driven Development”, https://www.youtube.com/watch?v=vQkGO5q52uE
32. “A Real-World GraphQL Application in Production”, https://www.youtube.com/watch?v=6bRFElKG0a8
33. “What it takes to run the world’s largest GraphQL server”, https://www.youtube.com/watch?v=nOYyix0NpvY
34. “From REST To GraphQL”, https://www.youtube.com/watch?v=ntBU5UXGbM8&t=690s

React component with Media queries for Responsive

This is a very simple approach using matchMedia support. We store media queries and a listener function as an object in an array. When the media query returns true the listener function will set the relevant component state true.

In our actual project, this approach is used with Redux. The queries instead of setting component state, update the Store. Containers connected to this store send the breakPoint, orientation, and other props to each child component.

Just an experiment. If pursued further, would probably make it a HOC (Higher Order Component).

Gist available here.

Demo
If packed and loaded into a web page as we resize the browser we should see:

Change size of browser to test

state: { 
    "orientation": "landscape", 
    "breakPoint": "LG" 
}

Why?
While CSS media queries are very useful, sometimes being able to respond ‘behaviorally’ to media events is very powerful.

Tested on
Chrome, FireFox, IE, and Edge. I guess they all have JavaScript MediaQuery support.

How it works

In the life cycle method ‘componentDidMount’ we add a query to an array in the component:

this.mediaQueryFactory(obj, '(orientation: portrait)', this.onPortrait);

this.mediaQueryLists.push({
	mql: obj.mql,
	listener: this.onPortrait
});

the onPortrait is the listener function:

onPortrait : function(flag){
  if(flag){
	this.setState({orientation: 'portrait'});
  }
},

Full Code

Other MediaQuery React implementations

Addendum

{
  "name": "ReactMediaQuery",
  "version": "1.0.0",
  "description": "Attempt at using Media Queries in React",
  "main": "./index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "start": "webpack index.js dist/react-media-query-bundle.js"
  },
  "keywords": [
    "react"
  ],
  "author": "JosefBetancourt",
  "license": "SEE LICENSE IN COPYWRITE.TXT",
  "dependencies": {
    "babel-polyfill": "^6.9.1",
    "classnames": "^2.2.5",
    "react": "^15.3.0",
    "react-dom": "^15.3.0"
  },
  "devDependencies": {
    "babel-core": "^6.10.4",
    "babel-loader": "^6.2.4",
    "babel-plugin-add-module-exports": "^0.2.1",
    "babel-preset-es2015": "^6.9.0",
    "babel-preset-es2016": "^6.11.3",
    "babel-preset-react": "^6.3.13",
    "babel-preset-stage-2": "^6.11.0",
    "eslint": "^3.2.2",
    "webpack": "^1.13.1"
  }
}

package.json

var path = require('path');
const webpack = require('webpack');

module.exports = {
  /* entry point */
  entry: './index.js',
  /* output folder */
  output: {
    filename: 'dist/react-media-query-bundle.js'
  },
  devtool: 'source-map',
  module: {
    loaders: [
      {
        test: /\.js$/,
        loader: 'babel-loader',
        exclude: /node_modules/,
        query: {
          presets: ['es2015', 'react', 'stage-2']
        }
      },
    ]
  },
  eslint: {
    configFile: path.join(__dirname, 'eslint.js'),
    useEslintrc: false
  },  
  plugins: [
  ],
  resolve: {
    extensions: ['', '.js', '.json'] 
  },
  resolveLoader: {
      modulesDirectories: [
          'node_modules'
      ]
  }
};

webpack.config.js

<!DOCTYPE html>
<html>
  <head>
	<meta charset="utf-8">
	<meta name="viewport" content="width=device-width, initial-scale=1.0">  
  	<title>React Media Queries</title>
  </head>
  <body class="body">
    <div id="root" class="">Loading page, please wait ....</div>
	<script src="dist/react-media-query-bundle.js"></script>
  </body>
</html>

index.html

Reactive client application state constraint contracts

As in Design By Contract (DbC), the contents and structure of a Store mutation can be constrained by Preconditions, Postconditions, and Invariants. These constraints are part of the Store implementation.

Background
Modern JavaScript frameworks are now emphasizing a form of “global” state storage. This is not really new, but gained new impetus when React came out, which synergistically combined this with other factors, like components, virtual-DOM, and one-way data flow.

React Stores

“… contain the application state and logic. Their role is somewhat similar to a model in a traditional MVC, but they manage the state of many objects — they do not represent a single record of data like ORM models do. Nor are they the same as Backbone’s collections. More than simply managing a collection of ORM-style objects, stores manage the application state for a particular domain within the application.” — https://facebook.github.io/flux/docs/overview.html

In very large apps and multiple development teams, this Store could become a high maintenance cost and a source of defects.

Non-React Store
For example of a reusable Store implementation see Redux. Amazing video: Dan Abramov – Live React: Hot Reloading with Time Travel at react-europe 2015

Interesting Java implementation of Redux: redux-java.

Maintenance
Today’s great application and shiny new framework are tomorrow’s maintenance nightmare. Thus, it makes sense to add in features that will aid this future support.

Example
A store has a JSON map of { foo : true, fee : false, fum : {…}, … }.

A particular flow in the application will update foo to false. Is that ok?

If foo and fee are in the same ‘context’, can they both be false? Note that the ‘application’ or maintenance developer may not know that there is a relationship between foo and fee. With annotations, dependency injection, and so forth, even perusal of the source could make this determination difficult.

Sure, in a small app and simple Store, it would be easy to trace the program and find out the implicit constraints. In reality, Stores grow complex over time, the app gets complex, new requirements mean many changes, and the developer culprit has left the company. Stores will mutate to resemble the catch-all Windows’ Registry.

To compound this, the control flows are hidden within the framework being used. No matter if its today’s MVC usurper, React, or some Observable liberator, there is spaghetti flow and implicit state somewhere.

“At some point, you no longer understand what happens in your app as you have lost control over the when, why, and how of its state. When a system is opaque and non-deterministic, it’s hard to reproduce bugs or add new features.” —
http://redux.js.org/docs/introduction/Motivation.html

Constraints
One way to reduce issues is to use the techniques from data design and use schema constraints. This does not necessarily mean the use of actual schemas as in XML’s XSD or its wannabe JSON equivalent. One alternative is that the Store has an API for constraint binding to content, new life cycle functions apply the constraints. The result of a constraint violation depends on dev or production deployment, log or throw of exceptions.

Afterword
Notice how in the development world we are going back in time? Central stores, Data flow diagrams, functional programming, FSM (obfuscated), etc. Maybe Object-Oriented Programming was a great distraction like Disco. Time for some real funk. “Maceo, I want you to Blow!

Wow, this synchronicity is weird. I’ve been thinking of this post’s contents for a while. Started to write it and my twitter app on phone rings. @brianloveswords tweets about a blog post by @jlongster: “Starters and Mainainers”. Not exactly the same subject, but was about maintenance.

Links

Adaptive log level based on error context

The other day I was thinking about my companies’ application logging system, and wondered why we don’t increase the log output when there is a problem? I mean doing this automatically.

Problem
If a method in a class is logging at the ERROR level, and an exception occurs, the log output is useful at that level, it contains a stack-trace, for example. However, there are more levels below ERROR that contain much more info that could be useful in maintaining the system. Levels are used to control the logging output in logging systems. For example, in java.util.logging there are seven default levels: SEVERE, WARNING, INFO, CONFIG, FINE, FINER, FINEST.

Solution
One way of getting at this useful information is by the class with the detected exception setting the log level in that problem method or function to a more verbose level. The algorithm would probably be similar to the Circuit Breaker design pattern.

Like “First Failure Data Capture” this approach could be called Nth Failure Data Capture.

Issues
Of course, while this may be easy to do this programmatically, in practice, this is not simple approach. Many questions remain: performance; resources; is one error occurrence enough to trigger a change; are all threads effected; which target level; how much logging, how to reset the level, and so forth.

Funny, I’m sure this is not a new idea. A quick search makes it look like it is not a well-known approach.

Alternatives

  • Record everything instead of logging just some things. This is possible with some systems, for example Chronon?

Links

JSON as a configuration language

JSON as a configuration data file is already a given. I wrote about this before. Google has now made this even more powerful by open sourcing Jsonnet, a “mini-language”, domain specific language (DSL) on this data format. Jsonnet it supports inline comments. Nice!

If your scratching your head and wondering why, JSON is JavaScript, then you don’t understand what JSON is. JSON is a ‘data-interchange format’. It happens to be based on a subset of the JavaScript language, but it could have been in Python, Groovy, or any other scripting language. Undoubtedly, JavaScript’s ubiquitous use on the web is why JSON was optimal.

Links

Relation generation rules from ER diagrams

A few general purpose purpose rules can help in creating relational database models. I found these rules years ago in a library and photocopied them. Of course, these are for very simple situations and not intended for a database professional’s use.

Rules

Rule 1

When the degree of a binary relationship is 1:1 with the membership class of both entities obligatory, only one relation is required. The primary key of this relation can be the entity key from either entity.

Rule 2

When the degree of a binary relationship is 1:1 with the membership class of one entity obligatory, and the other nonobligatory, two relations are required. There must be one relation for each entity for each entity, with the entity key serving as the primary key for the corresponding relation. Additionally, the entity key from the nonobligatory side must be added as an attribute in the relation on the obligatory side.

Rule 3

When the degree of a binary relations is 1:1 with the membership class of neither entity obligatory, three relations are required one for each entity, with the entity key serving as the primary key for the corresponding relation, and one for the relationship. The relationship will have among its attributes the entity keys from each entity.

Rule 4

When the degree of a binary relationship is 1:n with the membership class of the n-side obligatory, two relations are required: one for each entity, with the entity key from each entity serving as the primary
key for the corresponding relation. Additionally, the entity key from the 1-side must be added as an attribute in the relation on the n-side.

Rule 5

When the degree of a binary relationship is 1:n with the membership class of the n-side nonobligatory, three relations are required: one for each entity with the entity key from each entity serving as the
primary key for the corresponding relation, and one for the relationship. The relationship will have among its attributes the entity keys from each entity.

Rule 6

When the degree of a binary relationship is m:n, three relations are required: one for each entity, with the entity key from each entity serving as the primary key for the corresponding relation, and one
for the relationship. The relation from the relationship will have among its attributes the entity keys from each entity.

Use dojo's promise.all in non-failure mode

The “all” function “Takes multiple promises and returns a new promise that is fulfilled when all promises have been resolved or one has been rejected.” This makes a lot of sense. If you are invoking multiple asynchronous services in parallel and want to wait for all of them before continuing with a “reactive” function, you probably want to not run this function if one of the services fails.

In some scenarios you do want all the service invocations to complete and then deal with the results whether success or failure. Thus, it would be nice to have a version of ‘all’ that did this.

The now deprecated DeferredList may have supported this. See this reference.

Some() function implementation
Below in listing one I created a function ‘some’ that transforms the input to ‘all’ to accomplish this. This simply takes each promise argument (in array or object) and chains an always(…) Thenable. This Thenable promise link takes a handler that will be invoked whether the prior promise is resolved or rejected. It acts like a ‘finally’ in a try .. catch .. finally.

A better approach would have been to create an alternative to the ‘all’ function that did this.

/**
 * Takes an object or array of promises and creates a new array of 
 * promises that do not fail.
 * <p>
 * The Dojo/All function is fulfilled when all promises 
 * have been resolved or when one has been rejected.
 * In certain scenarios even if some promies
 * @author J. Betancourt 
 * @since 2014/11/04
 * @param objectOrArray  Object|Array?
 * @returns {Array} of promises.
 */
;function some(objectOrArray) {
    "use strict";
    
    var promises = [];
    require([ "dojo/_base/array", "dojo/Deferred", 'dojo/when' ],
      function(array, Deferred, when) {
        var origPromises = [];
        if (objectOrArray instanceof Array) {
            origPromises = objectOrArray;
        } else {
            for ( var key in objectOrArray) {
                if (Object.hasOwnProperty.call(objectOrArray, key)) {
                    origPromises.push(objectOrArray[key]);
                }
            }
        }
        // create array with each item a promise chain.
        array.forEach(origPromises, function(item){
           promises.push(when(item).always(function(res){return res;}));
        });
    });
    
    return promises;  // array of dojo/promise/Promise
    
} // end some(objectOrArray) function

Listing 1

Example
Below in Listing 2, this function is used. Available on jsFiddler.

<!DOCTYPE html>
<html>
<head>
<script src="http://ajax.googleapis.com/ajax/libs/dojo/1.10.0/dojo/dojo.js"  
    data-dojo-config="async:false"></script>
<script src="scripts/some.js"></script>

<script>
    require([ "dojo/Deferred", "dojo/promise/all",
	 "dojo/dom", "dojo/on","dojo/domReady!" ], 
        function(Deferred, all, dom, on) {
        "use strict";    
        
        function show(msg){
            dom.byId('output').innerHTML += msg + '<br/>';
        }
        
        /**
         * Use setTimeout to simulate an async process.
         * @param {string} name the name of this async invoke
         * @param {boolean} fail if true fail the promise
         * @returns {Promise} dojo/promies/Promise
         */
        function async(name, fail){
            var deferred = new Deferred();
            setTimeout(function(){
                if(fail){
                    deferred.reject(name + "Failed!");
                }else{
                    deferred.resolve(name);
                }
            },1000);
            
            return deferred.promise;
        }
        
        /**
         * On click of 'StartButton' button 
		 * wait for an array of promises. 
         */
        on(dom.byId("startButton"), "click", function() {
            var multiples = [async('V0'),
			 async('V1',true),async('V2')];
			 
            var p = all(some(multiples))
            .then(function(result){
                show(result);
                console.log(result);
            },
            function(err){
                // this should not be reached.
                show(err);
                console.log(err);
            });
        });
    });
</script>
</head>
<body>
    <button type="button" id="startButton">Start</button><p/>
    <div id="output" style="border: 1px black solid;width:20em;margin-bottom:20px;">Ready<br/></div>
</body>
</html>

Listing 2

Further reading

JavaScript focus event has wrong target?

I had an amusing debugging experience. Well, now it’s funny, then not so much. A focus event handler was getting events that said the same field was generating them. Even if other fields were selected.

I am using Dojo’s event API. This, like most other frameworks, make JavaScript event handling very simple, and most importantly handle the cross-browser issues.

The event tracer
I put a listener on a field’s ‘change‘ event using dojo/on, example:

on( theNode,'change',
      function(evt){
          mode = 'rent';			
      }
);

The callback handler just set a state variable. I also attached a listener to the ‘focusin’ event on a few other fields using the ‘on‘ method of Dojo’s NodeList:

query.NodeList().concat([moeNode,larryNode,curlyNode]).on('focusin',function(evt){
	evt.stopPropagation(); // needed?
	if(mode == 'rent'){
	    doSomeRentyThings(evt);
        }
});

This second handler used the value of the state variable. That is, I wanted to know that the user went from a particular field ‘A’ to another group of fields ‘B’. Hopefully someone can tell me the correct way to do something like this. Yes, a kludge, but it had to be done immediately, not on a single ‘submit’ button.

What should happen is that when the mouse leaves the first field (or user tabs away) and lands on one of the second fields, the status variable would have the correct value and be used by the second handler. It works!

Since the field setting is very fast, very few users could be faster. I know, async stuff; the browser could have a delay, etc. Don’t try this at home. Hmmm, but the browser is single threaded, so how could …. Anyway, in this app that would not be a disaster.

It stopped working
But, then I was developing something else on the same page using the Chrome debugger, and the above kludge stopped working. Huh? Even if I clicked on empty space in the browser, the event captured would be from the last field that the mouse was on. I re-read the Dojo docs, went back to the W3C stuff on Events, etc. Could find no web site where someone experienced a similar issue. I started blaming the Dojo toolkit, maybe there is a bug or the docs are incorrect.

I asked a co-worker, but they didn’t want to solve the problem just tell me that the app requirements were wrong, it’s too complex, blub, blub, blub. Most of the time feedback like this is important. One can get stuck in fruitless paths or need to see a different view of the goals. Perhaps, but in this case something obvious was wrong and I will probably face the same debug problem in the future.

Solution
Then I noticed that when I went from the Debugger back to the browser and clicked on another field, a very brief underline appeared in the previously selected field. Ah, how dumbskevisky, when I click on the browser anywhere, the previously selected field regains the focus and the focus event is generated. Obviously. The switch from debugger window to the browser window was the real change.

How did I confirm my observation? I closed the debugger window, and added console logging, like: console.log(“focus event on: ” + evt.currentTarget.name);

Take away

  1. Blame the tools last.
  2. Was working now not, what really changed?
  3. If it is hard to find a similar problem on web, your doing something wrong.
  4. When you ask co-workers for input be mindful of changing the context.
  5. Debuggers, while useful, can cause problems. The console is your friend.
  6. Take a break and work on something else.

What’s next?
There just has to be a better way of satisfying my web page requirements. No app at any point in time wanted to know what was previously changed?

Links

Groovy implementation of INIX file format, part 2

Continued experimentation with the INIX “ini” file format. I take the original post code and add an alias feature.

Background
In three prior posts I presented a very simple metadata file storage approach. Pretty much using the INI file format with the sections as “heredocs.” In original INI format, afaik, the data within the sections were properties only, x=y.

Updates

  • Dec 25, 2015: Just saw a mention of Tom’s Obvious, Minimal Language (TOML) This is not directly related to what I’m after with Inix, but interesting as an example of a simple markup language.
  • Alias
    Now I am working on adding the ability for a section of data to load data from another section. The sections to load are indicated by the ‘@’ symbol and then the section path. Multiple sections can be indicated. Though I’m using the term ‘alias’ for this, perhaps a better term is ‘importing’. So far, I can parse the alias from the tag string.

    I have not implemented the actual import. One complexity left to solve is recursion. If this section imports another section, what if that section imports others sections?

    Alias use case
    Since currently the Inix file format is being used for test data, aliasing allows reuse of data without duplication, i.e., DRY. This is problematic with hierarchical data like JSON or XML, but much easier with lists or maps. Further features like overriding and interpolation would be useful for Java Properties data. The goal would be to eventually support use of the Cascading Configuration Pattern.

    Example 1

    [>First]
    Data in first    
    [<]
    
    [>Second@First]
    Data in second
    [<]
    

    Now when the data in section “Second” is loaded, the data from the aliased section is prepended to the current section data:

    Data in first    
    Data in second
    

    Tag format
    The section tag format is now: [>path#fragment@aliases?querystring]. Note that unlike a URI, the fragment does not appear at the end of the string.

    The section ID is really the path#fragment. Thus, the end tag could be [<] or can use the section ID: [<path#fragment]. Example 2

    [>demo1/deploy#two@account897@policy253?enabled=true&owner=false]
    stuff here
    [<demo1/deploy#two]
    

    Grammar
    The start of a grammar follows, but has not been ‘checked’ by attempted use of a parser generator like Antlr.

    grammar Inix;
    section: start CRLF data end;
    start: '[>' path (fragment)?(alias)*('?' args)? ']';
    end: '[<' path? ']';
    path: NAME ('/' NAME)*;
    fragment: '#' NAME;
    alias: '@' NAME
    args: (NAME=NAME ('&' NAME=NAME)*)?;
    data: (ANYTHING CRLF)*;
    NAME: ('a'..'z' | 'A'..'Z')('a' .. 'z' | 'A'..'Z'|'0'..'9'|'_');
    
    TODO:

    1. Do the actual import of aliased section data.
    2. Allow multiple params per param key: ?measure=21,measure=34,measure=90. Or better yet, just allow array in arg string: measure=[21,34,90],color=red

     

    Implementation
    Source code available at Github: https://gist.github.com/josefbetancourt/7701645

    [expand title=”Listing 2, Implementation”]

    [/expand]

    Test class
    Note that there are not enough tests and the implementation code has not been reviewed.

    [expand title=”Listing 3, Test class”]

    [/expand]

    The test data is:

    [expand title=”Listing 4, data file”]

    [/expand]

    Environment
    Groovy Version: 2.2.2 JVM: 1.7.0_25 Vendor: Oracle Corporation OS: Windows 7

    Further Reading

    1. The Evolution of Config Files from INI to TOML
    2. Groovy Object Notation using ConfigSlurper
    3. Configuration Files Are Just Another Form of Message Passing (or Maybe Vice Versa)
    4. INI file
    5. Data File Metaformats
    6. Here document
    7. JSON configuration file format
    8. Creating External DSLs using ANTLR and Java
    9. Groovy Object Notation (GrON) for Data
      Interchange
    10. Cloanto Implementation of INI File Format
    11. http://groovy.codehaus.org/Tutorial+5+-+Capturing+regex+groups
    12. URI
    13. Designing a simple file format
    14. The Universal Design Pattern