Jim Driscoll's Blog

Notes on Technology and the Web

Archive for the ‘DSL’ Category

DSL Slides from GR8 Groovy Conference

leave a comment »

Has it really been a year since I posted here?  Time to catch up.

I presented on the topic of DSLs at the GR8 Conference in Minneapolis during July.

As always, you can find the slides up on Slideshare:

Turtle Graphics in Groovy featuring the Napili program that I’ve blogged about here.

Groovy in the Cloud was a talk about what I actually do for a living, it was a refreshing change to actually be able to discuss it.

While the GR8 Conference was interesting, what really caught my attention was a talk entitled Grails for Hipsters – it was a pretty wide survey of the current state of web development.

All I can say is, wow, have things changed. In only three years, pretty much the entire landscape of web development has completely transformed.

Partly inspired by that talk, I attended The HTML5 Developer’s Conference in San Francisco in October. While phrases like “blew my mind” are certainly overdone, my mind was utterly blown.

More on that in my next blog entry. After a couple years of DSL work, it’s time to revisit the Web.

Advertisements

Written by jamesgdriscoll

December 7, 2013 at 10:45 AM

Posted in DSL, Groovy

Turtle Graphics DSL Implementation, part 2

leave a comment »

In the previous post, I described the first part of how Napili implements a simple Turtle Graphics DSL.  Please follow the link to download the whole program, if you want to see the code in context.  This time, we’ll cover the Binding class.

As I’ve talked about before, the Binding class provides default values to variables.  I’ve also previously discussed Property Dispatch, as well as a trick in using the Binding to work around the problem of optional parenthesis only being possible for methods with arguments.  I’ll go over all of that again here, but just in case you’re joining us for the first time, rereading those older posts might be handy.

The BasicBinding class in Napili serves several purposes – it creates the Turtle object, and serves it out to either the user directly or via the BaseScript (as I covered in the previous post).  It also handles the “out” Binding variable (more on that later).  And it handles no-arg methods on the Turtle object.  Let’s go through the class, section by section, and we’ll discuss how it does that.

private static class BasicBinding extends Binding {
  Turtle turtle

  public BasicBinding() {
    super();
    turtle = new Turtle()
  }

This part should be self expanitory – the constructor  creates the Turtle object which is used to draw, and saves that for later use.

The next method, getVariable, provides the Binding variables to the user scripts.

  public Object getVariable(String name) {
    if (name == 'out') {
      return NapiliOutput.getPrintWriter();
    } 

out is a special value for Bindings – it’s the value that’s used by the println() method.  So, by supplying our own value for it, we can redirect user println calls from going to System.out (and the console) and instead send the output to somewhere more useful and visible to the user. In this case, that redirection happens to an output tab in the UI.

    if (name == 'turtle') {
      return turtle;
    }

The turtle variable is used in the BaseScript file we covered last time.  It’s also present for the user to manipulate directly – which could be handy for advanced users who want to create two or more turtles.

    // treat all no-arg methods on Turtle as binding variables with side effects
    if (turtle.metaClass.respondsTo(turtle,name)) {
      return turtle.metaClass.invokeMethod(turtle, name)
    }
    return super.getVariable(name)
  }

The next method, setVariable, isn’t anything special – since all the code will stop working in a strange and mysterious way if the value of turtle is overridden, don’t allow that to happen.

  public void setVariable(String name, Object value) {
    if ("turtle".equals(name)) {
      NapiliOutput.println('Unable to set "turtle" to value' + value)
      return;
    }
    super.setVariable(name, value);
  }
}

And that’s it.  I’ve now gone over all the DSL magic in the Groovy Turtle Graphics program.  Hopefully you found this relatively clear – if not, don’t hesitate to ask questions.  

Written by jamesgdriscoll

November 3, 2012 at 9:20 AM

Posted in DSL, Groovy

Turtle Graphics DSL Implementation, part 1

leave a comment »

Last post, I described a Turtle Graphics DSL, as implemented in my sample program Napili.  (Please download it if you want to see the full code I’m discussing here).  This time, I’ll discuss how to implement it.  It’s surprisingly easy – though long enough that I’m going to tackle it in chunks, this is just Part 1.

First, to run the program, there’s the following snip of code inside a Java class (which is triggered when the user clicks the Run button):

Class clazz = Class.forName("org.netdance.napili.language.BasicRunner"); 
InvokerHelper.getMetaClass(clazz).invokeStaticMethod(clazz, "run", new Object[]{});

All this does is use Groovy’s dynamic invoker structures to invoke BasicRunner.run() – this code is only there so I can swap out the Groovy DSL classes at a later time, when I’ll discuss things like security.  I could have done the same thing with Java’s reflection classes, but if you already have Groovy in your class path, I heartily advise using InvokerHelper, there’s lots of handy stuff in there, and it’s considerably easier than using Java’s native reflection library.  One thing to be aware of, though, if you do:  it’s in the package org.codehaus.groovy.runtime, which means that the Groovy team reserves the right to change the APIs whenever they need to – which could make an upgrade of your Groovy version painful (though it’s not likely that InvokerHelper.getMetaClass(Class) is going to change anytime soon).

There are three classes that make up the language portion of the code.  The first, and by far easiest, is the BaseScript (you’ll remember, I hope, that I already discussed BaseScripts).  Written in Groovy, it becomes fairly trivial:

package org.netdance.napili.language
abstract class TurtleDelegateBaseScript extends Script {
    def methodMissing(String name, args) {
        turtle."$name"(*args)
    }
}

This uses a different method of Method Dispatch than I’ve previously discussed – he methodMissing method is called by Groovy whenever there’s no method found which matches a given name.  Since we don’t provide any methods at all, that means that pretty much every method called is going to go through this method.  (Since Scripts define other methods which will be inherited, like run and getBinding, and those won’t go through methodMissing, but that’s a problem for another day).  

The meat of the program is just a single line, in bold above.  I’ve already described what that kind of call does when I discussed Method Dispatch, but it’s worth going through one more time.  The first word, turtle, is interpreted as a Binding variable.   So the Groovy script goes into the Binding class (which I’ll describe shortly), and fetches the value corresponding to turtle.   Next, the “$name” is substituted with the method name called, which will then call the method name on the Turtle object.  Lastly, * is called the spread operator, and serves to break up the list of arguments into individual objects.   At runtime, a command like forward 100 will thus be transformed into something like: getBinding().getVariable("turtle").forward(100) – moving the turtle forward 100 steps.

The next portion is the BasicRunner itself – you’ll recall that we called the static public run method of that class, above.  It looks something like this:

static def run() {
  String scriptStr = Napili.code;
  def config = new CompilerConfiguration()
  config.setScriptBaseClass("org.netdance.napili.language.TurtleDelegateBaseScript")
  def ic = new ImportCustomizer();
  ic.addImports('javafx.scene.paint.Color')
  def ti = new ASTTransformationCustomizer([value: 15L],TimedInterrupt)
  config.addCompilationCustomizers(ic, ti)
  def shell = new GroovyShell(config)
  try {
    def script = shell.parse(scriptStr)
    script.setBinding(new BasicBinding())
    script.run()
  } catch (Exception e) {
    // error handling here
  }
}

In this method, first you fetch the script String to execute from the passed class.  Then, create a CompilerConfiguration object.  To the configuration object, set the BaseScript to the TurtleDelegateBaseScript just described, above.  Add a new import of javafx.scene.paint.Color, which will be applied to every script, so the user can say things like pencolor Color.PURPLE without having to do an import themselves. 

The next line, with it’s creation of an ASTTransformationCustomizer, looks a bit mysterious, but it’s effect is very straightforward – it sets a timeout on the script of 15 seconds.  After that, the script will throw an exception.  How it does this is a bit of an involved explanation, but we’ll have a lot more to say about AST Tranformations in a future post.

Then, as I’ve previously covered, just create a GroovyShell, parse the script, add a Binding (as I’ve also previously covered), and run the script.  If the user makes an error in the program, it’ll be caught as an Exception in the catch block.

The Binding itself isn’t terribly complicated either, but this is getting long, so I’ll continue on to describe the Binding section in the next post.

Written by jamesgdriscoll

October 13, 2012 at 1:38 PM

Posted in DSL, Groovy

A Turtle Graphics DSL

leave a comment »

In my previous post, I discussed the example program that I wrote to exercise the things I’ve described so far.  Today I’ll discuss what the DSL will actually look like – as with most programming tasks, designing your DSL before implementation is almost always advised.

There are a couple of different ways that you can use to describe languages, the most popular (and my favorite) being BNF, but for our purposes, that’s certainly overkill.  Instead, let’s just do this informally.

First, we’ll allow all existing Groovy operators and control structures – there are good reasons to strip those out on occasion, but this isn’t one of them.  We’ll cover how to do that at a later date.  What this means is that structures like while (condition) {block} and if (condition) {block} will work just fine.  We’ll also be able to create methods inline, just as you can with Groovy scripts.  And standard comments (i.e., // and /* */) will also work as expected.

We’ll need the following commands to create an environment that allows for turtle graphics:

  • Turtle / Movement Commands
    • forward number 
      • move the turtle forward by number of steps
    • back number
      • move the turtle backward by number of steps
    • right number
      • turn the turtle number of degrees to the right (clockwise)
    • left number
      • turn the turtle number of degrees to the left (counter clockwise)
    • home
      • return the turtle to the starting position, without drawing any lines
  • Pen / Drawing Commands
    • pendown
      • set the pen to draw when the turtle moves
    • penup
      • set the pen to not draw when the turtle moves
    • pencolor Color
      • change the pen color to Color
  • Animation / Demo Commands
    • show
      • show the turtle icon
    • hide
      • hide the turtle icon
    • speed number
      • change the speed at which the turtle draws, larger is faster
Keywords/methods are in bold, parameters are in italics.  Since we need to decide what number means, we can define it as a double, and since Groovy will auto convert int to double, we’re done with that.  Color is a little more problematic – ideally, we’d make it easy to use and understand, and have a whole bunch of code to do that, but since we’re currently just making a sample program, we’ll just use javafx.scene.paint.Color.
 
If you download the Groovy Turtle Graphics program Napili, you’ll see that all of these methods are already implemented in org.netdance.napili.Turtle.  I’ll talk about how to turn those methods into a DSL in the next post.  For now, here’s a sample program that you can run using these commands, along with the standard stuff that Groovy provides:
 
def circle(size) {
    45.times {
        forward size
        right 8
    }
}

speed 3
penup
forward 250
pendown
pencolor Color.PURPLE

12.times {
    circle 10
    right 30
}

home
hide

This program draws a series of interlocking circles, like you’d get from the child’s toy Spirograph.   We define a method (circle), set the speed a bit higher to make it all draw quickly, move forward without drawing to center the design, set the pen color to purple (PURPLE is one of many predefined values in javafx.scene.paint.Color), then draw 12 circles, turning 30 degrees after each.  At the end, we park the turtle in the home position and then hide it.  If you’re unfamiliar with Groovy, Integer.times(Closure) is a method that Groovy places on all values of Integer (and primitives are auto boxed to their class type in Groovy) – it just executes the closure as many times as the value of the integer – remember, parenthesis are optional for methods with arguments, so the circle method is really the equivalent of Integer.valueOf(45).times({forward size; right 8;}) – I trust you’ll agree that the format I used is actually easier to read, once you’re used to it.

See you next time, where I’ll talk about how to turn the methods on Class Turtle into the DSL above.

Written by jamesgdriscoll

October 6, 2012 at 12:46 PM

Posted in DSL

DSLs – What we’ve covered so far…

leave a comment »

At this point, we’ve covered many of the basics of creating DSLs in Groovy:

With that, we’re ready to tackle a real example…  Which I’ll detail in the next post.

Written by jamesgdriscoll

October 1, 2012 at 7:33 PM

Posted in DSL

Property Dispatch on Groovy Objects

leave a comment »

In my last post, I covered Method Dispatch on Groovy Objects.  The next topic is similar, Property Dispatch.

In Groovy, you can specify properties on objects in the same way that you can specify fields in Java:  With the “.” operator.  I.e., if I have an object named “Test”, and it has a class variable named “test”, you can access it with the phrase “Test.test” (just like Java).

As I already mentioned in my Introduction to Groovy, Groovy provides automatic creation of accessors and mutators (get and set).    That’s actually an oversimplification, but we’ll leave it at that for now.  As a result, consider the following Groovy class:

class GetSet {
   def firstField = 1
}

This can be used in the following way, inside a Java class:

GetSet gs = new GetSet();
System.out.println(gs.getFirstField());
gs.setFirstField("test");
System.out.println(gs.getFirstField());

That’s because, under the covers, void setFirstField(Object) and Object getFirstField() are added to the Groovy class.  Note that because the class field is declared as def, the type of these messages is Object.

While that’s handy in Java, in Groovy, we can do so much more with properties.  You can access them with dot notation:

def gs = new GetSet()
gs.firstField = 2.5
println gs.firstField

You can access them with get/setProperty (which is part of GroovyObject, the base object for all objects in Groovy):

def gs = new GetSet()
gs.setProperty('firstField','test2')
println gs.getProperty('firstField')

And you can even access a Map of all the properties on the object, and operate on that:

def gs = new GetSet()
println gs.properties.firstField

This last example is interesting since it’s using a few features of Groovy: the properties property is actually a shortcut to the getProperties() method, which returns a Map.  In turn, you can access the values of the map by using the key in dot notation.  So this is actually a shortcut for the following Java code: 

GetSet gs = new GetSet();
System.out.println(gs.getProperties().get("firstField"));

While all that’s handy, that’s still not dispatch.  In order to handle properties dispatch, there are two different methods we can use.  Consider the following code:

GetSet gs = new GetSet();
gs.newprop = 'new value'
println gs.newprop

newprop doesn’t exist in GetSet – we want to handle adding it at runtime.  The first method to do so is to override missingProperty(String) and missingProperty(String, Object).  The first method handles gets of properties that don’t exist, the second handles sets.  So the following code handles unknown properties seamlessly:

Map props = [:]
    
def propertyMissing(String name, value) {
    props[name] = value
}
    
def propertyMissing(String name) {
    return props[name]
}

In Groovy, Maps can be initialized to be empty with [:] and the values in that map can be accessed with the syntax pattern map[key].  So, in this code, if the property doesn’t exist, it will be added into an internal Map if you are setting it, and that value will be returned when the property is later used.  If the property is accessed before it’s set, a null is returned.  Of course, we could check for existence, and throw an exception if we desired to. 

This method will only add to the existing properties, however.  If there’s already a property (a class instance field, for instance) on the class, such as firstField in our example, this code is never touched.

If instead you want to completely override the access to fields, you can instead override getProperty(String)setProperty(String, Object) and getProperties() for good measure, if desired.  Here’s an example:

Map props = [:]
    
void setProperty(String name, value) {
    props[name] = value
}
    
Object getProperty(String name) {
    return props[name]
}
    
Map getProperties() {
    return props
}

This code actually does almost the same thing as the previous example, except that now, any instance or class fields are hidden from property access.    Meaning that the following code prints null:

GetSet gs = new GetSet();
println gs.firstField

But note that the following code will still print 1:

GetSet gs = new GetSet();
println gs.@firstField

This is because we’re using the “Java field accessor” in the second example – the .@  notation bypasses get/setProperty methods, and directly act on the underlying Java field.

Dispatch is just a short step away from the above code.  Instead of using the internal Map of properties, you could instead route the property lookup anywhere – an xml file, a database, or any other class or computation.

So, now that we’ve looked at method and property dispatch, we’ll run through some ways to use these in a DSL next time.  Until then…

 

Written by jamesgdriscoll

September 29, 2012 at 12:19 PM

Posted in DSL, Groovy

Method dispatch on Groovy objects

leave a comment »

 

As I promised my post about optional parenthesis, today I’m going to talk about method dispatch in  Groovy – specifically, dynamically dispatching methods on Groovy objects via the invokeMethod mechanism.  We’ll tie it into DSL creation in a later post.

The key method to know about for method dispatch is GroovyObject.invokeMethod(String name, Object args).  This method is located on GroovyObject, and is called when the method you called isn’t present on the object.

Consider the following Groovy code:

class Test {
  def invokeMethod(String name, args) {
    println "called invokeMethod $name $args"
  }

  def test() {
    println 'in test'
  }
}

def t = new Test()
t.test()
t.bogus('testing!',1,2,3)

 

This code prints out the following text, when run in groovyConsole:

in test
called invokeMethod bogus [testing!, 1, 2, 3]

the first call goes to the test method, as usual.  The second call, instead of resulting in a MissingMethodException, will route through the invokeMethod method.

This can be handy for all kinds of things, but the use I’ve put it to is as a dispatcher, routing calls to an underlying class or set of classes – here’s an example:

 

class One {
  static def test1() {
    println 'in test1'
  }
}

class Test {
  def invokeMethod(String name, args) {
    if (One.metaClass.respondsTo(One, name, *args)) {
      return One."$name"(*args)
    }
  }
}

def t = new Test()
t.test1()

This prints out 

in test1

Now, this example is flagrantly using a few tricks in Groovy, so let me mention how they work…  (Feel free to skip this paragraph if you’re content with this being a magic incantation.)   The first bolded line, with the respondsTo method, will retrieve the metaClass of the One class, which acts in a similar way to the Class class, but allows additional capabilities.  The respondsTo method of MetaClass takes, as it’s first argument, the actual object (or class, for static methods) that you’re checking, the name of the method, and then the arguments.  The asterisk in front of the “*args” argument is called the “spread operator”, and will derefrerence the array of args to instead be a list of arguments placed at the end of the method call.  The second bolded line is using the String substitution capability of Groovy’s double quoted Strings (called GStrings) to dynamically execute the method called name, again passed the arguments with the spread operator.  An additional wrinkle is that respondsTo actually  returns a ArrayList – which is treated as false by Groovy if empty, and true if not.  Now, you could do all this in Java, via reflection and other code, but the pain of doing so really makes this something you should probably do in Groovy.

There’s one other case we should probably cover – if your class implements the marker interface GroovyInterceptable, it won’t just intercept methods it can’t find – it will intercept all method calls on the class, including methods that are actually there.  That unfortunately means that we can’t directly call the method from within the invokeMethod though – instead, you must use the metaClass to invoke the method, getting around that interception.  Here’s how:

class One {
  static def test() {
    println 'in One.test'
  }
}

class Test implements GroovyInterceptable {
  def firstTime = true
  def invokeMethod(String name, args) {
    if (firstTime) {
      firstTime = false
      return One."$name"(*args)
    }
    return metaClass.getMetaMethod(name,args).invoke(this,args)
  }
  def test() {
    println 'in Test.test'
  }
}

def t = new Test()
t.test()
t.test()

 

This prints out:

in One.test
in Test.test

 

We’ll leave the explanation of how all this works for another time – the MetaClass is used pretty heavily in all kinds of Metaprogramming in Groovy, and we’ll cover it pretty extensively as we go on – but it’s use should be obvious from context.  Note that unlike our other use of args, we do not need to use the spread operator.  But keep the MetaClass in mind, because when it comes to adding methods dynamically at runtime, it’s the way to go.

Next up, we’ll talk about the related topic of Property Dispatch.

Written by jamesgdriscoll

September 29, 2012 at 10:30 AM

Posted in DSL, Groovy