Technically Possible

Getting Lombok to Play!

2011-04-26T20:40:00.000-07:00

The approach in this post works, but I created a Play! module that makes is easier. You can find it here: https://github.com/aaronfreeman/play-lombok

Recently I've been getting back in to using Play! for web development. I've built something small with it before, and it definitely is my favorite Java web framework. But there are still some things about Java that bug me. One of them is a complete lack of type inference. Static typing can be useful, but becomes a real annoyance if the compiler doesn't help you out a lot. In C# they have some basic type inference using the 'var' keyword and I have long wished Java would have something like that.

Then today I came across Project Lombok. It makes use of the AST transformations in the Java compiler to add features to Java. It can do some pretty cool stuff, and even integrates into your IDE so everything works as expected. And one of the features it has... yep, a 'val' keyword that does basic type inference. Nice! So you can do things like this:

val aList = Arrays.asList("Hello", "World");
for(val s : aList)
  System.out.println(s);

Behind the scenes Lombok turns the first 'val' in to "final List<String>" and the second into "final String". This is great!

One my first thoughts after seeing this was, can I get it to work with Play! Since most frameworks simple compile with javac Lombok is pretty simple to get working, you just have to have it on the compile class path. But Play! is a little different. It actually has a built in compiler (it uses the Eclipse compiler) and it compiles your code at runtime. This is great in development because it mean you and change Java code and just refresh our window and Play! will recompile the code and run the updated code. So no more restarting the server, or having to run in debug mode (which doesn't work for a lot of code changes anyway). It really makes things so much easier. But it also means in order to get Lombok to work, you have to get the Play! compiler to know about it. I couldn't find anyone saying how to do this, which is why I'm writing this.

I'll spare you the gory details of how I figured it out, because it took quite a while, but once I figured it out it's pretty simple to do.

First what you have to do it get Lombok into you Play! project. If you're using Play! 1.2, it's pretty simple (if you aren't yet using 1.2, you should really consider upgrading, lots of good stuff there). Just go to your dependencies.yml file and add Lombok. Like this:

require:
  - play
  - org.projectlombok -> lombok 0.9.3

Now this will work just fine, but if you're like me and you want the val keyword, you have to use the beta version of Lombok, which is in the maven repo. So you'll have to down load it and put it in your project. Easy enough, I put it in a jars folder under my project root and changed the dependencies.yml file to look like this:

repositories:
  - local:
    type:       local
    artifact:   "${application.path}/jars/[module]-[revision].[ext]"
    contains:
      - org.projectlombok -> *

Now we've got the jar, we just have to let Play! know about it when it compiles. I figured this out by reading what Lombok says it does to Eclipse to get it to work, and then duplicated it with Play! In order for Lombok to work, it has to be registered as a java agent, and be on the boot classpath. This can be done using JVM arguments -javaagent and -Xbootclasspath/a. The difficulty is in figuring out how to add these JVM arguments to the Play! startup process. The only way I know of is by making use of the jvm.memory config param in the applicaiton.conf file. This param just gets appended to the call to start the JVM, so you can really put any JVM arguments in there. So I added this line to my application.conf:

jvm.memory=-javaagent:lib/lombok-0.10.0-BETA2.jar -Xbootclasspath/a:lib/lombok-0.10.0-BETA2.jar

That almost does the trick. The only problem now is you'll get an error at startup saying it can't find the Eclipse compiler. That's because Lombok is now referencing it, but it's not at the right level in the classpath to be accessed. So we just need to add it to the bootclasspath, too. So now the arguments look like this:

jvm.memory=-javaagent:lib/lombok-0.10.0-BETA2.jar -Xbootclasspath/a:lib/lombok-0.10.0-BETA2.jar -Xbootclasspath/a:/opt/play/1.2/framework/lib/org.eclipse.jdt.core-3.6.0.jar

Now when you run Play! from the command line it will have Lombok available when it compiles your app. If you are using Eclipse and the eclipse launch configurations generated by Play!, you just need to add the parameters above to the VM arguments of the launch configuration and it should work fine.

Now go and use some type inference!

Overemphasizing technical practices

2011-04-20T12:11:00.000-07:00

I've been wondering lately if overemphasizing technical practices might be detrimental to a company in the long run. I'm not saying you don't need talented people, I certainly believe you do. I mean as a company (or department), making a big deal about whether a team/person does TDD, CI, or some other good practice. In the short run you'll probably get an increase in good habits among your developers. However, my concern here is that in the long run I think you're likely to create a culture that cares more about whether you're doing those practices than whether you're creating great products. Once a company switches to caring more about how it's doing things than what it's doing, problems start escalating rapidly. (Incidentally this one of the points that Jim Collins makes in How The Mighty Fall, great book!)

We have to remember that customers don't pay for or care about stories, TDD, CI, and many other good software development practices. They want, and pay for, awesome products! (Or as Kathy Sierra would say, products that make them awesome) I'm not saying you can't encourage good practices. You can! I just think you need to first make it clear that the goal is to build awesome products, and then let/help people gravitate to the practices that best allow them to achieve that goal. That way they are never confused about what is really important.

On Design

2010-06-08T22:58:00.001-07:00

The most important thing to learn about design is that it’s all about trade-offs. There are no perfect designs. Every design has a down side, some draw drawback that will cause difficulties. The is true for all kinds of design whether it’s a building, an organization, a user interface, or software.

Since all designs have problems, the designers job is not to get rid of all problems in the design, it’s to decide which problems are the least costly to have, or which problems are most likely to cause the least damage to the goal of the product. So, to do design well you have to be able to understand what the options are, what the issues are with each option, and in the particular context you are operating in, which issues are the least harmful.

It’s important to understand this when it comes to software design patterns. Design patterns are just common ways of trading one set of problems for another, with the belief that the new set of problems is better than the old. The only time it makes sense to use a pattern is when you have the particular set of problems it was designed to fix, and you’re better off with the new set of problems it leaves you with. But, in order to make good decisions on this, you’ve got to understand what the problem sets are, and how they affect your particular context.

This is not easy! And it takes experience to do it well. Often, the only way to know what the issues are with a particular design choice is that you’ve done it that way before, and seen what issues came from it. However, studying, practicing, and learning from others can take you a long way. The important thing is to never to be fooled into thinking that there are no drawbacks to a particular design decision.

Auto injection in jUnit

2010-06-01T22:04:00.001-07:00

After doing Java for many years, and TDD for several, I’ve settled into a fairly consistent way of designing and testing my classes. Any time you start to following a pattern in your code, if you’re paying attention, you’ll notice you’re writing the same code over and over. Now, it usually isn’t exactly the same or you’d easily just move it into a class and use the class. No, it’s usually code that is different every time, but it’s essentially doing the same thing, just with different classes. This is one example of what is generally referred to as boilerplate code. It’s aso one of the subtle forms of code duplication, and it’s something you want to avoid as much as possible.

I recently noticed that due to how I write my classes and tests, around 95% or more of my setup methods in my tests consisted of creating the class to be tested, creating some mocks, stubs, or fakes, and then injecting them into the class. Sure, technically every setup method was different, but this was boilerplate code for sure. And it was starting to get on my nerves. The setup usually looked something like this:

public class TestFoo {
   private Foo foo;
   private DependencyOne dependencyOne;
   private DependencyTwo dependencyTwo;

   @Before
   public void setup() {
      foo = new Foo();
      dependencyOne = new DependencyOneMock();
      dependencyTwo = new DependencyTwoMock();
      foo.setDependencyOne(dependencyOne);
      foo.setDependencyTwo(dependencyTwo);
   }
   ... then all the tests
}

One of the best ways to attack boilerplate code is by using the concept of convention over configuration. Basically all my setup methods were just configuration of the class I was testing, so what I needed to do is come up with a convention that made the configuration unnecessary, and some way to act upon the convention.

The convention starts with the @Target annotation. The idea is that you put this annotation on a field to signify that the field is the target of the test class that will need to have dependencies injected into it. You can also put @Target on a method, and whatever is returned from the method will be used as the target for injection.

Secondly, when you define a field on the test class if it matches a setter method on the target class, then it will be injected into the target using the setter. If no setter is found, but there is a field with the same name on the target, then the value will be copied to the field on the target.

In order to actually act on this convention I used the jUnit @Rule annotation. I called my rule AutoMockAndInject. If you aren’t familiar with how this annotation works, you can read about it here. I will put the code for my rule and the target annotation at the bottom of this post.

So, following the convention and using the rule my test class now looks like this:

public class TestFoo {
@Rule public AutoMockAndInject autoInject = new AutoMockAndInject();
@Target private Foo foo = new Foo();
private DependencyOne dependencyOne = new DependencyOneMock();
private DependencyTwo dependencyTwo = new DependencyTwoMock();

   ... then all the tests
}

One thing to remember here is that jUnit creates a new instance of the test class for each test method it runs. Otherwise, doing it this way could cause some problems.

I also use Mockito when I don’t want to make a hand written mock. The AutoMockAndInject rule works with the @Mock annotation from Mockito. It will create the mock object and inject it into the target. So if I want to use Mocito for my dependencies instead of hand written ones, the test class would look like this:

public class TestFoo {
@Rule public AutoMockAndInject autoInject = new AutoMockAndInject();
@Target private Foo foo = new Foo();
@Mock private DependencyOne dependencyOne;
@Mock private DependencyTwo dependencyTwo;

   ... then all the tests
}

I just started doing this a couple weeks ago, and so far I like it. It’s cut down on a lot of boilerplate code. But, in my experience, it usually takes at least a few months of doing something before you really see if it was a good idea or not. So, we’ll see if in the long run it really makes thing better.

Here is the code for my annotation and the jUnit rule I used for doing this.

@Retention(RetentionPolicy.RUNTIME)
public @interface Target {
}

public class AutoMockAndInject implements MethodRule {
   private static final String specialFields = "$VRc,serialVersionUID";
   private Object target;

   public final Statement apply(final Statement base, FrameworkMethod method, final Object target) {
      return new Statement() {
         @Override public void evaluate() throws Throwable {
            before(target);
            base.evaluate();
         }
      };
   }

   protected void before(Object source) throws Throwable {
      createMockitoMocks(source);
      if (hasTargetAnnotation(source))
      autoInject(source);
   }

   private void createMockitoMocks(Object source) {
      MockitoAnnotations.initMocks(source);
   }

   private boolean hasTargetAnnotation(Object source) throws Exception {
      return hasTargetFiled(source) || hasTargetMethod(source);
   }

   private boolean hasTargetMethod(Object source) throws Exception {
      for (Method method : source.getClass().getMethods()) {
         if (method.getAnnotation(Target.class) != null) {
            target = method.invoke(source);
            return true;
         }
      }
      return false;
   }

   private boolean hasTargetFiled(Object source) throws Exception {
      for (Field field : getAllFields(source.getClass())) {
         if (field.getAnnotation(Target.class) != null) {
            target = getFieldValue(source, field);
            return true;
         }
      }
      return false;
   }

   private Object getFieldValue(Object target, Field field) throws Exception {
      field.setAccessible(true);
      return field.get(target);
   }

   private Set<Field> getAllFields(Class<?> clazz) {
      return getAllFields(new HashSet<Field>(), clazz);
   }

   private Set<Field> getAllFields(Set<Field> fields, Class<?> clazz) {
      for (Field field : clazz.getDeclaredFields())
         if (notSpecialField(field))
            fields.add(field);
      if (clazz.getSuperclass() != null)
         getAllFields(fields, clazz.getSuperclass());
      return fields;
   }

   private boolean notSpecialField(Field field) {
      return !specialFields.contains(field.getName());
   }

   private void autoInject(Object source) throws Exception {
      ensureTargetExists();
      Set<Field> targetFields = getAllFields(target.getClass());
      for (Field field : getAllFields(source.getClass()))
         if (!callSetterIfExists(source, field))
            setFieldIfExists(source, targetFields, field);
   }

   private void ensureTargetExists() {
      if (target == null)
         throw new RuntimeException("Target value is null, did you forget to create it?");
   }

   private boolean callSetterIfExists(Object source, Field field) throws Exception {
      Method method = getMethod(target, getSetterName(field));
      if (method != null) {
         method.invoke(target, getFieldValue(source, field));
         return true;
      }
      return false;
   }

   private Method getMethod(Object target, String setterName) {
      for (Method method : target.getClass().getMethods())
         if (method.getName().equals(setterName))
            return method;
      return null;
   }

   private String getSetterName(Field field) {
      return "set" + StringUtils.capitalize(field.getName());
   }

   private void setFieldIfExists(Object source, Set<Field> targetFields, Field field) throws Exception {
      Field destField = getField(field.getName(), targetFields);
      if (destField != null)
         setField(target, destField, getFieldValue(source, field));
   }

   private Field getField(String name, Set<Field> fields) {
      for (Field field : fields)
         if (field.getName().equals(name))
            return field;
      return null;
   }

   private void setField(Object target, Field destField, Object fieldValue) throws Exception {
      destField.setAccessible(true);
      destField.set(target, fieldValue);
   }
}

Code duplication is evil

2010-05-27T21:16:00.001-07:00

As you’re programming, have you ever had the sense that a great evil was lurking in your code base? Well, most likely there is, and it’s name is: code duplication! This monster will sneak it’s way into your code with the promise of “fast” implementation and a “simple” solution. But make no mistake, once it has a foothold in your code base, what you thought was your friend will turn on you with a vengeance and destroy you. The code will rot in it’s place, and you’ll be cursed with recurring bugs that come back because you only fixed them in one place. Then your friends will laugh at your plight and make up nick names for you like “Mr. Duplication” or “Copy and paste Man.” In the end you’ll be left homeless and penniless. Then you will rue the day that you ever gave in to the subtle and poisonous promises of code duplication.

I’m not sure everyone sees it this way, though. I’ve said for a while now that the best way I know of to get to a well factored system is to have an acute aversion to code duplication. Both the obvious and subtle forms of it. However, despite the fact that the DRY principle is well known, I find that many developers (perhaps most) not only often repeat themselves in their code, but they also seem to be unconcerned when they find duplicated code and have to modify it. Personally I think the latter is the biggest issue.

We all write bad code some times, and we do stupid things like giving in to the temptation to duplicate. Many strange and terrible things can be done in the heat of the moment while programming and trying to implement a feature (even if you are pairing). To ere is human and it’s perfectly understandable. However, to come across blatant duplication and not only do nothing about it, but to be unmoved by it, that my friends is inexcusable.

Of all the code smells there are, code duplication is one of the most telling. So much can be learned from it, if you’re paying attention. Some times it can tell you that your design is not quite right. Or that you’re missing an abstraction. Or that you’re thinking about the problem in the wrong way. Often a lot of code duplication can be removed by solving the coding problem from a different angle.

There’s more that can be learned than just the few things I mentioned, but you will see none of these things if, when you encounter code duplication, you merely make the required changes in multiple places. You have to learn to hate it! You should be outraged by it. Don’t stand for it! When you see blatant code duplication, leap out of your chair, grab your keyboard and start slamming in on the table while yelling at your monitor, “NO! NO! NO! NO!” Sure, everyone will think you’re crazy, and I suppose you might even get fired, but after an out burst like that you’ll be determined to do something about the duplication. And, who knows, after a few of them, maybe others will be more careful about it, being frightened by what you might do next time.

When you see the evil of code duplication, don’t let yourself be unmoved by it. Be outraged if it helps, but do something about it. How long will we allow this great evil of our time to endure? As the saying goes: all that is necessary for the triumph of evil is that good programmers do nothing.

Good article on Gradle

2010-02-24T21:41:00.001-08:00

I happen to come across this great introductory article about Gradle:

http://www.javaexpress.pl/article/show/Gradle__a_powerful_build_system

It’s a lot better than the one I wrote, and should get you going in no time.

Gradle: building with bliss

2010-02-23T21:47:00.000-08:00

I’ve used Ant for years, and though the XML can get annoying, it’s a great tool. It’s so flexible, and once you understand it, you can do anything with it. Gant addresses the XML issue by introducing a Groovy syntax for writing Ant scripts. But one of the biggest issues with Ant, in my opinion, is the amount you have to write just to get a basic build that compiles some classes, runs some test, and builds a JAR or WAR.

One solution to this is Maven. Another nice tool, though it uses XML also. With Maven you can have a build up and running in minutes. But Maven takes a rather radical approach. With Maven, you don’t have a build file, you have a Project Object Model (POM). You use the POM to declaratively describe your project. Then all the Maven plugins use that information and fallow a build lifecycle to do the actual build. So, you don’t have a build file anymore, the build is just something that happens based on what you have in your POM. The Maven people are pretty big on the whole declarative POM describing your project. This line of thought is illustrated by a quote from the Maven AntRun Plugin page:

It is not the intention of this plugin to provide a means of polluting the POM, so it's encouraged to move all your Ant tasks to a build.xml file and just call it from the POM using Ant's <ant/> task

In other words, “don’t defile our beautifully declarative POM with your dirty procedural Ant scripts.”

The declarative concept is really neat in theory, but builds are a procedural process. So at times it feels to me like Maven is forcing something unnatural. And any time you want to add just a bit of logic to your build, you have to go through the effort of writing your own Maven plugin or do it in Ant and use the AntRun plugin. Seems like it should be easier than that, to me.

So, what I really want is a tool as powerful as Ant, as easy to add logic to as Gant, and as quick to setup as Maven. That’s exactly what Gradle is. It has tight integration with Ant, so anything you can do in Ant you can do in Gradle. It uses Groovy for the build scripts like Gant, so adding a bit of logic is easy and natural. And it has a build by convention concept (via plugins) like Maven that allows you to have a build running with a minimal amount of effort.

The most basic build

Here is the most basic Gradle build file you can have for a Java project. In your project root directory create a file called build.gradle and put this line in it

usePlugin 'java'

If you follow the Gradle convention (meaning you put your source code in src/main/java), this one line build file will allow you to compile and JAR your project. By default the name of your project that Gradle uses (and the name on the JAR) is the name of the folder that contains the build.gradle file. To learn how to actually run the build, check out the Gradle user guide.

Running some test

To actually run some tests, we’ll have to add some more too our build. Again, we’ll follow the Gradle convention of putting test in the src/test/java folder and change the build file to look like this:

usePlugin 'java'

repositories {
   mavenCentral()
}

dependencies {
   testCompile 'junit:junit:4.4'
}

With this build file we can now compile our code, compile our tests, run the jUnit tests, and create a JAR. Not bad for just a few lines of Groovy.

Changing the source locations

If you don’t want to follow the Gradle conventions, you can change the locations of your source files and test files. To do this you use the Source Sets concept from the Gradle Java plugin. Adding this code to the build file will change you source location to the src folder under the project root, and the tests to the test folder under the project root:

sourceSets {
   main {
      java {
         srcDir 'src'
      }
      resources {
         srcDir 'src'
      }
   }
   test {
      java {
         srcDir 'test'
      }
      resources {
         srcDir 'test'
      }
   }
}

There’s a lot more to learn about Gradle, and I may post more about it in the future. If you want to learn more, check out the user guide. But honestly the best thing I can say about Gradle is that I barely spend much time working with it. It’s so flexible and easy to work with that most of the time when I need to add something to a build file I can get in there add the logic I need quickly and get it all working in a few minutes, then get back to working on my software. Which is really what you want from a build tool because customers don’t care about builds! So you want a build tool that allows you to accomplish what you need as quickly and easily as possible, so you can get back to doing the work that your customers actually care about. To me, that’s where Gradle really shines.

Mercurial and Eclipse

2010-01-20T21:32:00.001-08:00

I’m not opposed to using the command line. Some times it’s the best way to get things done. But when it comes to source control I think having good IDE integration really helps with productivity. I’ve been in situations before where there wasn’t good integration, and, though you can certainly get things done like that, I end up fighting with the VCS more often. I think this is especially true when you are working in a team.

So one of the first things I want to know when looking at a VCS is how good the IDE integration is. Thankfully VecTrace has created a good Eclipse plugin for Mercurial. It’s designed to use an external Mercurial executable. So you’ll have to install something like TortoiseHg to do the actually Mercurial work. TortoiseHg is great and on the rare occasion where you can’t get the Eclipse plugin to do what you need, you can always use it, or the command line.

If you are familiar with either the CVS or SVN plugins for Eclipse, the Mercurial one should feel pretty familiar. It’s pretty much the same for the common things like marking a file that needs to be committed, the commit dialog, synchronizing with the main repository, and showing history.

One thing I really like about he Mercurial plugin is the intelligence it uses in marking a file as needing to be committed. With most plugins I’ve used in the past if a file is modified in any way, the IDE flags it has needing to be committed. Even if you undo the changes that you made, it will usually still flag it as needing to be committed. With the Mercurial plugin a file will only be flagged as needing a commit if it’s actually different from the latest version in the repo. So if you change a file, save it, then change it back, it will not show as needing a commit. It’s a small feature, but one I appreciate.

A lot of the time when you have to merge two change sets, the Mercurial will be able to merge them automatically. When there are conflicting changes, however, you’ll have to use the plugin’s merge manager. For the file comparisons it uses the same windows as the other plugins, but there is another view to show the merge status:

In this view you can double click the file with merge conflicts to resolve them. Then you can click ‘Mark resolved’ to mark the file resolved. You can also abort the merge from this view, or mark a file as no resolved.

The last nice view that the plugin has is the History view. Here is picture of the history from my Easyb jUnit project:

The history here shows each change set, who committed it, when it was committed along with the comment for the change set. From this view you can choose to update your code to any change set in the list.

One the left side of the view is the Graph column. This shows how each change set relates to the others. The graph for this project isn’t very interesting since I’m the only developer on this one. So, here’s one from a project with multiple developers:

This graph shows the branching and merging that went on in each change set.

One more nice thing with Mercurial that’s not related to the Eclipse plugin is bitbucket.org. This is the Mercurial equivalent of GitHub. We are currently using it for hosting our repository, and we’re pretty happy with it. It has a lot of nice features, good pricing plans, and allows you to see everything about the project through the web site. There’s also some nice integration with Hudson (which also has a Mercurial plugin) that allows the change descriptions for a Hudson build to link directly to the Bitbucket diff page. Which makes it really easy to se what changed in a particular build and who change it. Good stuff!

To sum it up, I’m really happy with Mercurial. It’s got speed, ease of use, good IDE integration, and a good repository hosting site. I highly recommend it!

Mercurial: the basics

2010-01-12T22:42:00.000-08:00

The first thing you have to understand when looking at Mercurial is what a Distributed Version Control System (DVCS) is, how it's different then things like Subversion or CVS, and why you would want to use it. So, let me start with a brief answer to those questions.

What and Why

Centralized version control systems like CVS, Subversion, Perforce, and the like operate by having a centralized repository server. Then each client does a checkout of the code from the centralized repository, makes changes, and commits their changes back to the server. A DVCS like Mercurial and Git differ from these in two main ways.

One, when a person wants to get the code form a repository, they don't just checkout the latest version. They actually copy (or clone) the entire repository. They actually pull down every revision of every file in the repo. This only has to be done once, from then on they just have to do an update to pull down any new changes to the repo. Then the person works with their local repo until they are ready to push their changes back to the main repo.

Two, the fact that everyone has a full copy of the repo means this is more of a peer-to-peer technology than a client-server technology. That is, with centralized version control it is clear that the server is the master repo, since it alone holds all the data. But with DVCS everyone holds all the data, so the only thing that makes one repo the master repo is that everyone working on the project decided it would be the master repo. But there is technically no difference between the master repo and all the cloned repos on other peoples machines. We'll talk about the effects of this more as we continue, but lets move on for now.

So, why would you want to use a DVCS? One of the things that I like about it is since it's peer-to-peer there's no server to run. If you want fancy things like web access to a repository, then you'll have to run some kind of server, but if all you want is a repository on your local machine or on your network, then all you need is a directory. All of the logic for a DVCS is in the program that runs on the client. So if you're working on something alone, but you want version control, DVCS is the easiest thing to setup. And if your on a large team of developers, it just so happens that DVCS will work really well for that, too.

With DVCS merging just becomes a way of life. But because of the way it works, merging is usually a lot easier than it can be with centralized version control systems. Also, since each peer has a full copy of the repository, it's like a bunch of automatic backups of your repo. Server crashed? Can't get your main repository back? Not a problem! Just have one of the peers copy their repository to a central location, and have everyone start pushing their changes to it.

Before I started using Mercurial I had heard a lot about DVCS and never understood why everyone thought it was so great. Now I've been using Mercurial for a couple months and there is no way I would ever want to use something like Subversion or CVS again. It is so much faster and better to work with. If you'd like to read more about why DVCS, check out Chapter 1 of Mercurials' definitive guide called How did we get here?

Mercurial vs GIT

Before I say anything on this, let me say that I agree with many others who have said that the important thing is moving from centralized version control to distributed version control. Deciding which DVCS to use is really just a matter of preference. Bot Mercurial and Git are great.

So, why did I go with Mercurial? To be quite honest it's mostly because I tried to get Git working on my machine once, and couldn't figure it out (probably because I didn't understand it). Then, a while later I tried to get Mercurial working, and got it working right away. I'm not sure if my problems with Git were normal, or just plain ignorance on my part, but the end result was I just stuck with Mercurial since then.

Git does seem to have more momentum around the development community, and it looks as though Eclipse might be embracing it as the sanctioned DVCS for Eclipse. But right now the Mercurial plugin for Eclipse seems much farther along that the one for Git. Also, the support for Git on Windows is fare behind that of Mercurial, and with the 139 Git commands it seems quite a bit more complex.

On a very technical side, the Mercurial definitive guide points this out, and I'm sure this is totally unbiased ;)

While a Mercurial repository needs no maintenance, a Git repository requires frequent manual “repacks” of its metadata. Without these, performance degrades, while space usage grows rapidly. A server that contains many Git repositories that are not rigorously and frequently repacked will become heavily disk-bound during backups, and there have been instances of daily backups taking far longer than 24 hours as a result. A freshly packed Git repository is slightly smaller than a Mercurial repository, but an unpacked repository is several orders of magnitude larger.

But, as I said, the important thing is moving to DVCS, not which one you choose.

Change Sets

One of the most important concepts with Mercurial is the Change Set. A change set is a set of files who's modifications where committed to the repo all at once with a commit command. Every change set has a globally unique identifier assigned to it so that Mercurial can always tell one change set from another no matter what machine it came from.

A change set usually has one parent, but will have two when you are doing a merge. A change set can never have more than two parents, though, which helps to limit the complexity of branches and merges in the version tree.

Commands

You use the clone command to clone a repository from the main location like this:

hg clone http://mysite.com/myRepo [destination folder name]

Once you have done this you now have a full copy of the repository. Under the folder you specified there will be a .hg folder. This folder contains all the repo history and information. Everything else under the main folder is called the working directory. (If you want to disconnect the folder from the Mercurial repo, you just have to delete the .hg folder.)

Once you have made some changes you execute the commit command to commit them to your local repository (this creates a change set). Then, once you are ready to share everything with the rest of your team, you do a push. This will push all of the change sets you have committed to your repo out to the main repo that you cloned from.

In order to update your repo with other people's changes you do a pull. This will pull in all change sets in the main repo that you do not have in your local repo. It's important to remember that this will not update your working directory. All this does is pull change sets into the .hg folder. In order to update your working directory to the latest version, you have to do an update.

If you do an update and the change sets you pulled are not descendants of the change set in your working directory, the update will fail, and it will tell you that you have to do a merge. Doing the merge creates another change set that represents the merging of two change sets. You now commit the merge and push the new change set back to the main repo.

That's the basics of how Mercurial works, but there's a lot more to learn. I recommend reading Mercurial: The Definitive Guide if you want to know more.

My next Mercurial post will be about using with Eclipse.

GWT and virtual toString methods

2010-01-11T22:58:00.000-08:00

I ran into something very mysterious today with GWT 2.0. It took me quite a while to track down exactly what the root cause was, but I've narrowed it down to one specific thing.

Let's say you create an object like this:

public class TestObject {
  private Object value ; 
     
  public void setValue(Object value) { 
    this.value = value; 
  } 
     
  @Override public String toString() { 
    return value.toString(); 
  } 
}

Then in your entry point you do this:

public class Gwt_test implements EntryPoint {
  public void onModuleLoad() { 
    RootPanel panel = RootPanel.get(); 
    HTML html = new  HTML(); 
    panel.add(html); 
        
    TestObject object = new TestObject(); 
    object.setValue(new  Object()); 
    object.setValue("Hello there"); 
        
    html.setHTML("<h1>" + object + "</h1>");
  } 
}

You would expect when you do the GWT compile and run your server and navigate to the page, you would see "Hello there" on the screen. Well, if you use Google Chrome, you'd be right. But if you use IE, Safari, or FireFox you'll just see a blank page.

Why is this? Well, it turns out to be quite complicated, and I only understand it up to a point. But this is what I have figured out so far.

You'll notice in the onModuleLoad method of the Gwt_test class after I create an instance of TestObject I set the value to 'new Object()' and then to a String. This is there so that when the GWT compiler generates code for the TestObject toString method, it has to take into account that the 'value' field may be an Object or a String. Which means it has to do a call to toString__devirtual$ in order to generate a string for the 'value' field. If you have GWT generate readable JavaScript you can look at it. The toString method for TestObject looks like this:

function toString_7(){
  toString__devirtual$(this.value); 
}

Now, this is the part that I don't understand. For some reason this code causes a JavaScript error in browsers other than Chrome. If you go to the page in FireFox and open Firebug, you'll see it has a JavaScript error that says, 'can't convert object to primitive type'. Kind of strange. I really don't get why this is happening, or why it works in Chrome, but I do have a fix.

The easiest way to fix this is to just change the toString method on the TestObject class. Instead of calling the toString method on the value field, just do a string concatenation. So now TestObject looks like this:

public  class  TestObject {
  private Object value ; 
     
  public void setValue(Object value) { 
    this.value = value; 
  } 
     
  @Override public String toString() { 
    return value + ""; 
  } 
}

And the generated JavaScript for the method looks like this:

function toString_7() {
  return this.value + ''; 
}

Now it will work in all browsers. I wish I could understand why the other code doesn't work. I looked at the toString__devirtual method, but didn't really understand where the problem was coming from. But at least I have a workaround.

An easyb jUnit runner

2009-12-19T14:17:00.001-08:00

I intended my next post to be about Mercurial, and I will get to it, but I had something else to write about. As I mentioned before, we are using easyb for our unit testing framework. It’s pretty nice, and I like it so far. It has an Eclipse plugin, but it just outputs to the console. Also, we are using Hudson for our CI server, and there isn’t really any Hudson integration for easyb either. Not having good integration was starting to bother me, so I decided to do something about it.

I don’t have an endless amount of time to spend on it, so I figured the path of least resistance was to create a jUnit runner that would run the easyb behaviors. That way the output would integrate into Eclipse and Hudson automatically.

It was a little tricky to get it to work properly because easyb tests are written in Groovy, and you can’t easily determine the makeup of a story or specification without actually running it. But I have something that works pretty well, and gives nice feedback, with good integration with Eclipse and Hudson.

I created a project on Google Code to host it, so you can check it out here: http://code.google.com/p/easyb-junit

I describe how to use it on the project site. I hope this is useful to other people. If you have any issues with it, or ideas to make it better, please let me know.

Technologies we are using

2009-12-14T21:55:00.001-08:00

My current project has given my team a great opportunity to use some newer web technologies. Prior to starting this project there were several things I had been looking at for a while and was interested in using. So, I’m going to list all the technologies we are using in this post and say a little about them. Then I’ll follow that up with a post about each one, and some details on what we are doing with them and things I’ve learned. These are in the order that we decided on using them.

Mercurial

In previous jobs I’ve use PVCS, ClearCase, CVS, and Subversion. I don’t really have anything to say about PVCS, except that I was glad when we quit using it. ClearCase was too heavy handed, and a pain to deal with. CVS works well, but has some major performance issues and is missing a lot of nice features of more modern VCS systems. Subversion does fix some of the issues with CVS, but every project I’ve worked on with Subversion, everyone ran into major issues when trying to merge code. Add to that some other oddities we ran into with Subversion, and I wasn’t very happy with it either.

I’d read a decent amount about the new distributed version control systems like Git and Mercurial. Although I never really understood why people liked them so much. I didn’t really get it until I read the first chapter of the Mercurial guide about distributed version control. I ended up deciding to go with Mercurial. The main reason I went with it over Git was the Eclipse plugin for Mercurial seemed a lot better. Git seems to be too tied to the command line. Not that I hate the command line, some times I use it with Mercurial, but I like good IDE integration with a VCS.

Gradle

I used Maven 2 back when it hadn’t been out very long. It was really cool how quickly you could get a build up and running with it. However, my team at the time had a bad experience with it. It cost me a few days of work a couple times, fixing weird build errors caused by plugins that got automatically updated. All those things that Maven just does for you can be a real headache when they aren’t working right. In the end, the issues we kept having with Maven lead us to switch to Ant and Ivy.

So, for the last four years or so I’ve been using Ant for builds and Ivy for dependency management. Ant is great because you can literally do anything with it, and I really like the way Ivy does dependency management. But it can take a lot of work just to get a build working. And all the XML can be a pain. GANT is a nice solution to the XML, but it still takes a lot of effort to get the build running.

A while ago I came across a new build tool called Gradle. It uses Groovy for writing the build instead of XML, so adding a bit of logic to your build is simple and straightforward. It also uses some of the build by convention concepts that Maven has, so getting a simple build running is a snap. It also has tight integration with Ant, so calling ant tasks is simple. It uses Ivy under the covers for dependency management, too. All in all, it looked like a nice option, so we decided to try it out.

GWT

I’ve kept track of GWT since it first came out. I’ve played around with it a few times, but never really got a chance to build anything with it. I really like the idea behind GWT. It’s not that I dislike JavaScript, I like it just fine, and jQuery is fun to work with. But I can’t really imagine building the entire front end of a rich internet application using JavaScript. It’s just too hard to test and debug, in my opinion. And I don’t really want to develop for Flash or Silverlight. So that leaves GWT. Which, to me is a great option.

Grails

I’ve done a couple small projects with Grails, and been pretty impressed. You can really get things done quickly with it. It takes some getting used to, and doing TDD feels different than Java. But once you get the hang of it, it’s a nice environment to develop in. We looked at a couple different options for the backend of our system, but decided Grails would be the best choice.

SOFEA

With the back end and front end in place, there was still a decision to make about how they would communicate with each other. A friend of mine told me I should read about SOFEA. It stands for Service-Oriented Front-End Architecture. The concept is that the front end is completely client side and controls the flow of the application. The back end is simply a bunch of stateless services that the front end calls.

Matt Raible is really big on this right now. He’s been using this architecture with GWT as the front end and Grails as the back end. We liked the idea of a clean separation between the front and back end that SOFEA presented, and had already decided to go with GWT and Grails. So it seemed natural to go with SOFEA also.

Easyb

The last thing we decided to use was Easyb. We were definitely going to be doing TDD, but we liked the concepts that the BDD style had. Although you can do BDD in jUnit, it doesn’t really feel all that natural of a thing to do, and it’s really easy to fall back into some bad testing habits. I was hoping there was a better solution out there.

I looked at JDave, but felt like it looked too verbose. I had heard of Easyb before, but never really looked at it. Once I checked it out I found that I really liked it. It uses Groovy instead of Java. Which adds some flexibility, and since we are using Groovy with Grails, it wasn’t another language to learn.

I found writing Easyb specifications is pretty similar to how I was doing TDD in jUnit, just nicer. I still use Mockito for mocking, and my tests are structured very similar to how they were before. They are just easier to read, and you aren’t tempted to fall back into the non BDD tests like you can be with jUnit. Also, Easyb has a nice HTML report that it outputs as part of our build. It’s a great tool! Well worth a look, if you haven’t seen it before.

Well, that’s it. That’s a lot of new stuff, but I think it’s already coming together pretty nicely. As I said at the beginning, I will be blogging about each of these technologies in the coming weeks. Until next time…

One line of code

2009-12-06T19:49:00.001-08:00

One is the magic number. At least it is for two things when it comes to writing code.

The value of really small methods

The first thing one is the magic number for is the number of lines of code in a method. I first heard this from Brian Button in a TDD class he gave a couple years ago. He was talking with someone else and asked them what they thought the optimal number of lines of code in a method was. I think the other person answered something like 10 or 15. “Nope,” Brian said, “it’s one!”

One line of code per method! That sounds crazy! It sounded pretty far fetched to me at that time, too. But Brian is a smart guy, so I thought I would give him the benefit of the doubt and really try it out. That was a couple of years ago, and I’d have to say that I’m definitely a believer now!

Now, like pretty much everything in programming this is not a hard and fast rule. You have to consider context and use judgment like with everything else. So, I’m not saying every single method should have only one line. That wouldn’t work. I just mean it’s an ideal to shoot for. The closer you get to it, the better off you’ll be.

I’ve worked with people in the past who said they believed in “self documenting code.” What they really meant by that is: “I don’t write comments.” They didn’t actually write self documenting code, they just didn’t document the confusing code they wrote. If a class is 500 lines long and made up of a bunch of 20 to 80 line methods, it’s not self documenting. Just because you don’t comment it and say it’s self documenting doesn’t make it so. Creating self documenting code takes intentionality, thought, and effort.

One of the biggest things you can do to make your code self documenting is to break it down into small methods. Methods whose size approach one line of code. If you do this, you will understand your code better. The next person that has to read it will understand it faster. And with good method names, you will have real self documenting code.

As I’ve worked at coding this way over the last few years, I’ve noticed a few things. One is that when I have to go back to a class that I wrote with really small methods, I rarely have to spend much time at all remembering how it works. I find that I can understand how it works much faster than I could when I wrote classes with big methods.

I’ve also noticed my definition of a big class or a big method has really changed. I used to work on a system that had classes that where thousands of lines long with methods frequently being hundreds of lines long. Now when a class gets to 100 lines, I start really feeling like I need to break it up. And I am rarely willing to put more than about 5 lines in a method. Quite a change.

Another thing that I’ve noticed is that shooting for one line methods helps reveal a lot of code duplication. The other day I was writing a class, and I had all the tests written and the functionality complete. Now it was time to refactor. So I started breaking the logic down into smaller methods. As I did this I noticed that two of the methods actually had a lot more in common than I realized. I was pulling several of the same small methods out of each. Then I realized that if I implemented another method they both used a little differently, I could collapse both methods into just one line of code. This change lead to a 10% reduction in the size of the class due to code duplication. Duplication that I wouldn’t have even noticed, if I hadn’t been creating all those little methods.

So, writing one line methods may sound extreme, and it is just an ideal, not a rule. But if you adopt it, I think you’ll find your code is easier to understand, easier to modify, and contains less duplication. That’s a nice combination.

But wait, there’s more

There’s another thing for which one is the magic number. I read a blog post a while back that I wish I could find, but I search around and couldn’t find it. In the post the guy was saying that if you are creating a new class, API, or feature you should not consider it done until it can be used/called with one line of code! This is something I wish people would think about more.

When you are writing something, think about the code that will have to be written to call your code. Is there too much setup for even simple uses? Can it be called with one line of code? Again, this isn’t a hard and fast rule, but it’s something you should shoot for. Especially for simple things that people may want to do with your code.

For instance, say you have a file and you want to read the lines of the file into a list of strings. Now, that’s a pretty simple thing and should only require one line of code. Well, this is how it would look using java.io:

I suppose that’s not too bad. It’s only about 6 lines, but that’s a lot more than one. This is a simple and common operation, there’s no reason it should take more than one line of code. Of course, if you have Apache IO Utils on the class path, it is only one line:

That’s the way it should be. I don’t really understand what the designers of this API where thinking on this one. I guess they got so caught up in making a flexible API that can do anything, that they forgot to make it easy to use for simple things. They forgot to make sure it could be used with one line of code.

And don’t even get me started on JDBC! Oh man, what a pain!! Look, if I want to query the database with a couple parameters and get a string out, that’s pretty basic, that should only require one line of code. If it takes more than one line of code with your database framework, then get a new one. I don’t even want to think about how many lines of code it would take to do it with raw JDBC.

So, the moral here is, when you’re writing a class or an API, think about the calling code. Refactor and add helper methods if you have to, but strive to make your code as easy to use as possible for clients. Don’t make them write the same 3 or 5 lines of code every time they want to do something basic and common with your code. Aim for one line of code.

After all, one is the magic number!

Using an ivy.xml with Gradle

2009-12-03T11:47:00.001-08:00

I’ve been using Gradle on my current project. So far I really like it. It follows the ‘build by convention’ idea that Maven uses and is even easier to extend and add to than Ant. One of the main reasons is the build file is actually just a Groovy file. So adding logic is simple.

One of the problems with any new tool, though, is wide spread support. Usually the IDE department is the biggest issue here. Gradle uses Ivy for dependency management, but it does it under the covers. You specify your dependencies in the build.gradle file using the Groovy syntax. It’s actually nicer than XML. The issue is getting your IDE to recognize the dependencies declared in your build file. There aren’t any plug-ins for this yet. This is no good as far as I’m concerned, because it makes adding jars to your project way too hard.

Today I decided I needed to add Mockito to my project, and ran into this issue of trying to figure out how to have it in my Gradle build file and on my Eclipse classpath. It annoyed me enough that I considered just dropping Gradle and going with Maven. Then I realized that since the build file is just Groovy, it should be pretty simple to have an ivy.xml file and just read it in the build file and insert the dependencies programmatically. It turns out this was pretty simple. Here’s how it worked.

In Gradle you specify your dependencies by declaring a dependency section in the build file like this:

You can also represent a dependency as a list of strings like this:

It’s important to note that all that is happening here is you are passing a list of strings to a method called ‘compile’ and ‘testCompile’. So, I realized all I have to do is write a function to build these strings off of the ivy.xml file.

Using the XML processing power of Groovy this is actually really easy. I changed the dependency section to look like this;

This code just parses the ivy.xml file and looks for dependencies that match the conf you specified (or have no conf at all) and adds them to the list of dependencies. It ignores some of the advanced features of Ivy, but it works for what I needed it to. Now I can just use IvyDE in Eclipse and have the same dependencies in my IDE as Gradle uses when it builds.

Customizing an Eclipse distribution

2009-11-28T08:25:00.000-08:00

When I do Java development I usually use Eclipse. When you’re working with a team, I’ve found it’s really helpful to make sure everyone is using the same version of Eclipse with the same plug-ins and settings. In order to do this you can just package up a few more things with the Eclipse distribution. Since I’ve done it several times now I thought I would document what you have to do. I thought others might find this useful, also.

Setting up the workspace

The first thing I like to do is to build a workspace that goes with Eclipse so I can setup all the settings in the workspace. Doing this for things like code formatting and templates ensures everyone’s code will look as similar as possible. There’s also a lot of other settings you may want to setup so no one else on your team will have to.
To do this just download the distribution of Eclipse you want to start with and unzip it (for this example I’m using Eclipse IDE for Java EE Developers version 3.5.1). Then start Eclipse. Since this is the first time you will be opening Eclipse it will prompt you for where you want your workspace to be. Just clear out want it in the box by default and type ‘workspace’ and check the box at the bottom that says ‘Use this as the default and do not ask again’. It should look like this:

Click OK. Eclipse will then create a folder called ‘workspace’ under your Eclipse folder. From now on when you open this Eclipse it will automatically use this folder as the workspace. Now you can install all the plug-ins you want and setup all the settings you want and they will be carried with this distribution.

Packaging the JDK

Another thing you may want to do is package a JDK up with the distribution so you know for sure that everyone is using the same one. Also, this means that Eclipse will run even if the person does not have any version of Java installed on their machine (this does introduce a platform dependency, though).
The easiest way to do this is to find a JRE you have already installed on your machine and just copy it into a folder called ‘jre’ under the Eclipse folder. When you run the eclipse.exe file it looks for a ‘jre’ folder in the Eclipse folder, if one is found, then it will launch using that JRE. However, this is often not preferable because some Eclipse plug-ins (like the Maven 2 plug-in) require you to be running Eclipse on a JDK not just a JRE. So it’s best to use a full JDK. You could just take a JDK and copy it into a folder named ‘jre’ under the Eclipse folder, but that could be misleading, and there’s another way.
Take a JDK you have on your machine and copy it to a folder called ‘jdk’ under your Eclipse folder. Also in that folder there is an eclipse.ini file that is used for setting certain parameters to the exe and the Java VM. Open that file and put these two lines at the top:

-vm
jdk/bin

This will tell Eclipse to use the VM you put in the ‘jdk’ folder.
Now you can just zip up the Eclipse folder and give it to your team.

Bonus: Bundling Gradle with the distribution

For my current project we are planning to use Gradle for our builds. I wanted to be able to bundle it with the Eclipse distribution and setup an external tool configuration so it would be easy to use for anyone else.
To do this I downloaded the latest version of Gradle and put it in a folder called ‘gradle’ under the Eclipse folder. Then, in Eclipse, I created an external tool configuration like this:

This will work fine as long as the person that uses it has their JAVA_HOME set to a valid JRE. But we’d rather have this work without that variable set. The gradle.bat file specifically looks for the JAVA_HOME variable to be set and will error out if it’s not. You can set this value to something using the Environment tab in Eclipse, but I couldn’t find any way to get this to work in a way that would function properly on everyone’s machine. So I ended up just modifying the gradle.bat file and adding a line that sets the JAVA_HOME variable to the ‘jdk’ folder under the Eclipse folder. The modified file looks like this (the added line is in bold):

...
set DIRNAME=%~dp0
if "%DIRNAME%" == "" set DIRNAME=.\
set JAVA_HOME=%DIRNAME:~0,-12%\jdk
...

It’s a bit of a hack, but it works. If anyone has any better ideas, I’m surely open to it.

Passion for your craft

2009-11-19T20:42:00.000-08:00

I've been thinking about starting a blog about technology and business for while, but never actually took the time. I decided now was a good time to do it. However, for my first post, I'm really just going to direct you over to the Software by Rob blog. If you don't read it already, you should consider it. He had an excellent post today about Passion as a Competitive Advantage. I thought it was great and truly believe in the importance of being passionate about what you do.

In the article Rob says:

Passion translates into being insulted when people don’t care about things as much as you do and are willing to hack a crappy solution together. It’s an insult to you, your product and your craft.

One thing that passionate people love is other passionate people. When you really care about what you do, you want to work with people who also care about what they do. The last thing you want is to have to work with someone who is just trying to put in their forty hours a week. And, as Rob puts it:

If you don’t have passion for your code/product/startup everyone will know. It’s obvious you aren’t that into it, and people will not take you seriously. Without passion it’s impossible to convince people to believe in your vision.

I started out in college in Chemical Engineering. I made it through my third year and was only 30 hours away from graduating. Toward the end of my third year I was sitting in a principles of chemical engineering class, and the professor was chewing us out for not putting much effort into our latests assignment. We were doing a design and analysis of a distillation column, and a lot of us (myself included) didn't do a thorough job and just turned in a half baked solution. Not surprisingly the professor, who was passionate about his field, was practically insulted by our lack of effort.

Then the professor said something that changed my path in school. Actually, he asked a question. He asked, "Why would you turn in something poorly done, if you care about what you're working on? And if you don't care, why are you getting this degree?" I sat there and thought, "That's a good question! Why am I getting this degree?" The was effectively the end of my career as a Chemical Engineer.

The summer before that I had taken a C++ class that was required to complete my ChemE degree. I really loved it. I did more than the class required, and would often just program things for fun. In one of my ChemE classes we had to write a program to solve a problem. The professor actually supplied a mostly complete program in some strange math environment and said we just had to complete it, if we wanted to do it that way. Everyone else did that, but I wrote my own from scratch in C++. I thought it would be more fun that way. Incidentally, mine was the only one that worked correctly.

So, when that professor asked us why we were getting a degree in ChemE if we didn't care about it, I decided that Computer Science made more sense. This was something I did really care about, and something I loved doing. And the rest, as they say, is history.

I graduated in 2001 with my CompSci degree and still love programming. It's been great getting paid to do something I love to do. And I've enjoyed all the time I've spent reading, practicing, and learning more about software development.

After eight years I believe even more in the importance of passion for what you do. I love working with passionate people, and I'm always baffled by people who don't seem to care. They should have had a professor like I did and had to face the question, "if you don't care, why are you getting this degree?" If you aren't doing something you are passionate about, then do yourself and everyone else a favor and go find something you are passionate about. You'll be glad you did.