Wednesday, August 5, 2009

Delivering Business Value Through Polyglot Systems (part 1)

My answer to Peter Ledbrooks's blog post "Are polyglot systems a good idea?" is "Yes!".

He wrote his post after reading "Polyglotism is the worst idea I ever heard" by Bill Burke. I don't know Bill Burke personally but the post has a troll-like feel. However, beyond the rhetoric there are concerns that I see raised over and over again. These concerns are nothing new. Almost three years ago, they were the inspiration for developing my presentation "Real World Groovy". In that presentation, I raise concerns about introducing Groovy into a Java project and I address each concern by relating my own experience having worked with Java and Groovy on paid projects.

Burke describes four "nightmares" in his post. These nightmares are based on the assumption that introducing another programming language into a project just a big excuse so the developer can learn and play with a new language, or for a language zealot/missionary to figure out a way to weasel in his pet language into a company. Plus, you’d probably end up being average or good at many languages but a master of none.
Personally, I never had trouble sleeping when using Groovy and Java on the same project. In this series of blog posts, I'll examine these nightmares one at time.

The "Maintenance Nightmare" concerns staffing and training. He describes the hypothetical situation that after the polyglot programmer has left the team,
...your group is a bunch of Java developers.For any bug that needs to be fixed, these developers need to be retrained in Ruby, a new Ruby developer needs to be hired, and/or a Ruby consultant/contractor needs to be brought it. Multiply this by each language you’ve introduced to your project.
Hypothetically, this group of Java developers won't be able to read and update the code written in another language without training.

My experience
is the exact opposite. The code written in Groovy was readable by the other developers on my team. No formal training was provided. The developers on my team were good, but they certainly weren't language zealots. One had about 8 years of DB2 & Cobol with less than one year of Java experience. Another developer had 10+ years in VB and various SQL databases, as well as some Java. The subject matter experts were insurance actuaries with programming experience in PL-SQL and Foxpro. On our project, all of these people were able to write and maintain the Groovy code without formal training.

How is this possible? Groovy wasn't added to the project because of some technical itch that needed scratching. The process of choosing an additional language involved taking a non-trivial piece of functionality and coding it in several languages, including Java. Each version was reviewed by those who would be developing and maintaining the new system. At that point in time, no one on the project had any experience with Groovy. If anything, there was prejudice against Groovy because of the uncertainty. Groovy was selected because it allowed us to easily express the algorithms and business logic in a way that reflected the business domain. The Groovy code was both concise and readable, with a very high signal-to-noise ratio.

Circling back to the maintenance concern, my experience has been that Groovy hasn't made maintenance harder. In fact, it has made maintenance easier. Even with significant time passing between authoring and maintenance, the Groovy code has been easy to comprehend because it reflects the business domain. I try to write my Java code this way as well. However, Groovy doesn't have a lot of ceremony around it, so it is accessible for developers with experience in other languages. My team did not struggle to decipher what was going on in a given piece of code because it was written in Groovy.

It is my experience that bugs can be resolved in minutes or hours that would have otherwise taken hours and days. The problems were solved quickly because those with domain expertise - my client's developers and subject matter experts - could see at a glance what was wrong in the code. Note that none of this required them to be masters of Groovy. I have heard similar stories from others who had seen performance improvements rewriting C modules in Lisp or Smalltalk. The performance gains came because it was easier to implement logical efficiencies that just weren't practical in C.

Based on what I've seen over the last 20+ years, technology issues are not what doom projects. I can't tell you how many times I've heard developers say, "If the client could just decide what they want, I could build it. I just need to know what to build." The hard part is getting systems to do what our client's intend them to do. In order to bring a system into production, the team must be able to reason easily and quickly through the code to fix business logic errors.

In summary, the readability and expressiveness of languages like Groovy make maintenance of polyglot systems easier. There is less code to write and maintain and it is more readable. The benefit to less code isn't in less effort typing at the keyboard. Rather, it is the ability to get one's head around the various parts of the system. In addition, this enables a broader group of people with business expertise to be able to understand what the program is doing. All of this helps avoid creating a big ball of mud. Keeping the code easy to read and understand is the key to avoiding the "maintenance nightmare".

In the next post, I'll address the "Refactoring Nightmare".


James E. Ervin, IV said...

Nice to hear from you Scott, maybe your example will inspire me to start writing some blogs again. It sure does take a bit of time to write a well composed and well thought out blog entry. Well at least it does in my case.

I noticed the article on polyglot programming and while the article could have done with less inflammatory comments at the beginning, he makes a series of valid points. I think if you notice my reply to that article in the comments section, you will see that you and he are not necessarily on opposing sides.

The fact is that there will not be 500 different JVM languages in use a few years from now, mainly for the reasons he lists in his blog. Those that survive are those that leverage new and effective programming practices with less boilerplate and those that facilitate the transition from normal Java language development to that new language. The languages that survive will be first class citizens in the Java platform, I think that Groovy and Scala will probably win in that regard. Yes, this means I think that jLisp (Clojure), Jython and others will never move beyond being niche languages. (The only exception I think that might hold true is JRuby, mainly because of the serious efforts to support it on the JVM and the large community it has on its own.)

This is for two main reasons I think. First, as you and I well know, it is hard to develop tooling for a language, so the *market* if you will will settle on a few languages to support. Those languages will get the cool tool support (read IDE support) and others wont. Second, the learning curve matters. It wont do to try to have part of your code base in jLisp or Ruby and the rest in Java. Why? Am I some non-polyglot ugly Javanian? Am I some kind of wuss that can't learn additional languages? No, its that why do I need to learn not only Ruby, but the Ruby idioms, when I can leverage something like Groovy that feels natural and I can focus on the coolness of the new feature that Java lacks? In other words, I try to be lazy in the good way, not that bad way.

I look forward to your new blog posts, because you and I know that Groovy does not fall into the trap that Mr. Bill states in his blog. In fact, I want to encourage something, don't call using Groovy with Java as polyglot programming, because it isn't really. I think that Groovy is not so much a Java language, but a Java dialect. A far more productive one.

Unknown said...

In retrospect I don't think Groovy fits into my concerns about Polyglotism. For one it is a superset of Java. It also uses the same class libraries as a Java application.

My blog was more targetted at Ruby, Python, Perl, Java, C/C++ where the installation, packaging, libraries, and syntax are different.