No Fluff Just Stuff

How toxic is your code?

Erik Doernenburg — Thu, 20 Nov 2008 18:54:00 CST

If you are somebody who writes code you probably know that moment when you look at some code you didn’t write, or some code you wrote a long time ago, and you think “that doesn’t look good.” Ok, more realistically, you probably think “WTF? I wouldn’t want to touch that with a barge-pole!” It is not even so much about whether the code does what it should do—that takes a bit longer to figure out—or whether the code is too slow. Even if it’s perfectly bug free and performs well, there’s something to the way it’s written. This is part of the internal quality of a software system, something that the users and development managers can’t observe directly; yet, it still affects them because code with poor internal quality is hard to maintain and extend.

Now, as a developer, how do you help managers and business people understand the internal quality of code? They generally want a bit more than “it’s horrible” before they prioritise cleaning up the code over implementing new features that directly deliver business value. Or even: how do you figure out for yourself how bad some code actually is in relation to some other code? These were questions that Chris Brown, Darren Hobbs, and myself were asking ourselves a couple of years ago.

The answer came in the form of a simple bar chart, arguably not the most sophisticated visualisation but a very effective one. And our colleague Ross Pettit had the perfect name for it: The Toxicity Chart. Read on to see what it is and how it’s created.

As usual, let’s start with an example. What follows is the toxicity chart for a version of the Hibernate framework.

In a toxicity chart each bar represents a class and the height of the bar shows the toxicity score for that class. The score is based on several metrics (more on that below) and the higher the score the more toxic the class. The individual components of the score are coloured. For example, the contribution of the method length metric to the overall score is shown in orange. This makes it possible to see at a glance not only how toxic a codebase is but also how the problems are distributed. If there is a lot of purple and orange in the chart, this indicates long and complex methods, which means that the code is probably hard to test on a unit level. Lastly, classes that score zero points are left off the chart.

The calculation of the toxicity score is based on metrics and thresholds. For example, the method length metric has a threshold of 30. If a class contains a method that is longer it gets points proportional to the length of the method in relation to the threshold, e.g. a method with 45 lines of code would get 1.5 points because its length is 1.5 times the threshold. The score for all elements is added up. So, if a class comprises two methods, one that is 45 lines and another that is 60 lines long, the method length component of the score for the class will be 3.5 points. This means that elements are not just classified as toxic but the score reflects how toxic the element is.

The following table shows the metrics that make up the toxicity score and the corresponding base threshold on which the multipliers are based.

Metric	Level	Threshold
File Length	file	500
Class Fan-Out Complexity	class	30
Class Data Abstraction Coupling	class	10
Anon Inner Length	inner class	35
Method Length	method	30
Parameter Number	method	6
Cyclomatic Complexity	method	10
Nested If Depth	statement	3
Nested Try Depth	statement	2
Boolean Expression Complexity	statement	3
Missing Switch Default	statement	1

At this point you might wonder where the selection of metrics and thresholds comes from. Well, we made them up. When we designed the toxicity chart we made a call on what constitutes “toxic” as opposed to just bad. Of course, staying with the method length metric, we normally wouldn’t want to see a 15-line method but that might be disputed. However, we hope that nobody thinks that a 30-line method is acceptable. And in case you’re really uncomfortable with the thresholds, you can obviously change them to build your own toxicity score. We do suggest, though, that you try the values presented here first.

Like many visualisations the creation of the toxicity chart falls into two steps: data acquisition and rendering.

Step 1: For Java projects Checkstyle is the easiest way to get the metrics. The score table above easily translates into a Checkstyle configuration file, which, by the way, is included in the Zip archive attached to this page. Assuming this file is named “metrics.xml” Checkstyle can be invoked like this:

java -jar checkstyle.jar -c metrics.xml -r <source dir> -f xml -o toxicity.xml

Afterwards, the file “toxicity.xml” contains a list of all components for the toxicity score in an XML format.

Similar tools for other languages should be able to create similar output. It is not a huge problem if the output is somewhat different as it’s always possible to change step 2 to handle different input formats.

Step 2: What is required next is to read the data from XML, aggregate it on a per-class level, and then render the bar chart. Microsoft Excel is good at the latter two but it needs a bit of help to read the XML file. This help comes in the form of a small piece of VBA code. The attached Excel workbook contains a sheet to load the data, an “Open XML” button backed by said VBA code, a pivot table to do the aggregation, and the chart based on the table. So, step 2, really is to open the Excel workbook and load the XML file. That’s all.

Now, if you know me you know that I’ve been working with Macs for a very long time. So, naturally, I’d like to use Excel on the Mac but, alas, in its current version it does not support VBA anymore. So, unfortunately, it’s VMWare Fusion for this one.

This Zip archive contains everything needed to create a toxicity chart for Java. The spreadsheet can be adapted to read input file formats created by tools for different languages; just edit the VBA behind the “Data” sheet.

Podjango

Mike Levin — Thu, 20 Nov 2008 12:42:00 CST

podjango: A Minimalist Django Application for Podcast Publishing: Podjango is a Django-based podcasting tool. Anybody taken a look? Of course, since we roll our own here at Swampcast, we're all ears!

Thanks to Jim White, semantic aficionado extraordinaire at Internet Foundation Classes eXtreme! for the shout-out

JarAnalyzer in Google Code

Kirk Knoernschild — Thu, 20 Nov 2008 11:01:00 CST

JarAnalyzer now has it’s own Google Code location. You can browse the source code online, check the source code out, and do all of the other exciting things that you can do with a subversion repository. The source in the google code repository is the same as can be found at the JarAnalyzer homepage (which is also where the binary is still found), except that the Google Code location also contains the JarAnalyzer XSLT.

Eventually, I hope to move the documentation over to the Google repository. Yeah…right! Update the documentation? I don’t think so!

Children and Microsoft Surface

Richard Monson-Haefel — Thu, 20 Nov 2008 10:06:20 CST

My wife is home schooling our 4 children - the oldest is 7. Although there is probably little or no market for Microsoft Surface applications for children I've recently become interested in the idea of using Surface for home education.

The idea of using Surface in this way really hit home when I started teaching my two oldest how to use a Mac to practice their spelling and for painting. I'm using an old G3 laptop so I don't care much if they break it.

Watching the children interact with the G3 has been interesting. The trackpad is challenging for them and clicking on small menu bars and icons is also difficult. They have gotten better, but the act of launching applications is still a mystery. Actually, the children have sort of lost interest in the computer and don't play with much now.

The loss of interest may be because of a couple of reasons. We have not exposed them to computers (other than an occasional session with my iPhone) until I gave them the G3. We also don't have a TV or video games so our kids are not used to interacting with a screen. The second reason is that its too tedious and the reward to low when working with the traditional keyboard and track pad. I'm convinced that a Microsoft Surface device would be a much easier and rewarding experience.

I was really pleased this morning to discover a blog entry by Richard Wand (a.k.a. Wandy) of Conchango who introduced his 4 year old, Bel, to the Surface this week. It turns out that Wandy had observed the same difficulty with his little girl when using tradition GUI applications. But when his daughter Bel started working with the Surface unit at Wandy's work she had a much easier experience. Wandy does a great job of documenting the experience and its worth reading.

Bel's experience illuminates a couple areas where Surface would need to be modified for children the first being the attract screen. I would imagine that its difficult to children to figure out that the corner access points did anything interesting. Also even if discovered I doubt kids that age would make the connection without accidentally hitting it a couple of times.

I'm excited about the prospect of using Surface to teach children and I plan to start designing applications for children as my first forays into Surface application development. After all, if you can design applications that are easy for small children to use than most adults would be able to use them too.

JavaOne 2009 Call for Papers Open!

Alex Miller — Thu, 20 Nov 2008 08:58:00 CST

Start your engines kids - the JavaOne 2009 call for papers is open…

Interesting new focus areas this year:

And here’s some good advice from previous years to keep in mind:

Last year was my first time both attending and speaking at JavaOne and while speaking to 1000 people was high on my list of terrifying life experiences, it was also a thrill and something I’d like to do again.

I suspect we’ll see a strong Java SE 7 focus this year as well, so should be fun. I was expecting to see the Java SE 7 platform JSR show up for last year’s show - I certainly hope it’s there in ‘09. Keep an eye out for my Java 7 preview article in an upcoming JavaWorld….

I’m on Twitter

Kirk Knoernschild — Wed, 19 Nov 2008 21:06:00 CST

I’ve jumped on the Twitter bandwagon. Possibly a little slow, but better late than never. I’ve started following a few people, and so far I find it fun and interesting. I intend to post mostly on tech stuff. I’ve also included my tweet feed on the right sidebar of this blog. Or you can subscribe to my tweet feed separately. Or you can start following me now! You decide.

Initial iPhone experience — disappointing

Brian Goetz — Wed, 19 Nov 2008 19:51:00 CST

I live in an AT&T-free state, so I have not had access to the cult that is iPhone. But recently, in preparation for AT&T moving into the state (through an asset swap that involves AT&T acquiring the VT GSM assets that Verizon bought in acquiring Rural Cellular), they are now willing to open accounts with VT addresses. So when in CA this week, I went to an AT&T store to plunk down my money so I could be cool like all my friends. I purchased a 16GB iPhone 3G.

I got out of the store and into my car, and noticed that the edge where the front metal rim meets the plastic case was extremely rough — almost sharp enough to cut. This was not the seamless tactile experience I was expecting from Apple. So I went back in the store, and asked for an exchange. I was told that “Apple prevents AT&T from making exchanges” and was sent to the Apple Store. When I arrived at the Apple Store, the rep informed me that they could make an exchange, but it would be a refurb unit, not a new one, even though mine was clearly new, because I’d bought it at an AT&T store and not an apple store.

So I went back to the AT&T store and argued with the manager. He tried to send me back to Apple. He ended up calling the Apple store, who must have told him to take the exchange, so in the end I got a new, non-defective phone. All was made right, but the experience was none too pleasant, involving three store visits.

While in the Apple store, which had many iPhones on display, I took the opportunity to do some sampling. I discovered that many iPhones had rough or sharp spots, and not all in the same places. Seems that in reducing the cost of the 3G, perhaps some quality-control corners were cut as well, since many were not very pleasing to the touch and there were significant variations in perceivable quality.

Ready for 5.0.17?

Howard Lewis Ship — Wed, 19 Nov 2008 19:15:53 CST

Looks like 5.0.16 will not be the final release, there will be a very modest 5.0.l7 that addresses a couple of annoyances that didn't have good work-arounds.

Android, G1 and Java

Mike Levin — Wed, 19 Nov 2008 09:27:00 CST

Tonight's GatorLUG topic calls out GatorLUG: "6:40 ~~7:45: Presentation~~ The Android G1 Cell Phone - What it does out of the box, How to write custom apps | Allen Rout"

A response: "To add,

I think Allen will be demonstrating using some Java programming skills

to make the G1 do new things (not unlike Brad Fitzpatrick, the creator

of LiveJournal, recently using his G1 + some Java to make a garage door

opener: http://brad.livejournal.com/2394707.html).

Cheers,

Martin Smith, Systems Developer"

So, Brad writes:

"This is a follow-up to my previous post to say:

SO. **ING. AWESOME.

I got it all working. I now have an Android Activity (GarageDoorActivity) which interacts with an Android Service I wrote (InRangeService), letting me start and stop the service's wifi scanning task. The service gets the system WifiManager, holds a WifiLock to keep the radio active, and then does a Wifi scan every couple seconds, looking for my house..."

Since my garage door opener is bigger than an Android phone, this appeals to me. You gotta read the whole post...

Yep, the times, they are a changin'.

UPDATE Allen reports he can‘t make it tonight, but here‘s a summary of his presentation:

+ Android is keen!

+ You can download eclipse and code for it directly – eclipse is keen too, and has emacs keybindings. – and svn integration. Woot!

+ let's code an app – hello world – make a button say hello world – make a button play a noise – read from the accelerometers, and display the output. – Whee, lightsaber! – coding model (callbacks)

+ rough spots in the developer universe – Docs not very introductory – Inconsistent APIs – Tasty looking APIs marked “Not ready for use” mostly by empty documentation. – Google seems to be swamped with code donations. (silver lining, that)

+ Conclusion: A great start. Not as kewl as Iphone, but profoundly more open.

- Allen S. Rout

Fast Sphinx indexing with foxy fixtures

Matthew Bass — Wed, 19 Nov 2008 09:24:00 CST

Can Sphinx and foxy fixtures place nicely together? Due to the way Sphinx indexing works, foxy fixtures will often slow down the indexing process drastically. This article explains how to overcome this limitation.

Printing an array in multiple table columns

Matthew Bass — Wed, 19 Nov 2008 09:21:00 CST

<% @categories.in_groups_of(2).each do |group| %>
  <tr>
  <% group.each do |category| %>
    <td><%=h category.name %></td>
  <% end %>
  </tr>
<% end %>

What are you staring at? Move along, move along.

OSGi interest via MarkMail

Kirk Knoernschild — Wed, 19 Nov 2008 08:39:00 CST

An image I swiped from MarkMail showing the increase in OSGi related posts on various mailing lists. No surprise that the most popular lists are Felix Dev and Spring-OSGi. The traffic shows the rise in interest in OSGi the past couple of years. Again, no surprise. It does appear, however, that most of the posts are closely tied to development of OSGi products (like Felix and Spring dm) and not from developers leveraging OSGi within their applications. OSGi hasn’t achieved deep enterprise penetration yet, and won’t until we get support from product vendors along with better tooling.

Centering images and text inside an li or div

Brian Pontarelli — Tue, 18 Nov 2008 19:23:00 CST

Found a good fix for this one as well. This will work on LIs or DIVs. Here’s how to do this in IE and FireFox:

li {
  display: inline;
  text-align: center;
}

li img {
  display: block;
  margin: auto;
}

The HTML looks like this:

<ul>
  <li><img src="..."/>Some text</li>
  <li><img src="..."/>Some text</li>
</ul>

Getting rid of spaces between LIs in IE

Brian Pontarelli — Tue, 18 Nov 2008 19:01:00 CST

Took a bit of research, but the only way I could get rid of all the space between LIs in IE was to make the LI inline and the anchor tag within the LI a block like this:

.some-class li {
  display: inline;
}

.some-class li a {
  display: block;
}

Grails vs. Rails: Are we seriously still talking about this?!

Jason Rudolph — Tue, 18 Nov 2008 14:27:00 CST

I had the pleasure of meeting a local Grails enthusiast last night who was new to the Triangle, but I was disappointed to see the same old argument spoil the show. The Rails vs. Grails debate is tired, and it’s curiously - perhaps even embarrassingly - unidirectional. I hereby call on the great philosophers to weigh in:

Philosopher, The First

Philosopher, The Second

Now, go in peace.

–

Update 2008-11-19 - Robert has deleted his comment (originally linked to above and referenced by Robert below) from his blog. His original blog post remains intact, sans incendiary comment.

Grails at the TampaJUG tonight!

Mike Levin — Tue, 18 Nov 2008 12:30:00 CST

Tonight TampaJUG are having a Grails workshop. I'll be helping out with the HomeVan here in Gainesville, so I won't be able to make it. But, it truly looks like a workshop not to be missed. Full details at Codetown.

If you make it, we‘d love to hear about it here at Swampcast!

Be the Change You Seek

Mike Levin — Tue, 18 Nov 2008 12:26:00 CST

I enjoyed meeting some Atlassian folk at the last Java Posse Roundup in Crested Butte, CO, which is about to happen again in March, 2009. You‘ll enjoy this video about core values, which can translate to many areas of life. Atlassian also have some cool underground advertising. They contributed to the design of the Java Posse Roundup T-Shirt and if you see one untucked, look at the backside on the bottom. That‘s where you‘ll see the Atlassian logo!

The XO Laptop - Give 1, Get 1

Kirk Knoernschild — Tue, 18 Nov 2008 09:46:00 CST

The OLPC has instituted their “Give a laptop. Get a laptop.” program again this year. If you’re not familar with the XO laptop, it’s a cool little deviced in a pretty small package. The picture at left shows the XO sitting next to a Dell XPS with a 15″ display (click to enlarge).

The OLPC offered the same deal around the same time last year when I ordered mine, and their was quite a backlog. It took almost six months for them to ship, so if you’re interested in getting your hands on the XO, it might be wise to place the order soon.

You can order your XO through Amazon. For $399, you give a laptop to a child in need, and also get your own laptop.

The Role of Architects and Architecture in Agile

Ryan Shriver — Mon, 17 Nov 2008 20:38:00 CST

I’ve returned from the Agile Development Practices conference in Orlando, FL and have uploaded my Agile Engineering for Architects slide deck and also the Impact Estimation tool used in the presentation. Thanks to all those that attended, I thought for the first time out this new presentation came off pretty well. We’ll see what the reviews say.

Some questions I was asked and conversations I had afterwards pertained to the question “What’s the role of Architects and Architecture in Agile?” The context is that in the Waterfall process, the role of Architects and Architecture is fairly well understood. Architects are heavily involved in the design phase and they’re expected to produce an Architecture that can be used in the Development phase. But as companies are moving to Agile, the architects in my session were asking me, “What’s my role now?” and “Where do I fit in?”

It’s questions that, honestly, I had never thought about before. I could tell by the tone of some of their questions that they had been told there’s not much of a role for them, and this left them a bit miffed. They had been told that agile teams “design on the fly” and continually refactor their system’s design. Better judgement told them this wasn’t right, but they didn’t know.

I know what’s worked form me when I’ve been the lead architect on a large agile project, but the thought of putting this into some best practices hadn’t really struck me until I was getting these questions. Maybe this would be something good for a future blog post.

So, what do you think? Any opinions on the role of Architects and Architecture in Agile?

Smart asset management for Rails plugins

Matthew Bass — Mon, 17 Nov 2008 15:08:00 CST

Many Rails plugins require that certain files like images, CSS, and JavaScript get copied to the public folder during installation. The typical way to do this is add code to install.rb, which then gets executed when you first install the plugin in your Rails project. What happens when the assets in a plugin you’re using change, though? You end up having to manually copy files around anytime the plugin gets updated, which is an extremely error-prone process.

I extracted asset_copier from an existing Terralien project. It fixes this problem beautifully. Once installed, it keeps the assets in your Rails project synchronized with your plugin. It even deletes assets from the Rails project that have been removed in the plugin.

To learn more, check out this post. If instant gratification is more your thing, install the gem directly and apply it to your target plugin:

sudo gem install pelargir-asset_copier --source=http://gems.github.com
cd ~/some_rails_project
script/generate asset_copier some_plugin

Generative Ambient Music: Bloom

Neal Ford — Mon, 17 Nov 2008 13:52:00 CST

One of the techniques I describe in The Productive Programmer is focus, distancing yourself from the all too common distractions in modern office environments. I suggest that you can wear headphones (or earbuds) while coding as an indicator that others shouldn't bother you. Some developers can code to music (in fact, some developers have a hard time not coding to music), but others fine it distracting. If you are in either group, you should try ambient music. The goal of ambient music is to create non-offensive background music. I find Muzak-style background music incredibly annoying and distracting, partially because they always choose sappy music (or taking good music and sappifying it) and partially because of a personality quirk described here. Ambient music strives to create a sonic background that not only doesn't distract but qualifies as good music. You can search for ambient music as a category on Amazon. Ambient music frequently uses techniques borrowed from Minimalist music, where you create a melody (which is too stong a term for some minimalist music) that repeats with minor variations.

Brian Eno is one of the pioneers of ambient music. He and Robert Fripp recorded the track The Heavenly Music Corporation on the album No Pussyfooting in 1972 by combining some experimental tape loop techniques from Eno, combined with Fripps guitar loops (called Frippertronics). No Pussyfooting is still available, and quite enjoyable. But the interesting part about it for this post concerns how it was created. Once Eno and Fripp had all the equipment set up, they started the tape loops and recorders for the background track and retired to Eno's front room and drank tea while the album recorded itself. No Pussyfooting has some early examples of generated ambient music (they were certainly not the first to experiment with generated music, but the first in a main-stream context). Eno and Fripp created several albums together. Eno has created a large catalog of ambient music (including the classic Music for Airports, recorded by several ensembles including the incredible Bang on a Can All-stars), and Fripp expanded the ideas behind Frippertronics using electronics for a stunning collections of recordings called Soundscapes.

Which brings me finally to Bloom. Bloom is an iPhone application created by Brian Eno and Peter Chilvers. When you launch Bloom, it starts a gentle drone sound as a background, and gives you the option to create ambient music by tapping the screen. When you tap, a note plays based on the location of your tap. You can tap single notes or chords. After about 5 seconds, the note repeats and continues while it gently decays. Using Bloom, you can create your own ambient music. I can play with Bloom for hours. It is one of the best iPhone applications I've found: a trully innovative application that takes great advantage of the iPhone interface.

The other option when you start Bloom is for it to generate music for you randomly. That brings us back around to focus. Using Bloom, you can generate unique, non-distracting music for an entire day. Rather than buy a collection of ambient recordings, you can generate your own ambient music. Bloom lets you set several "moods", changing the tonal range and drone sounds to add just enough variety in the music to keep it just interesting enough to not distract. What used to take tons of studio equipment now runs on the iPhone. This makes a great way to help enhance your focus in noisy environments. Allow Bloom to create music for you that doesn't distract yet effectively drowns out all the other distracting sounds in your cube-ridden environment. Bloom is a great example that the iPhone isn't just another mobile device: it's a new platform for software development.

Tapestry 5.0.16 (Release Candidate) it OUT!

Howard Lewis Ship — Mon, 17 Nov 2008 09:09:32 CST

The latest release of Tapestry, Tapestry 5.0.16 (Release Candidate), is now available.

Tapestry 5.0.16 is the release candidate; we encourage users to download this version. In about a month, the Tapestry PMC will run a vote to grant it release status, barring any blocker bugs (critical bugs with no workaround).

In the two months since the previous release, we've addressed over 80 issues, including many bugs and a few last minute improvements. New features include a LinkSubmit component (dearly missed from Tapestry 4), new support for reporting Ajax errors on the client side, smarter client-side validation, support for several new locales, and much new documentation.

Tapestry 5.0.16 is available for download, or via the central Maven repository.

St. Louis Lambda Lounge December meeting

Alex Miller — Sun, 16 Nov 2008 14:05:00 CST

The Lambda Lounge is a new St. Louis area user group focusing on functional and dynamic languages. The first meeting will be December 4th at 6 pm.

At this point, I think topics for the first meeting have solidified. I’m planning on kicking off the meeting with an open discussion about what functional and dynamic languages are and we can maybe arm wrestle about whether the two have anything to do with each other.

Next, Matt Taylor of ~~G2One~~ SpringSource will talk about Groovy MetaProgramming with Categories and Mixins.

Finally, Ryan Senior of Ferguson Consulting is going to do an overview of OCaml, of which I know not much, other than that it is a mixture of object and functional styles, derived from a ML-style static type system (but with type inference).

Should be a great December meeting. The tentative plan for January (on Wednesday the 7th due to the holiday) is for Mario Aquino to talk about functional programming in either Android or Dashboard widget development and for Mark Volkmann to talk about Squeak Smalltalk.

Hope to see you there!

Constraint, Chaos, Collapse

Michael Nygard — Sun, 16 Nov 2008 09:23:00 CST

Patrick Muellr has an interesting post about being brainwashed into believing that the outrageous is normal. It's a good read. (Hat tip to Reddit, whence many good things.) As often happens, I wrote such a long comment to his post that I felt it worthwhile to repost here.

My comment revolves around this chart of the Dow Jones Industrial Average over the last eighty years. (For the record, I'm not disputing anything about the rest of Patrick's post. In fact, I agree with most of what he says. This chart and my comments aren't central to his discussion about web development.) Some of you know that I've worked in finance before, and most of you know I have an interest in dynamics and complex systems. It's been an interesting year.

Here's a snapshot of the chart in question. It's from Yahoo! Finance, and the image links to the live chart.

Most of the chart looks like an exponential, which suggests the effect of compound growth. In a functioning capital-based system you'd expect exactly that. Capital invested produces more capital. Any time an output is also a required input, you get exponential growth. One of Patrick's other commenters points out that it looks almost linear when plotted on a logarithmic scale... a dead giveaway of an exponential.

No real system can produce infinite growth. Instead, they always hit a constraint. That could be a physical limitation on the available inputs. It could be a limit on the throughput of the system itself. In a sense, it almost doesn't matter what the constraint itself happens to be. Rather, you should assume that a constraint exists.

In systems with a chaotic tendency, the system doesn't slow down at all when approaching the constraint. In fact, it may be increasing at it's greatest rate just before the constraint clamps down hardest. In such cases, you'll either see a catastrophic collapse or a chaotic fluctuation.

I don't know what the true constraint was in the financial system. Plenty of other people believe they know, and I'm happy to let them believe what they like. Just from looking at the chart, though, you could make a strong case that we really hit the constraint in 1999 and the rest has been chaos since then.