In the Spotlight - Brian Pontarelli

I went to the 37 Signals event last night sponsored by CPB. The speaker was Jason Fried, who is a founder of the company, a designer and all around smart guy.

Jason spoke mainly about small business. He presented a list of things that companies can do to become more productive and better at whatever they do. His list included things such as:

• 4 day work weeks
• Virtual companies
• Saying no to your customers more often than not
• Do the easiest thing possible
• Don’t design, just do
• Don’t go to meetings
• Turn off email and IM

A number of these things I do each day including turning off email and IM when I’m really cranking, be as virtual as possible, reduce lengthy design and just start working. Overall, Jason has great ideas for small companies. 37 Signals is probably in the 3-5 million dollar range or lower, with about 2-3 million dollars of overhead or more. They are a life-style company and will not grow beyond a certain threshold because of these ideals.

Having worked for a large company, worked on vastly more complex systems than Jason, created many open source frameworks and now starting my own company, I found Jason’s ideas exciting and total crap at the same time. I regularly find myself battling with creating solutions that will scale, versus creating the simplest solution. This naturally occurs to anyone who has ever worked with applications that do hundreds of millions of transactions per day (or per hour).

I pushed Jason a bit to see where he would break ranks with some of his ideals. From his responses, he absolutely would not abandon these ideals at any cost. However, I think this is a marketing tool created by 37 signals to promote their products as well as Ruby on Rails. The reason I know this to be true, is that Jason is intelligent and capable or understanding moderately complex problems with more ease than say a college student. In addition, he spoke briefly about how a single feature request such as adding time-stamps to ToDo list items can have a larger impact than any customer might think. These features requests require a large amount of thought and planning because they impact the entire system. Similarly, changes to the Rails internals require a large amount of thought and testing to ensure they don’t break existing applications.

This dichotomy is something that is a fact of life. It is something engineers, product managers and executives must deal with, no matter how small or large your company is. 37 Signals has built a reputation that they can safely ignore these problems and be wildly successful. However, it is 95% smoke and mirrors. The reality is that 37 Signals is composed completely of phenomenal employees who can work in complex domains without losing their minds. The can distill complexity into manageable pieces and then combine those pieces in the end to produce a complex yet functional system.

Just read an interesting new I, Cringely about a piece of software named Talkinator. The technology seems just like a new IRC for the web with some APIs and Web 2.0 style, but that wasn’t what caught my eye. What caught my attention was that Cringely states that Taklinator doesn’t allow profanity. Being that Inversoft’s main product is a profanity filter, I had to try it out. It failed pretty badly.

Of course it catches your straight forward profanity like ass and fuck, but it does very little for anything more complex.

Here are some examples of stuff it doesn’t handle:

$hit
asslips
assface
phuck
fuk you
|)ick
sh it

That’s just a few, but the list goes on. I definitely think that people don’t understand the complexity that comes with attempting to filter profanity. It is much more difficult than it seems. I definitely think Talkinator should check out Inversoft’s Profanity Filter.

I have recently gone through the exercise of restoring a medium sized SubVersion repository from an older backup and wanted to share my experience with everyone. First, the problem:

The problem

After you restore the older backup, if any work was performed between the last backup and when the repository crashed, the repository will be “older” than what developers have on their boxes. Here is an example:

• Let’s say you have a backup from Monday evening
• The repository crashed sometime Tuesday
• There was work done during the day Monday

Now, let’s say there is a project A that looks like this:

• The last change to the project was at revision 100 just before the repository crashed
• The last change to the project from the backup was 80
• Frank’s computer contains a checkout of the project at revision 85
• Mary’s computer contains a checkout of the project at revision 100

This means that both Frank and Mary’s computers contain newer code than the repository, but not the latest code. Mary’s computer contains newer code than the repository, but might not be a complete snapshot of what version 100 was before the repository was corrupt. The reason why Mary’s computer might not be 100% correct is that Mary might have committed files to the repository but not performed a “svn update” prior to committing.

SubVersion is like CVS in that each file contains a version number. So, you might have a local checkout that contains version 100 for one file and 90 of another file. Therefore, you might be missing an updated version of the file from revision 93 when it was checked in.

Okay, now onto the fix:

The fix

Each developer’s computer must be analyzed before anything new is put into the repository. You must have a complete picture of the entire company, otherwise you might miss some changes. These changes can be merged in by hand from each developers machine, but this could be error prone and lengthy process. It is usually better to script out as much as possible.

Step 1

In order to determine the local “revision” of a project on a developers computer, you will need to look at each file in the checkout. You can run an ’svn stat’ on each file to determine the version number of that file. Write a script to output a file like this for each local checked out project on all developers machines:

src/java/foo.java: 90

The first part is the file and the second part is the revision of that file.

Next, once you have the complete list of revisions for all projects on all developer’s computers in the entire company, you can compare each file with the current revision in the restored repository to determine if the developer has a later version of a file than the repository. This should ignore all files that the developer has modified locally, but not committed.

This comparison will look like this:

Restored repository
src/java/foo.java: 80

Mary's checkout
src/java/foo.java: 90

Mary has a later version of foo.java!

You should script all of these comparisons out. If a developer doesn’t have later revisions than the repository or any locally modified files, they can safely take these steps:

1. Make a backup of the local checkout
2. Delete the local checkout
3. Re-checkout the project
4. Don’t do anything until the restore is complete

Step 2

The next step is to make a list of the revision that each file in the project was lasted changed on in the restored repository. This report will look like this:

build.xml: 10
src/java/main/foo.java: 16293
src/conf/main/logging.properties: 12093

Next, for each project, collect all of the revision reports from the previous step into a single location. These reports look like this:

frank-project-a-report.txt
build.xml: 10
src/java/main/foo.java: 16301
src/conf/main/logging.properties: 16405

mary-project-a-report.txt
build.xml: 10
src/java/main/foo.java: 16410
src/conf/main/logging.properties: 16321

Combine each of the developer reports into a global report that gives you the revision number for each file on every developers computer. Next, use this global developer report to determine whose computer contains the latest version for each file in the project. Based on the examples above, you can see that build.xml didn’t change recently enough to matter since the version in the repository is the same as the version on Frank’s and Mary’s computers. However, Frank made the most recent commit in revision 16405 to logging.properties and Mary made the most recent commit in revision 16410 to foo.java. Both of these files are more current that the repository and therefore need to be re-committed.

Step 3

Finally, setup a staging area that contains a copy of every checked out project from every developers computer who has a later revision than the repository (from step 1). This will be pretty large, but necessary. Based on the results from step 2, copy the latest version of each file over to a clean checkout of the project from the restored repository. Once you have all of the changes copied over for a single project, commit all of those files for that project back to TRUNK.

You should now have a fully restored repository based on the files from various developers computers.

Depending on the API you are using or how you are using SSL, you might have received an error stating that the certificate is invalid, not path to certificate, invalid certificate chain, no chain found, PKIK error, or something similar. This occurs when the certificate is self-signed or signed by an authority that has not been verified by the JDK you are using.

There is a simple way to handle this for self-signed certificates:

1. Open Firefox
2. Go to the site that is using SSL (i.e. https://svn.example.com)
3. Click on the lock down in the lower right corner of the browser window
4. Click the “View certificate” button
5. Click the details tab
6. Click the export button to export the certificate
7. Save the certificate in x.509 (PEM) format
8. Go to a command prompt
9. Add the certificate to the keystore

Here is the command to add the certificate to your global keystore:

*nix

$ keytool -import -keystore $JAVA_HOME/lib/security/cacerts -file <your-pem-export>
-alias <anything>

Windows

c:\> keytool -import -keystore %JAVA_HOME%/lib/security/cacerts -file <your-pem-export>
-alias <anything>

Found a good shortcut for getting access to hidden folders in OS X file dialogs and the Finder. It requires some typing and it doesn’t auto-complete like Linux does, but it is better than nothing. Just hit Shift-Command-G to open the “Go To Folder” dialog and then type the path to the hidden file or folder. The path can be relative.

Now, just need to figure out how to access hidden files from the Finder and file dialogs.

[Edit 08/09/2008] The issue is that using the AppleShowAllFiles configuration doesn’t work in dialog boxes and if you enable it, opening Finder in your home directory gets REALLY messy. What Apple needs is the same feature that Linux has had for a long time: the ability to show and hide hidden files in dialogs and finder using a keyboard shortcut or menu option. I believe Linux uses ctrl-h for this. Apple could use something like cmd-shift-h or something.