Rockstar Programming

A couple months ago I joined a new company. They are a management consulting firm. So of course they let everyone get business cards. The idea is that you would hand out your business cards to prospective clients and get new business for the firm. I thought I would apply to get some business cards of my own even though I am a programmer. I thought I would choose a title of “Rock Star Programmer”. The other choice I was mulling over was “Programming Rock Star”. The firm let’s you choose the title line for the business card. They do say that they will reject inappropriate titles. I figure my official title is just too boring for a business card. So I might need to get some authorization for the funky title.

Here is the kicker. I am not going to run out and get new business for our firm. I am going to write tight code and keep the customer happy on my current project. The only reason I am going to get business cards is to give to my friends and family. Yeah they will only cost the company 20 bucks. But there will be a lot of perceived good will on their part if they move ahead and give me the Rock Star title on the card. Now whether or not I am actually a programming rock star is a different story. I am, in fact, not a programming rock star.

I did a little research on the programming rock star phenomenon. I found the best description of such an individual at ReadWriteWeb. There was a post there by Alex Iskold. He said the rock star programmer:

• Loves to code
• Gets things done
• Refactors code
• Uses design patterns
• Writes tests
• Uses existing code
• Is concerned with usability
• Writes maintainable code
• Can code in any language
• Knows computer science

I score about a 6 out of 10 on this scale. Unfortunately that is still a failing mark. That is not a crime. It just means I am not a rock star programmer. However only the chosen few can honestly claim the title. I just slowly work on improving my skill set. And I think it would be fun to have a business card that falsely labels me a rock star.

Build Script Environment

Our development team turned our code over to configuration management this week. It is CM’s job to run the official builds we sent to test. This year we upgraded to new versions for many of our tools. So the build scripts had to be updated. We also have annual changes to the build scripts based on our source code control and other yearly dependencies. The build script updates were performed by three of the developers on our team. That sounded like an upgrade because we only had two people working on the scripts last year.

The main CM guy was about to go on vacation. So he hurried up and tried to build our 5 applications. He had all kinds of problems. And he kept asking for help. I felt bad because this guy is a buddy of mine. So I provided whatever assistance I could. In the end he declared that 5 install packages had been built. Then he left for vacation. I asked one of the script developers to include a new DLL in one of the application installs. This seemed like a good opportunity to see if he successfully did what I asked. I tried to install our application and the thing bombed. I tried some of the other installs and found them lacking. I immediately went and told the software development manager that we were in trouble. He told me to go visit the backup CM guy the next day and see if I could make things work.

So I headed to our office on the other side of town. The backup guy never seemed to have done a build before for our project. So I gave him some pointers. The first build completed but was no good when I tried to install it. The second build completed, and installed, but would produce errors when you try to run the application. I spent a few hours try to debug these build problems. Every time I would fix something, it would uncover a whole set of new problems. This was crazy. As it was getting late into the afternoon, I called the original developers that made the changes to their scripts.

Each guy assured me that they tested the build scripts. The only problem was that they only tested them on their own development machines. It seems that the first guy to test it on a fresh machine was the CM guy. Of course these scripts were not going to work. There were all kinds of problems. Many files required were missing from source code control. Other files were in wrong directories. Crucial steps that the builds need to perform did not work correctly. Other input that the build scripts required were not correct in source code control. It was a miracle these things worked even on development machines.

After a long day with little to no success, I called the software development manager. I told him we were deep in problems. Then I said I was going home because there was zero percent chance of me making these scripts work that day. It was already getting late in the night. The next day the manager called all the script developers to one location and had them trace through the problems. I felt like I got set up. At least do one test in a new environment if you are mucking with build scripts. This should be fundamental.

Defect Injection

The topic of code or peer reviews are sometimes on my mind. That’s why I enthusiastically read a blog entry on the CoffeeGhost web site. The entry was entitled “Bugging Your Code Reviews”. It was written by Al Sweigart. He had found that a lot of requests for code review got no feedback other than “everything looks good”. So he invented a tool called Jester which randomly injects what he calls Jester bugs into the code. This coincides with an idea that I pitched our software manager. So this blog post is very relevant to me.

Jester is officially a unit tester for Java. It injects bugs in the code that have correct syntax, but are nonetheless problems. Al runs the buggy code through his unit tests. This helps him validate whether the unit tests are truly testing the code. He also uses Jester to inject bugs into the source code before he passes it on for code review. The buggy code will compile fine. But the injected bugs cause run time errors.

Al played devil’s advocate and envisioned the potential drawbacks of using Jester to test your code review process. Injecting false bugs may slow down the process of detecting legitimate bugs. And if developers are not careful enough, they might check in the bugged version of the source code. Reviewers who got caught slacking off on their code review duties may get angry that they were duped. Finally reviewer may learn how Jester works and look just for the Jester style bugs.

As always I love to read the comments of other people. Some people who read Al’s blog post thought that code reviews were in essence a waste of time. Others who used a code review tool thought it would be nice if the tool itself supported Jester style bugs. One way that could make the defect injection process fun is to keep metrics on who finds the most bugs. There could be an online top scores for code reviewers to spurn them on to deep reviews. Finally some people in the commercial software world though that code reviews in general were not for them.

Unfortunately my project at work is almost exclusively written in C++. However that does not mean I couldn’t code up a version of Jester that works with C++. The idea is sound. My manager said that defect injection in general sounded like a good idea. The only problem we have is that it seems like nobody is doing actual code reviews any more. Ouch.

Hands On Architecture

I read an essay by John Davies entitled “Architects must be hands on”. It made a lot of sense. John says that a software architect needs to be able to lead by example. The architect should be able to do any of the duties on the team. For example, the architect should be able to configure the build or write unit tests.

The architect needs to be more than a project manager. The architect needs to understand both the technology and the business. The project manager, on the other hand, must take care of the day to day mundane tasks of running the project.

An architect has to be able to demonstrate clearly in order to be effective. Architects should be brought onto a project early. When faced with an important decision, architects should confer with their architect peers. The architect should also have expertise in at least one tool, preferably the Integrated Development Environment (IDE).

A lot of programmers and developers strive to be an architect. Architects sit at the top of the totem pole. It is a good title to have on your business card and resume. Theoretically the architect is making more money than the rank and file person on the project. The problem is that not everybody can be the architect. That being said, it would behoove an architect to follow the advice shared by John.

I think about my own career path. Having worked on the same project for many years, I have become an expert in the business of our client. This gives me an extreme advantage over other developers. Most of the time, the hard part of our job is figuring out what the client wants. It is rare that the difficult part is the technology side of the house. In fact, given our specific client and project, you have to know a lot of legacy technologies to be good at development.

Two strong developers in the past have been promoted to the software architect position before. Now this was mainly a way to not increase their pay, but persuade them to stay on the project. Don’t get me wrong. These were sharp guys. They were strong in technology. And they were top developers. Come to think about it, they were well respected by their peers. Perhaps I should follow on in their footsteps. One went on to be a technical director. The other reverted back to a simple software development role.

Sake of Technology

I stumbled upon a home page for a list of things every software architect should know. The number one item was an essay written by Nitin Borwankar entitled “Don’t put your resume ahead of the requirements”. Nitin said that architects are called on to recommend technologies for clients. He cautioned good architects to choose technologies that would benefit the customer, not the architect’s resume. You should always choose the best solution for the project. This policy will benefit you in the long run as you will have less stress on the job and happier customers.

Nitin cautioned that his principle did not mean you should not keep up with emerging technologies. However you should learn that stuff on your own time. If you truly want to use a hot new technology, you should search out a project where that technology would fit best. You should not try to use that technology on your current project and customer just to beef up your resume.

I found this principle to be sound on the surface. However I got the feeling that in this day and age, your ability to get a job through normal channels requires you to have all the good buzzwords on your resume. And recruiters don’t want to see this as a result of home projects. They want actual work experience in the field. This coincides with a shorter term perspective in the software industry. You often are judged by whether you know the latest hot tech. That makes it hard to make good decisions when looking out for your own interests on a project.

Now I know that the best way to get jobs is through personal networking. That being the case, it would behoove you to have successful projects and happy customers so that you get good references. Maybe this is what applies more to a more senior individual at the software architect level.

Personally I have seen great disasters when some new technology was forced onto a project I was working on. One time it was as simple as moving to Windows NT 4 as the target operating system. The project was already in a tough schedule. Then the architect decided it would be good to have NT 4 experience on his resume. We found some of our main code to break when switching to the new OS. Disaster ensued. This example is not a lone one. Time and time again some hot shots in charge chose a new and unproven technology. The result was long hours, and usually a project that got the plug pulled.

I am ashamed that I have been on more than one of these in my career. Both times I was not the software architect. It might be time to step up to the next level. Things may fare better if I have this golden principle under my belt. Thanks Nitin for your timeless essay.

COBOL in California

The governor of California has declared that all state workers need to revert back to minimum wage until the budget is finalized for the upcoming year. There is one fatal flaw in this plan. The California payroll system was written a long time ago in COBOL. The developers are saying that they cannot make the pay cut changes quickly. Now this may be a big joke. If I had to implement a change that would make my salary minimum wage, I would take a few years to implement it. However there may be some actual truth to this story.

In general it seems tough to get good COBOL programs. A buddy of mine has made a career out of consulting to federal, state, and local governments. He comes in and does COBOL code maintenance for them. And he charges a pretty penny for it as well. The state of California looked into overhauling the system. The selected vendor quoted a price of $177M to do the job. And nobody knows whether that will be the actual price, or whether there will be cost overruns as there normally are.

It is not too uncommon for the state of California to keep such an old system alive. My own project is based on interfacing with a big monolithic system written in COBOL for mainframe computers. Yes there is a contract to modernize the code and rewrite it in C++. That project has had a lot of pain. The contractor is using a lot of subcontractors that work long hours and charge big bucks to support the effort. Nobody knows whether the project will eventually be a success or not. The problem is that the client cannot keep the old timers around to maintain the COBOL and even assembly language code.

There were a lot of interesting comments on the California payroll story. Maybe the California payroll needs to be outsourced to a company like ADP. It is agreed that the state probably already pays too little to the COBOL programmers. And you cannot forget that COBOL itself is a 40 year old language. Somebody posed the question that, if wages are dropped to minimum wage, what would happen if normal deductions take the net pay less than $0? It is quite funny that the COBOL programmers have the ability to code in their annual cost of living increases. Some die hard hackers said any programmer worth his salt should be able to pick up the COBOL programming language. However they people may not want to work for state wages. They definitely will not be willing to do so for minimum wage.

Days of VisiCalc

A couple days ago I read the most interesting post on Bhanwara’s blog. He is a guy from India. In the 1980’s he got a job at Software Arts. He worked on the VisiCalc project. This was programming on the early IBM PC. His initial task was to get the program to fit in 256k of memory. He was able to do this in a couple months. So he then went on to tackle some of the bugs submitted for the system.

Here is where Bhanwara got into a little trouble. Most of the guys did not like to do maintenance programming. So they took their time when fixing bugs. They might correct a bug every couple days. And they did not strive to tackle the hard problems. However Bhanwara was hungry to knock out the bugs. So he averaged around 5 bug fixes a day. At first they thought there was no way he could sustain this pace. They thought he was not actually fixing the bugs. Then the quality assurance people checked his fixes. And they were good.

It is generally not a good thing to make the other guys on your project look bad. I know this from personal experience. It does not matter what your intentions are. If you make the other people look bad, you are going to be in for a lot of pain. This was especially the case for Bhanwara. He was the new hire and the young guy on the project. The developers got together and labeled Bhanwara an idiot savant. Yes he was real productive in knocking out the bugs. But they called him dumb and a freak and said he was something akin to the Rain Man there.

Bhanwara had some good insights into both VisiCalc and the Software Arts company as well. VisiCalc was written in a dialect of Lisp which made it slower. The competition was Lotus 1-2-3 which was much faster since it was written in assembly language. Bhanwara himself wrote code in assembly language when writing new graphics code for VisiCalc. Eventually Bhanwara left the company because it was a bad scene all over. They other guys had it out for him. And the company and the product were headed for extinction.

The real gem of the post was that Dan Bricklin decided to comment on the story. Dan is the cofounder of Software Arts and the inventor of VisiCalc. Bhanwara himself was pleased that Dan posted a comment. Dan is something of a legend having created the first spreadsheet computer application. You should check out Bhanwara’s blog post. Good work man.

Going to the Web

There has been a desire by my customer organization to migrate our client/server application to a web environment. I think they are tired of rolling out our Windows application to hundreds, if not thousands of desktops. They have a whole system administration team who is responsible for dealing with application installation. These guys have a lot of better things to do with their time. Moving to a web based system may help alleviate this type of problem.

My background is developing client/server applications on Microsoft Windows using Visual Studio. Specifically I have worked with Visual C++ for over 10 years. Although I have done some Oracle PL/SQL back end programming, most of my experience is desktop applications. The problem is that I really like the current system I am working on. I really spent a lot of time learning the business of my users. So I do not want to throw away all of this business domain expertise. The solution for myself may be to move to the web along with my project.
The question is where do I start to become a good web developer. Another contractor already tried to rewrite the system for the web using Java. I do not know a lot about that effort. I did some back end PL/SQL work for them that had to do with complex business rules. But I do not know a lot about the details of the technologies used. In general, what type of topics should I learn about? I checked out a local training vendor. Here were some interesting course topics that seemed relevant for this change:

• Developing Web Applications
• JavaScript
• Cascading Style Sheets (CSS)
• Java and XML
• Java Enterprise Edition
• Struts
• Spring and Hibernate
• Ajax

Do any of these seem like good investments in the future? My company allows some budget for training. I am looking to make the business case that they need to invest a little more in my training. The result will be that I can help them make it to the web in a safe manner. The hard part of our system is hardly ever the technology. It is the business side. What kind of courses do you recommend I take to get to be that Java web developer that is essential for my company?

Maintenance Haters

I was reading a software development discussion board this week. One person posted a rant about how he hates maintenance programming. He said the code he was forced to work on had absolutely no comments. There was a lot of redundant code. The code base used a number of different frameworks. He thought they were at the point where they needed to dump the app. His boss did not agree. So he eventually quit. I find it interesting that this scenario closely resembles the project I am working on. I was glad to see a lot of comments to this person’s entry. I am going to take the time here to review some of the finer points of maintenance programming.

Just because code is old, that does not mean it is bad. It can be used as an opportunity to hone your skills. You have a great opportunity to perform code refactoring. You can strive to remove all the warning you will get when you compile the code. Maintenance coding is the norm after all. Some people put the split as 20% new development versus 80% maintenance coding. A strong recommendation would be to initiate good unit tests if you do not already have them. If some of the code is bad, just rewrite that portion and ensure you did not break anything.

Working on code maintenance is not a waste of your time. You can find many examples of what not to do by looking at the code base. There will be plenty of puzzles to solve while fixing tough bugs in the system. It a lot of code is bad, you have the liberty to delete lots of code. You can use the situation to your benefit as well. Not many people want to do maintenance programming. And there may not be many people who do it well either. So you can potentially make more money doing it. There can be other perks as well. You can ask your boss for flexible location or work hours. And you can ask for 20% of your time be spent to explore new technologies. What kind of good manager would refuse any of these reasonable requests?

Maintenance may be, in fact, the hardest type of programming. It is a type of programming that will dominate the software work. This is because it is almost always more expensive to build a new system than maintain the old one. You will not always be just fixing bugs. You will probably be updating the legacy code to meet new business needs. Even if you work on some new development, the project will quickly turn into a maintenance phase. Somebody wise once said that “all programming is maintenance work”.

It is true that in maintenance work, most of the design decisions have already been made. And it will be tough to upgrade to the latest tools. This is due to the risk involved with such an upgrade. My own project neglected this for many years. Then it took a lot of pain to update to newer database clients and code compilers. In maintenance you can expect to spend a lot of time reading code. But as you can see, this may not be a bad thing.

Business of Software Conference

Previously I had blogged about the Software Development Best Practices conference. I was disappointed by the difficulty in finding out the pricing, and more frustrated that the pricing structure was confusing. So I thought I would compare it to another software development conference I have heard about. This one is the Business of Software conference. It is a Joel on Software conference. I found out about it while I was reading Joel’s blog (Joel on Software).

I went to the Business of Software web site. Joel did not send me any print marketing materials for his conference. I clicked on the register link and got to the pricing page. In one sentence it stated the pricing:

The full price for the conference two-day pass is $1,795, but if you book before August 22nd then you can book for only $1,595.

Now that’s what I call a clear message. There are only 2 options. And it is very clear what the 2 levels are. In addition, it was easy to find the page with the pricing. Now I would expect nothing less from Joel. He is a stickler for good design. Let’s move on to some other points that might influence my decision.

There were a total of 4 “famous people” speaking at the conference. These were Joel Spolsky, Seth Godin, Eric Sink, and Richard Stallman. These individuals are better suited to business than programming I would think. Then again that is what this conference centers around. This compares about the same with the Software Development Best Practices conference where I recognized about 5 names of the presenters.

To tell the truth I would rather go to the Business of Software conference. They came right out with a simple pricing plan. Now I just need to find out how to get my company to pay for the conference. I am already having my company pay for a week of XML training. We shall see how persuasive I can get.

Best Practices Conference

I received a flier in the mail for a software development conference featuring best practices. The conference was being held in Boston this October. It sounded somewhat interesting. The flier advised me to go online to register. There was only one problem. There was no price listed on the flier. Was this thing free? If so, I am booking my flight to Boston right now. Nothing is free in this world though. And I was unhappy that they did not come straight out and announce the price on their marketing material. Don’t piss off your potential customers man.

Well I decided to go online to get more information. I found that the keynote speaker was an engineer with expertise on “robot swarms”. Now I had never heard of this engineer. And it sounded like a strange keynote topic for best practices. I was excited by a two day super sessions on C++ by Bjarne Stroustrup and Herb Sutter. Those are the heavy hitter names that might attract me to a conference. It was still not easy to figure out how much the darn conference cost.

Finally I dug down into the guts of the web site. There was a whole set of prices for the All Access Pass. The prices ranged from $2295 to $2995 based on how early you registered. However there were also other attendance levels:

• VIP Pass (starts at $1895)
  
  
• Conference Pass (starts at $1695)



• Tutorial Pass (starts at $795)



• Expo Pass (starts at free)

No wonder they did not list their prices on the flier. There was not enough room to list the conundrum of pricing structures. Now they got two dings. They appeared to have hid the price. And they also had an overly complicated pricing structure. They do know that three strikes means you are out right?

I scanned the list of speakers presenting at the conference. There were just a few names that I instantly recognized such as Scott Ambler, Larry Constantine, and Dan Saks. The other people must be filler material. Or perhaps I am not up to speed with the latest players in the Best Practices space.

This conference may have been able to attract me if they listed their pricing up front, and the pricing structure was easy to understand. In a future post I shall review the marketing for another conference. Let’s see if that one does any better.

Analysis is King

I read a recent blog post that told a story which seemed to generate a lot of comments. Here is the scenario. Two developers interviewed for a job. Developer #1 went home after the interview. He designed and coded up a solution to a question posed in the interview. He implemented quite a solid solution in 24 hours. Developer #2 stuck around after the interview. He asked a lot of question about how the users would actually interact with the software. He also got more information on the existing processes and software that the users need. Although the guy spent just a half hour extra after then interview, word got out. Developer #2 was offered the job. The next day Developer #1 got a polite rejection call. What do you think about this outcome?

At first I must confess that this was a huge mistake on the hiring firm. All they got was a guy who is probably pretty good at requirements and analysis. The other developer has the skills to design and implement good stuff. There did not seem to be any information that validated that Developer #2 could code his way out of a paper bag. Yeah he has some skills. But these did not seem to be developer skills. And they are certainly not the ability to get things done. That is the quality that Developer #1 possessed and demonstrated.

However upon further thought, I started to think the company made the right choice. This is based on experiences I have observed at work. Yes the ability to knock out a technical solution is a skill. Not everybody has this capability. But there are fewer people who can actually interact act with users (the customer), and figure out exactly what they need. Then there is a lot of time wasted trying to guess what the customer wants. Developers spend a lot of time cooking up great designs and new technology. However that is of little use for actual users. Given a solid individual, I assume they can pick up whatever technical skills are required for a job. But competent requirements analysis is a rare trait indeed.

Maybe I should try to incorporate this scenario the next time we are interviewing to hire some more developers. I may indeed end of recommending my company hires Developer #2.

Google Mondrian

I watched a YouTube video of a Google Tech Talk. The talk was done by Guido van Rossom, who initially started the Mondrian project at Google. Mondrian is a tools which helps developers conduct peer reviews on code. The tool produces contextual code diffs. It negotiates continual reviews and fixes until the reviewer gives approval. The author of the code cannot normally check the source code into the repository until the reviewer agrees.

Internally Google uses Perforce for code configuration management. They do not create branches on a per developer basis normally. Development takes place on the trunk. Branches are only used for release management. All code needs to go through review before being checked in. The previous tool set used for managing reviews was a 3,000 line shell script which wrapped the Perforce commands.

Guido essentially rewrote the old shell scripts in Python. This was his started project at Google. One of the problems with the old system was that parts of it were too slow to use through a VPN from home. Apparently Google employees like to work from home. The new tool was supposed to allow you to click on any line of code and add inline comments. Guido confessed that, with the web based Mondrian, this amounted to a lot of Javascript code.

Having completed the project, Guido reflected on the development of the Mondrian tool. He said that he codes a new web application approximately once every 3 years. He said that he learned Django along the way of implementing Mondrian. He also learned how to make use of the Web Service Gateway Interface (WSGI).

The first version of Mondrian which implemented minimal features was very popular with his beta testers. Guido said the tool did encounter some performance problem with weird scenarios like a change that affected 100,000 files. It also choked on a 20,000 line file that had every single line chanted. The tool has limited configurations which made debugging simpler.

I think my own project could benefit from a peer review tool such as Mondrian. Guido’s presentation made reference to 2 other related tools: (1) Review Board and (2) Rietveld. It is time to do some more research and get my project out of the peer review stone age.

Printing Problems

Sometimes I browse books on Amazon dot com that I want to purchase in the future. Since I am an old school kind of guy, I like to print out the book information so I can look at them on paper later. However I have found that Amazon has problems printing the book detail web pages. I usually get a blank sheet when I try to print an Amazon web page. Sometimes I get a single sheet which is part of an advertisement. I seldom am able to print out the book information. So I resort to copying the web page to a Microsoft Word document. Then I print the Word document to get the information.

Now since Amazon is such a big company, I am sure they know about this problem. Perhaps this is some sort of scheme to force me to buy the book right away. Or maybe they do not want me to print the info and go to a local bricks and mortar book store. Whatever the reason, I suspect this phenomenon does not adversely impact their sales. It might even increase sales. But I just do not like it. I guess I could report the problem to Amazon. But if this printing problem is done by design, I might try to find a way to penalize Amazon. I could take my business elsewhere. Lucky for Amazon, the rest of the service from them is pretty good.

I want to take these bad experiences to heart. The software I work on should never behave like this. If you press print, I would want my software to accurately print what the user sees or expects. Now my project’s software suite is client server. So it may be easier to guarantee success. But that is not to say that we will not migrate to the web sometime in the future. At that point I will want to ensure my users printing experiences are good. It is best to be a customer first so you can feel the pain from problems. That makes you a motivated maintenance developer.

So far I can remember some problems when our customer does printing. Mostly they revolve around performance issues. Sometimes printouts do not show up and take a day to reset before the users can try to print again. I think it may be time to roll up my sleeve and get to work on our own printing problems. I do not want to be the kettle calling Amazon dot com black. To do so would be hypocrisy.