Freezing Requirements

I have been working on designing a solution to the latest batch of requirements to our system. There was some error on the scheduling of this task. Somebody put a placeholder in the schedule, estimating about how long prior jobs like this took. That is good for a placeholder. That will not work for the real schedule, especially when the customer wants a whole lot more than usual this time around.

I brought up the issue a couple times. Finally someone took note and realized the schedule was all screwed up. The managers got together and put two senior developers on the task. The problem is that one of those senior developers works more like a junior development.

The project manager asked me if we could pull this off. I told him yes if there were no other interruptions. Another manager said we needed to fix a few bugs. I said we could fix some bugs, but all that needed to be factored into the scheduel. A final schedule was drawn up. Then we got down to work.

Along comes a requirements analyst. She told me the customer wanted to make a change to the requirements. I Said that was fine and could be captured. But we could not act upon those changes with the current schedule. The analyst told me the customer would not be happy with this. I replied that happy or not, the scheduled could not accomodate changes to requirements. This holds true even if somebody thinks the changes are small.

In the end, I got management backing. The new changes will be addressed in a future delivery. The moral of the story is simple. Figure out all the requirements up front and then freeze them. If you do not do that, there is no way to keep a constrained schedule, especially if it is tight.

PHP Developer

I am taking a PHP development class at college this semester. So I thought I would browse around to see what qualification you need to get a PHP Development job. Of course you need to know the PHP programming language. It appears you also need to know some associated technologies like Drupal.

PHP is the fourth component to the LAMP stack. I guess you better know the other components as well. Seems like there is a whole host of technologies you must learn to be a web developer.

Some of the PHP jobs I found paid a nice wage. Others were weak. I liked that I saw PHP posistions that require things like Python and/or ActionScript. I plan to take classes on those languages soon. Here is the whole list of technologies I found PHP developers should have:

• Drupal
• WordPress
• Zend
• CakePHP
• Joomla
• ColdFusion
• Perl
• Python
• Ruby
• Apache
• Linux
• NoSQL
• Cucumber
• ActionScript
• cross-site scripting

Pay Based on Programming Language

I just read a post by Benjamin Podgursky on the relationship between programming language and income. Very interesting. PHP was near the bottom of the list (least paid). That was disappointing. I am trying to learn PHP this upcoming semester. Then again, the rationale behind the low salaries was that PHP is easy to learn. I will be the judge of that. But I am optomistic.

Near the top of the list was Java. Yeah. I am brushing up on my Java right now. My manager told me I should probably be learning J2EE better in the next year or two. I concur. Something surprising at the top of the list was ActionScript. I thought Flash was on the way out. Perhaps its demise was overstated.

Hard Problem

We had an old trouble ticket that we wanted to resolve in the next maintenance release. One of the developers got assigned to determine the root cause of the problem. Everyone was hoping the analysis would take around a day to complete. After a week of analysis, this developer made no progress. This was going to cause us to slip our release. Not acceptable.

The analysis got reassigned to me. I was told I had a day to complete the analysis. There was already a work around in place. A script ran hourly on the cron that tried to detect and correct bad records. I traced down one of these corrections in detail.

I was frustrated because I could not figure out how the records were wrong. In fact, I kicked it into high gear and proved that the records were not wrong. That’s when I made a breakthrough. The workaround itself was buggy. It was rooted in a timing issue. Not a rookie mistake.

Once I knew that I could not trust the work around, I found a real example of the bug. Then I reconstructed what happened to cause the problem. It was tricky too. There was an optimization in one of the database triggers that tried to skip some unnecessary updates. However one of those updates was masking crucial changes from another update. Doh.

Digging in

One of the mid-level managers on my project wanted me to show a new guy what I was doing on a big bug fix. So every morning I would call him up and check up on him. I would give him tasks to do. Then I would check back on him at the end of the day. My first assignment was for him to replicate the problem I was fixing.

You would think that this was an easy task. I did all the heavy lifting, mining the audits, and discovering the root case of the problem. I came up with an ironclad set of steps to recreate the problem. I turned the new guy loose on thse steps. And then ... he got nowhere.

New guy did not have a database schema. What? How the heck was he doing any work. I told him to call the lead DBA and get them to setup a schema. Next he did not have any database tools. So he got a copy of SQL Developer installed and configured. Then he did not have a working version of our application.

Wow. This is tough work. But it is all good. If it was too easy to step in and get settled with our system, they could hire any lackeys to do the job at a cheap price. Today the developer said the IDE was complaining that a C source file was missing. I had him check whether the Pro*C file was generating the C file. Nope.

There was a mismatch between his Pro*C code calls and the stored procedures in his database. Showed him how to get the source code of the stored procedures. He then compiled in the latest stored procs and still had the errors. Turns out he was using the wrong Clearcase view to obtain the Pro*C code. Killer. This guy is going to need to go a long way just to keep up, let alone add any value.

Using Merge Manager

I had finished a big effort in development. Checked my code into the main branch we were working from. There were two separate branches that were making changes for another version of Windows. Then I found out that I needed to deploy my changes to Windows. You would think I could just move my code changes to the other branch. Not so fast.

In the past, I would manually do the merges. That is just painful, especially when my target is two other branches. This time around I decided to use the help of the Clearcase tool. I think it is called the Merge Manager. You choose your version, click on Merge, then click the target of the merge.

Things worked fine when the merge was simple. Sometimes I could not figure out how to select the correct destination. I just reverted back to manually promotion of the code. The real trouble came when the merge was complex. Me and other people made changes to the same places in the same files. Sometimes I could use Merge Manager to sort though the mess. Other times I had to go back to manual merging.

The bottom line is that for simple cases, Merge Manager helped me merge a lot of files much quicker. Yippee.

Life in the Real World

Just read a blog post from a young developer working on the Microsoft Azure team. He is finding that corporate development is nothing like writing code for his personal products. Good thing he learned this lesson early in his career. Some people spend a lot of time raging against the machine before they figure out how things are done in the corporation.

One realization was that there is no good documentation. Yep. It is hard to write things down in a readable manner. Who has the time for that? You need to ship products. Plus hackers say that the source code is the documentation. Amen to that brother. Another thing this young buck found was that code quality is not job 1. Yep. There are many times when you need to cut corners to get things done. And you never come back and fix things up later. This is probably a big reason why a lot of software sucks.

Full Stack is Where its at

I heard it through the grape vine that FaceBook only hires full stack develpers. Oh snap. Guess any specialists are out of luck if they want to work there. Normally when I hear full stack, I think of front plus back end web development. But I guess it can go further than that. Might also mean you can do database work. Or you can also do system administration stuff. You know what they say. The more you learn, the more you earn. Time to get some back end web development knowledge. PHP anyone?

MSCS Worth Less Now

I majored in Eletrical Engineering when I went to college. Not sure exactly how I picked that major. I think an advisor said it would be optimal. It was around my junior year, when I was taking a course on transistors, that I decided Computer Science was better for me. I signed up for Comp Sci 101. It was boring. I tried to skip the remedial courses and jump right into the good stuff. However the Computer Science Department woud not let me. Therefore I decided to just get my BSEE.

I tried my hand at an engineering job. It sucked. Well it sucked for me at least. Then I searched far and wide for a company to take a gamble on me to be a programmer. I got lucky and was hired as a Junior Programmer Analyst. My company paid for educational expenses. So I decided to go back to school and get a Masters in Computer Science. I figured I could take all those interesting courses I was denied during my undergraduate studies. For the most part, it worked out well. I got into a program that was a non-thesis option. Take ten courses approved by an advisor. Get an MSCS.

Now I am starting to hear that the MSCS degree is worth less and less. In fact having an MSCS without a BCSC might be a liability. This is due to colleges that church out MSCS degrees to make money. Yeah. I might have been in one of those programs way back when. But I did learn a bunch of stuff during graduate school. Just maybe not as much as someone going for a normal research style masters. Oh well. Luckily I have a lot of experience under my belt. By now, my degrees do not matter that much. I just pity the people coming out of college, looking to go into an MSCS program.

Triple Fail

Some memebers of our testing team called me the other day. They wanted help figuring out how to test a fix for one of the customer problems. This was an easy answer. Nobody had fixed the bug they were talking about. So the test team showed me the documentation stating that the bug was fixed.

I dug deeper and determined that it was a documentation error. Our database team had fixed a different trouble ticket that sounded the same. I coached our test team on figuring out how to duplicate the problem for the bug being fixed. We were off to the races.

Later our configurtion management guy produced the final documenttion for our software release. I noticed the wrong trouble ticket was still being referenced. I called our CM guy up because I know him. Told him that he had a problem. He showed me the guidance from the test team. Then he realized what they sent him. I was hoping this woud be the end of this issue.

Finally I caught an email going by to our customer. Again the wrong trouble ticket was being referenced. I alerted our CM guy once again. Then the database guy who wrote the orginal documentation asked me some questions. I set him straight. In enters my boss cold. He had a million questions. It took about 5 phone calls with him to answer all his questions. When things get screwed up, he usually has to ask a lot of questions to be sure.

The thing I hate about all this is that there were many screw ups. At no time was I the source of the screw up. I was joint pointed out that somebody was failing at different points of the flow. In the end, I get all this extra work to clear it all up. I do this to try to ensure the end product to the customer is as good as possible. But this is not my job. That's why we have quality assurance. I guess the bottom line is that someone is not doing their job right.

Windows 7 Pain

Our customer is finally getting around to updating workstations to Windows 7. Part of our team is dedicated to getting our system to work with Windows 7. Guess what? Everything does not work. I could have told you that. Initially I figured we would have problems with registry access, maybe file writing, and some GUI controls. Turns out our biggest pain is the Windows registry.

Turns out we cannot write keys to HKEY_LOCAL_MACHINE. Leave it to Microsoft to break this feature that used to work. There is some new Windows 7 beast called the UAC. If we could turn this feature off, maybe we would be cool. But we do not manage the system settings for our customer.

One of our lead developers put together a hack to use HKEY_CURRENT_USER instead. However that leads to some other headaches. A hack is a hack after all. This might be what we roll out with. There is a short schedule to ship Windows 7 compliant software.

I remember when I first tried to port some of my own tools to Windows 7. Had the same problem. Could not write to the registry. Oh well.

Good Programmer

Last week I read this post about the signs you are a good programmer by C. Lawrence Wenham. Was some food for thought. What I found more interesting were the signs that you are a fantastic programmer. The first sign is that you "can recite from memory all of the includables in the C Standard Library". Oh snap. I should be able to do that. Been working with C for over 15 years. But do you know what? I probably cannot name all of those includes.

Of course I know and . I think there might also be an assert header as well. I also know and . But what are all those other include files? Damn. I looked them up. There are 29 total files. Looking at them, some were relatively familiar. Others I could guess their content. There were one or two that I had never heard of, and could not fathom what they were about.

Need some more practice to graduate into the fantastic programmer position. And that's just the first requirement!

Keeping it on the Down Low

I got tasked with a big project at work. Luckily I had vetted the schedule, and signed up to meet it. Then the boss decided to review my design. He went gang busters with ideas for changes and new ideas. Hey I am down for more work. I just need more time. Nobody wanted to give me more time.

I don't implement extraneous ideas on my own nickel. Somebody needs to give me the budget. I tried to keep my boss in check. But he is the boss. He wanted a lot of extra things for my task. In the end, I drew a line in the sand and costed the pricier extra items. I got the go ahead to extend the schedule. Great. I told you I like more coding when I get the budget.

When I code, I am in the zone. I work to ensure I can deliver something on time. That is my style. Part of the technique includes avoiding distractions from my task. Another is doing tasks in the optimal order. For once it looked like I would actually beat the estimate. I told my team lead. But I warned him to keep the information private.

As soon as a manager hears you are ahead of schedule, they revise the schedule, and then you might be late even though you are ahead. Then I made a mistake. A mid level manager asked how I was doing. I told him great. I confessed that I was projecting an early delivery. Then came the fail. The mid level guy said that the new things my boss requested put the whole big schedule at risk. They need the delivery earlier.

Hey. I was doing my part. I was going to complete earlier. Then the mid level manager told me I should work extra hours to deliver even earlier. Epic fail. Don't beat up the guys that are delivering early. I learned an important lesson. Give one of these guys a positive inch, they try to take a foot. I have ways of dealing with this nonsense though.

Keyboard Accelerator Fail

I added some new buttons to various screen for a recent change. First steps was to produce a design. I always put screen shots in there so users know what to expect. My boss took a look at my draft screen shots and declared that I needed keyboard accelerators. No problem. Just stick an ampersand in front of the name in the resource editor.

Problem was that the accelerators were just not showing. That's okay. I hacked up a screen shot with them using Microsoft Paint. Made a note to get this fixed during the coding phase. I did notice that when you pressed a key, the accelerators seemed to show up. Weird.

Finally I got down to business. Tried to figure out what was going wrong. OnInitDialog() looked okay. Resources looked okay. WTF is going on? I turned to Google. That got me to Raymond Chen's Old New Thing blog. Sure enough. He described why I could not see the accelerators. You don't get the visual cues unless you press the Alt key. This has been so since Windows 2000. This is a "feature" from Microsoft. Great.

Brogrammer Test

I went on the web to google "brogrammer". One of the top links was a test to determine if you are a brogrammer. Here is a spoiler. I got a Broficiency Quotient of -90, making me a Standard Nerd. I must say that some of the banter based on my answers was hilarious.

For example, I said that soda is my coding fuel. The web site responded that this was weak. LOL. Then I chose RPGs as the most fun pastime. The response was for me to GTFO. LMAO. Who made up this test? Answer: John Polacek.

Fork, Dongle, Fired x 2

Check out this story of how an indicent at the PyCon conference blew up and got two people fired. Adria Richards attended PyCon as a developer evangelist for Sendgrid. She heard two guys behind her making inappropriate jokes about "dongles" and "forking". She took a picture of them, and tweeted about the behavior.

Pycon staff came and questioned her. Then they took the two guys aside. The one who made the joke apologized. You would think that would be the end of the incident. Nope. Turns out these guys worked for Play Haven. The guy who made the joke got fired.

Truly unfortunate. But wait. There's more. There has been a large Internet backlash against Adria Richards. She has received many threats. Her blog hs been the target of a DDos. And her employer, Sendgrid, has fired her.

People are referring to the incident as DongleGate. The PyCon code of conduct already states that offensive jokes are inappropriate. Seems the code of conduct has been ammended to prevent future nonsense. The sad outcome is that two people have been fired from their jobs. And that's a huge fail.

Critical Path Analysis

I had to estimate how long it would take to get our product ready for Windows 7. This was pretty easy since we had our test team smoke test our applications. They found some minor GUI alignment issues. Then there were some interproces communication troubles using FindWindow. All in all, I determined it would take one person about 300 hours to take care of everything.

Then I got the call from my team lead. We needed to come up with a new estimate. Why? Well he said some managers will see the 300 hour number, put 3 developers on the project, and schedule it to complete in 300 calendar hours. Oh no. That will not work.

You see, all the work cannot be done in parallel. Somebody needs to figure out the FindWindow replacement for Windows 7. Then that technology will be applied to the places in the applications that use FindWindow. There is a critical path which must be worked serially. Assuming everything can be done in parallel will result in a nasty schedule surprise.

My team lead told me to determine the criical path, then multiply those hours by the number of developers we think will be assigned. That will save us from anybody thinking they can divide the total hours by the number of developers to figure out how long the effort will take.

This feels wrong. Shouldn't we teach critical path analysis to whoever is doing the schedule? That way we don't have to do these estimate hacks? Apparently that task is harder than it seems. We got some managers who are looking to slash the schedule. Their angle is to slice and dice the work up amongst multiple developers to decrease time to delivery. They apparently do not want to know about the critical path. Go figure.

Memory Trouble

Our flagship application has the ability to query many records in the database. We limit the query results to 10k. However the user can decide to get unlimited extra batches of 10k records. When the user wants to view the details of one of those records, we load in a little information for all the records.

Part of the record is a set of subrecords. The way the memory layout was done was to include a fixed size array within the main record. Initially this worked because the maximum number of subrecords were capped. However that maximum was recently opened up. The result was that some queries resulted in an out of memory error. Ouch.

A lot of people pitched ideas on how to solve this problem. However those people were not famliar with the code. We have a lot of C-style code that accesses the array of records, as well as the subarrays of subrecords. Plus the database code is Pro*C, which is in effect the C programming language. You can't do things such as just stick an STL vector in there.

I gave this problem some thought. After I was assigned the task of solving the problem, I figured the simplest solution was best. Change the record structure to have a variable list of subrecords, whose size is dictated by the actual amount of subrecords. This is effective use of the memory we allocate. The implementation of this was a little tricky.

The app is a mix of C and C++ code. Some of the memory is allocated with a call to the C malloc function, while other memory uses the C++ operator new. These techniques are similar, but you should deallocate with the corresponding technology such as a free or delete. In the end, the only thing that caught me off guard was the application's subtle dependency on there being an empty unused subrecord memory be preallocated. Luckily a sharp coder can handle all such nuances.

Requirements Analysis

I got a call from a member of our requirements gathering team. She asked if the requirements for the latest change were good. I scanned them and decided not. Then I asked her what one of the main words in the requirements meant. She said she did not know, and was hoping I could figure that out when I designed the solution.

Now I can design all kind of goodness. I can create slick user interfaces. I can do database magic. And I know how to put together algorithms that work. But if I don't know what the customer wants, then I can't do much. I told the requirements analyst that she needed to figure out and understand what the customer was asking for. Only then could we move forward.

The next morning I get a message that I must attend a meeting with the customer. They were going to discuss the latest change. So the requirements analyst started asking the customer about some ideas she had. She imagined the customer wanted some sort of complex system based on the feedback they had already provided. This meeting was going nowhere. Then I was asked if the discussion answered my concerns. LOL!

I started at the beginning. First I went over how the current system worked. Then I started defining some of the new words the customer was using. Turns out they were just synonyms for other well known understood parts of the system. Then I took a concrete example, and walked the user through how they thought the system would need to work. Then I went through the main scenario a few more times, picking up more details, and getting confirmation and consensus from all the customers at the meeting.

Requirements analysis is hard. You cannot just act as a clerk and write down some ideas, hoping you have captured any requirements. You need to dig in. You need to get the customer themselves to think about what it is they want. Then you should walk through some concrete scenarios. Only then can you truly understand what it is you are talking about. I love designing and inmplementing software. But I definitely don't want to spend my life working on features that the customer does not want or even need.

Logging for C++

Our main apps write a log file to the local disk. This comes in handy when diagnosing bugs in the code. Some time this year, we had a bug. A manager thought we could look at the log to figure out the source. Well the log is not comprehensive. We just log things we think we might need. Now we are on the hook to add more logging!

A couple other manager types got involved. Their answer? Log4j! Well out code is written in C++. The managers said we should just implement the Log4j equivalent for C++. Hold on there partner. If we are going to add in a logging framework, shouldn't we check what is out there first? Yes indeed. I decided to look into the matter one afternoon. Here is what I found.

While there were many options to choose from, I found the most information on log4cxx, log4cplus, log4cpp, glog, and Pantheios. I paid attention to the license type each framework is released under. Also was interested in whether the framework is still being maintained.

Log4cxx is an Apache project. The config files match log4j. The project is being actively maintained. Log4cplus is a port of log4j. There is no a lot of documentation available for this project. Log4cpp is released under the LGPL. It is not an actively maintained project. It also goes by the name "Log for C++". Glog stands for Google Log. It is new, and is based on streams plus macros. It is released under a BSD license. It is thread safe. Pantheios is open source and type safe. It is recommended for large apps. It is simple to use. It is released under a BSD license.

Schedule Fail

I had a big problem to solve at work. That's okay. Solving problems is my job. I cranked out the old estimation spreadsheet. It would take a normal developer four days to complte the task. I said it would take me four days. Now I could do it faster. But I know my work. People are always taking up my time, and taking me off the main task at hand. Sure enough, I barely finished up the task in four days.

The next big task had a short turn around. I was working hard to meet a deadline. I told my team lead that I could not take on any other work. Then I got the call back. The managers said the extra work was the priority. I shifted gears, generated another estimate, and said how long it would take. Then I got down to business.

As I am scrambling to stay focused and make up time to beat my estimate, I get an email from a manager berating me. He said we were going to miss a deadline promised to a customer because of my negligence. WTF? I fired back an email stating that we needed a meeting pronto. Somebody had their facts mixed up. I have become really good at estimation. I never miss my estimates, even when everything goes wrong.

I get a response immediately. No need for a meeting. The manager backed off. I responded, carbon copying everyone who was in the first email. If anyone even remotely thought I was the cause of a delay, we needed to meet. I never heard back from anyone.

I can guess what the situation was. Somebody made some promises without knowing the true estimate of the work. Either that, or they did not take into account when the task started. All standard fare. Then they made a mistake. They tried to blame me. Nice try guy.

Now I don't care if someone tries to badmouth me to the customer. And I don't care if you lie to yourself, or to someone else. Just don't expect to send out an accusation to me and not be called out for your BS. I just won't stand for it. Anyone who is competant should not stand for it either.

When in Doubt, Reboot

I had some crucial data correction scripts to write. The deadline was tight. People kept trying to get me off task. But I completed coding in time. I went to check my code into source control, and the tool froze. I tried again. No luck. After a few hours of messing with it, I gave up.

Other people on the team had the same problem. That was no good. It was a global problem. Some other people tried late through the night, and even the next day. They were unsuccessful. I got a message from the team responsible for the source code control. They recommend we clear up space on our C: drive and reboot out machines.

Now I can understand that a reboot might correct a single user problem. But this was a global problem. I did reboot my machine for grins. Of course that did not solve the root cause. Now they are rebooting servers to hope it fixes the problems. I suspect we are going to be dealing with this problem for the long haul.

All in the Positioning

The customer detected some unusual data conditions while testing out some new functionality in our system. Our test team was tasked with trying to replicate this behavior. Turns out they glossed over testing this function. Might have been due to a lack of time or insight. They were a bit green.

The testers could not replicate the exact behavior from the customer. They could also not make the system as they expected. They asked me for some help. I got similar results as them. We were at a loss. Then I figured we could stop trying to replicate the issue from the customer. Instead we could try to figure out what was going wrong.

We found some data. But it seemed to be chopped off at the end. Some other data was just all garbled up beyond recognition. The data getting truncated came from a file that was SQL*Loaded. I figured a truncation might be due to wrong positioning in the input file.

I researched the record layout documented in the design. Sure enough, the testers were putting a response in the wrong position in the file. You would think they would have double checked that. I guess everyone is busy. Let's hope this gets us past this problem.