Software Quality under the Bus

When you are not doing continuous deployment and deployment windows are limited to say twice a month, shipping valuable software features to customers is like taking the bus where the bus only stops for you twice a month. If you miss the bus, you have to wait two weeks or another month.

Given the dire consequences of missing the bus, you will do everything possible, perhaps even take every possible shortcut to ensure you will not miss the bus the next time it comes around. You would skip breakfast, not take a shower, rush out the door without kissing your wife or hugging your kids, in such a hurry you even forget to brush your teeth – yuck! The only thing you can think about in your panicked state is making that bus.

With a Lot of Busses

On the other hand if buses arrive and pickup continuously, when you needed a few more minutes to get ready, there would be no disincentive to taking the time to get ready before you go. You need a few extra minutes, no problem, you take them! Hell, you might even spend a few minutes more on extras like shining your shoes or double checking that nothing had been missed. Or go wild and add a little something to make yourself stand out.

Uber

But if the city budget is never going to allow for a continuous bus schedule, another option you might want to try is Uber! Instead of getting ready to make a delivery schedule, schedule the delivery when you are ready. Uberize the process!

The idea here is for the deployment countdown to begin after the delivery team has signed off on the feature set for final Qa certification and production deployment. The deployment date would be determined by the date when the software is ready for deployment.

This approach has a few advantages.

Quality. Scheduling delivery when the goods are ready means no more need for shortcuts, assuming quality and completeness would be the first order of business under the right conditions.

Time to market. No time is wasted waiting for a deployment window because the deployment mechanism kicks off immediately once the software is completed.

The Uber model also has a disadvantage when you have many projects in concurrent development, and would be particularly problematic if there is concurrent development of the same components on different components in different environments.

Experiences of a Paperboy

Back in the olden days, about 30 years ago, kids were allowed to have a job as a paperboy. These days, probably for insurance and liability reasons, only adults can have that job. If you grew up in a family on the low income end of the spectrum, a paper route was a coveted source of income for a kid.

I had a couple of “paper routes” over a year or two as an 11-12 year old and even had two concurrently. I delivered newspapers for now extinct Los Angeles area newspapers including the Herald Examiner and the San Gabriel Valley Tribune.

One of the my most difficult tasks as a paperboy was having to wake up and get out of bed at around 4am on Sunday mornings so that my customers could have their Sunday morning comics with breakfast. You had to get up early because you had to fold the papers and put rubber bands around them to mold it into aerodynamic wing which would sail like a Frisbee as you fired them at the front door from your bike. It was really important to crease that paper perfectly with properly centered rubber band placement to produce the optimal aerodynamic qualities.

Throwing the newspaper was the fun part which is why it was so important to make that folded paper a streamlined flying wing. Folding was drudgery and cost me an extra hour of sleep. I often dreamed of a machine I wanted to invent which would do the folding and rubber banding of the newspapers automatically for me. This machine had robotic arms at the sides and a claw in the middle; one arm held the paper down at the point of the fold and the other pulled the paper over to complete the fold. The claw expanded a rubber band and would slide it over the folded paper held in place by the robot arms. The claw would then hand me the streamlined newspaper flying wing for the fun part. This contraption would be mounted on the front handle bars of my “Sting Ray” bicycle with a banana seat which was the latest in bicycle coolness at the time.

With my new invention – the handlebar mounted newspaper folding robot, I would get an extra hour of sleep on Sundays. I would just load the unfolded papers into the hopper, hop on my bike and ride away. All I had to do was the fun parts, ride my bike and pitch sleek flying wing newspapers. Unfortunately this was all just a fantasy and nothing ever came of it. I spent a lot of time reading comic books in those days.

The other part about delivering the Sunday morning paper was that for some reason some people in the neighborhood unleashed their hounds on Saturday nights, and these dogs finally free from their captivity reverted to their pack mentality which is to chase chubby paperboys riding a bicycle slowed down by two bags full of newspapers loaded on the handle bars. It was a terrifying experience. I was cold and alone riding the dark streets dark and vicious packs of carnivores were trying to attack me. Every Sunday morning, there were the same two German Shepherds near the same cul-de-sac trying to get me. I peddled extra fast and built up a healthy speed as I approached the area.

Even when throwing the most aerodynamically sounds flying newspaper wing, at the extreme velocity one attains on a well oiled Sting Ray bicycle, a severe impact on the accuracy of the newspaper pitch is simply unavoidable. But accuracy be damned, and nobody in that area was going to get their Sunday paper “porched”.  I pitched my paper at the target house front doors and my Sting Ray always outran the vicious creatures, except for one time. One of the German Shepherds caught me jumped up and bit me on the hip, drawing blood. Then he ran away. It was a hit and run and I was crying and hurt and more terrified than ever. There was nobody around to help me or witness the attack. My awesome Sting Ray had done all it could to help me escape the beasts, but it was not enough.

During the week when I was “collecting”, which means to go to the door of each customer once a month to collect payment, I told the owners of the German Shepherd who bit me about what had happened. They blew it off and denied it could have been their dog. I was on my own against them the next Sunday. They hardly acknowledged me and my ordeal.

The next Sunday, I changed my paper route so that by the time I got to the dog pack neighborhood, my paper bags would be almost empty and very light so I would be able to achieve even greater speeds. The new route was not the most efficient and meant it would take me longer to get the papers delivered, but it worked. I was whizzing by and the dogs never caught me again, but it was a hassle.

One weekend, I got up late and had to rush my deliveries, so I went back to the old route so I could get the papers out early enough to avoid complaints. I was still half asleep and not in a very good mood because I was not ready to get up and I was rushing. Unfortunately the beasts were ready for me this time and I had a full load of papers to slow me down. When I saw them coming at me I knew I could not escape this time and something just snapped. I knew I was doomed. I decided to stand my ground this time. I threw my bike down in the street, faced the beast and screamed like a crazed killer madmen, or more accurately, like a terrified chubby prepubescent kid with a high pitched voice about to die.

A strange thing happened. The lead beast stopped and looked confused. I was too scared to move but continued to face him. Then I stomped one foot forward and he ran away. Wow! I bluffed him and he bought it! Every Sunday after that instead of speeding by those sorrowful mutts, I slowed down instead and pointed my bike in their direction. When they came from behind, I circled my bike around went towards them, slowly though, very slowly because I didn’t want to actually get too close. I just needed to give the impression I was ready to stand up to them but I did not want to risk getting into biting distance.

I was in sixth grade. Basically I was an elementary school senior. Top of the food chain. Respected and held in awe by 2nd and third graders. This was in the olden days, way before today’s current popular campaigns against bullying and every school had it’s bullies. The school bully at our school was Ernie Gomez.

Ernie had a reputation like bad, bad, Leroy Brown. Everybody knew about Ernie, everybody was submissive when confronted by Ernie and everybody avoided Ernie, except some who wanted to be like Ernie. They were Ernie’s legion of sycophants. They sucked up to Ernie and followed Ernie around and did Ernie favors to have Ernie’s protection.

One day Ernie declared me his next target for a beating after school. I dont remember what I did to make this happen. I was terrified. On the day of his pronouncement after school I was nowhere to be found. I hung around after class. I left the school grounds after everybody was gone.

I was varying my departure route every day from then on in dire fear of running into Ernie after school. I either darted for the gate at top speed as soon as the bell rang to beat Ernie out of the school gate, or I had a lot of questions I had to ask the teacher after class. Ernie lived just one street over from me so almost my entire route home was the same as Ernie’s. I knew this was not going to work forever.

I figured out that if I jumped the playground fence in back of the school and crossed through two backyards, I would arrive in my neighborhood and would never cross Ernie’s path. I tried it out one day and problem solved! The only thing I had to worry about was getting scratched by the sharp top of the school’s chain link fence and the homeowners whose backyards I was running through would come out and yell at me.

This worked great the first day, but everything changed the second day. On the second day, after jumping the first fence, a dog came from out of nowhere and was barking and running at full speed in my direction. God dammit! Where the hell was that dog coming from? I guess he was chasing out chasing chubby paperboys the day before. I was able to get to the next fence and jumped over it before he got me. Crap. Here we go again. Another beast out to get me. At least it was not a German Shepherd. This time it was some kind of poodle mutt, but he did have a mouth full of sharp teeth and I was invading hid territory this time, so he was just doing his job. My plan was to quietly sneak up to the fence, jump over and run as fast as possible to the next fence, I was pretty sure I would always be able to get to the second fence before he caught me.

So this was my life going forward. After school, run to the playground fence, quietly, jump over the fence without getting scratched, get chased by dog, jump over next fence into the next yard, run to the next fence and hope the homeowners would not see me and come after me too. As terrifying as dogs and homeowners and the tops of chain link fences were to me at the time, they were still less terrifying to me than the school bully Ernie’s declared fatwa on me.

Sadly, my new life of relative safety from Ernie was about to end. As sixth graders, elementary school seniors, some of us had certain leadership responsibilities, such as leading small groups of second or third graders to various playground activities during lunch recess. They looked up to us and respected us. As sixth graders we were important and accomplished and had power entrusted to us by the noon aids to shepherd the little ones around.

One day, me and my second and third graders crossed paths with Ernie and his legion of sycophants. We were way out in the furthest regions of the playground, well beyond the sight of any noon aid. I was trapped. I could not run away in front of my second and third grade admirers, but I was also tired jumping of sharp chain link fences, being chased by stupid poodle mutts with teeth and worrying about Clint Eastwood coming out of his house with a shotgun because I was running across his lawn.

I decided it was time to die and I was just going to get it over with. The sycophants were cheering and egging Ernie on to kill me. My second and third graders were scared. I also decided that I was going to get beat up anyway, but I was going to fight back. I just stood there with Ernie taunting me and waited for it to start. Finally Ernie shoved me. I reacted by punching him in the face knowing it didnt make a difference if it made him mad because he was already going to kill me anyway. But something unexpected happened. Ernie recoiled, started crying and he and his legion left the scene. I was just standing there in a state of shock and then the second and third graders were cheering. It was kinda like “Ding Dong, the Witch is Dead!” and I killed the witch. Ernie stopped being a bully after that and little kids felt more safe at school.

It was very strange to me that Ernie went down with a single punch. What I came to realize later is that I am one of five brothers and we had fights everyday. Physical fights, but these fights were not to hurt each other. They looked like fights to grownups, but we held our punches. They were more like a competition. We ambushed each other. We took something from the other to launch into a scuffle to get it back. It was a daily occurrence and I think it was a form of training against people like Ernie that I was simply unaware of.

The takeaway for me from my experiences with vicious beasts and bullies is that you have to stand your ground and you have more power than you think. In the end life is better for everyone.

A Quick Workaround to Get ACA Enrollment Back on Track in One Day

 

The ACA enrollment site is plagued with usability issues due to architectural and implementation constraints, which are aggravated when there is heavy concurrent usage which ultimately prevents users from creating accounts and enrolling in the program.

Since we already know the website cannot handle high concurrent usage, and a proper resolution ranges from a full architectural redesign to rewriting of major components – none of these issues can be addressed quickly. Taking this approach we are doomed to weeks or months of a long and painful process or iterative defect resolution at the expense of the website’s users. Let’s just acknowledge the limitations of the software as it currently is and work around it.

My Proposal in a Nutshell

• Replace the account creation page with a lightweight version.
• Manage the demand of the Research and Enrollment process by implementing an appointment system.
• Give priority access to users with the greatest need.
• Users would provide minimum input with no validation to get started; appointment hours preference and basic need info such as income and family size.

With my plan, utilization of the heaviest components of research and enrollment would be governed such that the site would never receive more load than it’s known capacity. The users with the greatest need would get enrolled first and they would be guaranteed a good experience without load-related issues. Users who would not be able to signup immediately would have the assurance they are in the system and will have a guaranteed time slot to sign up. Appointments can be rescheduled by the consumer; no harm here since the system will control the demand it can handle.

The Enrollment Process

The enrollment process can be simplified as these general steps take by the consumer at the AHA website.

1. Create an account on the site.

2. Calculate the consumer’s need and ability to pay.

3. Research plans available to this consumer based on input from the user.

4. Enroll in a plan.

 

A New Enrollment Process

We know we have very high demand to research and enroll into ACA. There are potentially tens of millions of consumers who want to enroll or research ACA insurance plans.

Since the website cannot handle the demand, we need to manage the demand.

My proposal would lighten and manage the demand of the enrollment process.

1) Create an account on the site. “Strip down the account creation process.”

• Replace the current enrollment page with a new simplified account creation page comprised of two steps, the first to validate the email address, the second to create an account using bare bones input. The first new page takes as input only 1 field: email address. The page explains to the user to confirm their email address via an email, which is being sent. This page saves nothing since the email address has not yet been validated. The email address is checked against the database to determine if an account has already been created. An email is sent to the user with a link to the site to validate the email address. If the email address is already in use, redirect the user to the login page. If we find that the volume of new email addresses to be overwhelming, we can always send these to a queue for a paced send via SMTP, but prefer to avoid complexity in this step to minimize load if we can get away with it.
• When the user is redirected to the site to confirm the email address they are given an account creation page where they will enter: password, select income and family size from a dropdown, select appointment time preferences to complete the registration process. Family size and income selections would be based on existing data for these groups with a goal of managing demand. All of these variables can be changed later and there will be no validation of these things at this time. Some users will game the system by inputting values they think will help them get a priority appointment so a warning should be posted that if anything is incorrect, the appointment may be rescheduled. Some users will get away with false info, but so what. We want everyone signed up and they would not do this if they didnt need the insurance. We are going to control the load so no harm to the user experience can occur from this.

2) Calculate the consumer’s need and ability to pay. “By Appointment.”

• The site already has this in place, not too much needs to be changed here. We are going to work around its limitations with an appointment system, accepting the number of users the system can handle. The only thing we change is a users ability to reach this page based on their appointment.

3) Research plans available to this consumer based on input from the user. “By Appointment.”

• Same as 2. Nothing much to change, just need to implement a restriction to use these pages based on their appointment.

4) Enroll in a plan.  “By Appointment.”

• Again, not much to do here except to implement a restriction to use these pages based on appointment.

Estimates

The appointment system will require four developers:

1) A developer who knows the database schema.

2) A developer who knows the enrollment code.

3) A developer who is a quick hand at creating new pages.

4) A good services developer familiar with the service implementation currently in use.

5) A Tech Lead who coordinates the efforts of the developers and provides solutions to unexpected issues.

For item 1:

SQL Developer to create appointment tables and CRUD queries. – 4 hours.

Services developer to create a service to implement appointment data access. 8 hours

Quick page developer to create the new lightweight email validation and account creation pages: 8 hours

Enrollment developer to disable existing account creation and enrollment pages and redirect to the new: 4 hours

For items 2-4

Enrollment developer and quick page developer to implement appointment check, base page functionality, into the calculate, research, and enrollment pages: 8 hours each.

This plan can be implemented by an existing Tech Lead or by me. If I do it, the cost will be zero to the government because I will use my volunteer PTO time, 16 hours, to do the job.

Gary Smith

Charlotte, North Carolina

Siberian Beauty

I was surfing the web one night last year looking for something interesting and I somehow came upon the art of Solongo Monkhooroi. I was fascinated by this particular painting – The Fishing Hole:

The Fishing Hole

I ordered a print and was very happy with it.

The artist had a few other “Beauty and the Beast” themed paintings. Another one named “White Nights” was of particular interest because it was a Russian theme and I am a fan of Russia and particularly of my Russian wife.

White Night

I was becoming a big fan of Solongo’s work. I asked Solongo if she would paint a Russian Beauty and Beast with a Russian Bear and my wife. She agreed. I provided pictures of Lena for Solongo to get a likeness from.

After a few emails back and forth to work out the details of the piece, Solongo sent me some sketches to choose from:

First Sketches

It was a very tough decision. I wanted some Russian scenery and a Russian Bear but I also really like Solongo’s tiger from The Fishing Hole. So, I asked if Solongo could also add a Siberian Tiger, which made sense because my wife Lena is from Siberia. Here is the sketch she came up with:

Revised-Sketch-2

And the final painting:

Siberian Beauty

Solongo was also nice enough to throw in a copy of White Night.

My First Trip to Siberia

I just got back from vacation in Siberia. Siberia was great. I stayed in ” Tomsk ” (Tomck). The hotel I stayed in was first rate. One of the best hotels I have ever stayed in. It’s a small town, but also a university town, There are several medical and other universities there. The town is about 400 years old and was founded as a fortress to protect Russia ‘s eastern border. 
I was surprised that the people seem to be rather well off, or at least better off than I was expecting. Most people wore nice clothes and it seemed that everyone has a cellphone. Along the main drag, there was at least three cellphone stores per block. 
The day I arrived there was some random religious procession down the street in front of my hotel. Also, Friday was the last day of school for the summer break. The girls were wearing traditional white “buns” in their hair and some garment that looks like a maids uniform. 

Alcohol was everywhere. You can buy beer from little stands along the street and just about everywhere. The coffee shops had a full bar which was liquor for coffee drinks.

One place I went to had a full array of liquor and was manned by two 16 year old girls. While there is some ordinance against drinking in public, it was not enforced and it was common to see people walking around with a bottle. I even saw some kid with a bottle of beer in his hand walk up to a policeman on the street and ask for directions.

 

There was almost no English as Siberia is not a typical tourist destination. I could not have ordered a meal without Elena’s help. Even then, it was a challenge. Elena’s English is basic, so while she could read the menu, she had a hard time translating it to English for me sometimes and had to consult a Russian-English dictionary just to order food.

 

I had a great time with Elena. My favorite part of the trip was the Russian bathhouse. There was one commercialized version in town, and we also went to her father’s “banya” which was located at his Dacha a few miles outside of town. It was the best vacation I’ve ever had. I  am going to bring her over here. I also want to build a dacha and a banya.