(C) 2009 Hank Wallace
This series of articles concerns embedded systems design and programming, and how to do it with excellence, whether you are new to the discipline or a veteran. This article is about universal concepts that overarch design and programming practices, concepts that will save time and money.
“Money, you say?” Yes, the world, even the world of embedded design is governed by economics. Fortunately, we don’t have economists writing code, but the discipline of economics is crucial to the success of any design. If you can’t design and build it in a certain time frame for a certain cost (or less), you don’t eat. It is amazing the number of engineers who understand the laws of conservation of energy and momentum, but ignore economics.
We have all worked on projects from hell that seem to never end. The reasons for this are manifold, but there always is someone or a group of someones who don’t understand that the programming and design tasks must be accomplished in finite time in order to ship a product and produce a profit. Product design is about accomplishing a result, not doing research (a concept that new-hire students struggle with). Sometimes the engineering managers or designers themselves don’t comprehend the problem, or the MBAs in marketing keep moving the target. Whatever the cause, “money don’t lose at the money game,” 1 but you might get the ax in the process. Even if you get it right and beat the schedule and all the management charts, the sales team might blow it and you’ll both be standing in line for free soup. Economics is not a respecter of men.
It is so rare to meet an engineer with an understanding and care for economics that you can distinguish yourself quickly with simple comments like, “What is the cost target?” Knowing the cost target, you can work backwards to determine exactly how little budget you have for hardware and programming labor. This, however, is the right question to ask, but so many engineers ignore it altogether.
Here is a list of economics-related questions to ask at the next kickoff meeting:
- What is the projected sale price?
- What is the required cost to produce the product?
- What is the design schedule in calendar months?
- What is the design budget in man-hours or dollars?
- What is our department’s historical labor rate?
- What is the support budget for the product after deployment?
- What is the installation budget?
- What is the intended monthly product volume?
- Who is the user?
- What is the user’s educational level?
- What is the user’s motivation to learn how to operate the product?
All these things, and many more, bear on the economic aspects of your next project. Please understand that most of the answers are synthesized on the spot by the marketing staff. On a conference call, you’ll frequently hear someone muffle the phone in their palm and shout one of these questions into the next cubicle. Not all of the numbers are concrete, but at least you’ll have a starting point, and you will be prompting the marketing and sales people to do their due diligence before the next long winded conference call.
Understanding the Problem
Once you have the economic details on paper, you are ready to bang out some code, right? WRONG! The next step is creation of a specification that contains as much of the product’s functional detail as possible. I covered this in depth in The Unspoken Secrets of Software Development. The point I want to convey here is that your team must create and comprehend the functions of the product with their own brains first, capturing the details on paper in the process. All the powers of abstract reasoning must be used before the first line of code is written and the first two components are connected on a schematic. Get this:
- The most powerful emulator in the universe is the HUMAN BRAIN!
I was working in a testing facility years ago when an engineer approached me asking how I would test a certain part, as she was having difficulty getting started writing the test program. I had never seen the part before, so I asked her how it worked. “I don’t know,” she responded. I told her that’s how you get started, by having an abstract understanding of what you have to accomplish and the tools at your disposal. She did that and succeeded.
Many times I’ve come upon a perplexed programmer staring at a screen full of emulator windows, with watch panes, call stacks, source and assembly dumps, etc. After about 30 seconds of inquiry and code inspection, I find that the programmer does not have a grasp of what he’s coding for. But that does not stop him from coding! “Man, I have to get this working. I’ll just throw something together and let the emulator work it out.” That creates a Microsoft type situation where the code is obviously not designed correctly, and is band-aided using a debugger and little abstract reasoning ability.
Part of your specification process should involve interested parties batting around what it is and how it works, from high level details to the nitty gritty. This does not always happen, causing misunderstandings late in the project when engineers are heard to say things like, “you never told me you needed megabytes of memory.” All your powers of abstraction should be brought to bear to understand the problem and its solution. If the marketing guys start losing consciousness, just let them sleep it off in the corner while you engineers do the dirty work.
While you are engaging in abstract thinking, consider other designs you have completed and how those sections of work may be reused. There is of course no seamless and cost-free reuse of hardware or software (I’ll address those myths in a later installment), but copying a section of a schematic or program beats designing from scratch when something close to what you need already exists.
Testing your Ideas
Sometimes I’m called in to a project late, to fix some problem that is stalling production or causing field issues. This relates to everything from software bugs to circuit boards with no bypass capacitors, to poorly designed communications protocols. All these issues could have been solved by testing the product before release to production.
When you are sitting in those interminable product feature meetings, it is important to have everyone in the room walk through the function of the product in his or her mind. I know that this is an abstract process, and some people don’t like to or cannot think abstractly, but it must be done. Writing out the operational steps a user would take helps.
Often this activity happens, but only to a limited extent. “Well, it appears we are on the right track, Bob. Go build me ten and let’s see what happens.” Have you heard this? It’s a recipe for budget and timeline disaster. Bob’s going to make assumptions that the rest of the crew disagrees with, resulting in the waste of many hours of design and build time. This could all be avoided by walking through the details of product operation on paper first.
Anchoring to Reality
There is a danger in all this abstract thinking, that being cost overruns for implementing wild ideas. “Man, if we only had a small nuclear fusion cell to power this product, the thing would run forever! Get it done, Janice.” That’s going to be a budget killer, no matter what the product is.
As an engineer who has completed many designs, you have a feel for what is workable and what’s not. Don’t be afraid to pipe up and tell the MBAs that the features they are requesting will take months and much money to realize. You have to be able to put numbers to your assertions because the MBAs are used to ‘whining engineer naysayers’ shooting down their dreams. Give them the numbers.
There are also practical details that the marketing people are not expected to know. For example, I worked on one product some years back that accomplished its connectivity through AT modems. All the marketing guys thought that the modems were universal and interoperable. They selected a US Robotics modem to ship in the box with our industrial product, thinking that USR had the greatest market share and would likely work the best.
It turned out that the USR modems worked horribly, requiring hours of diddling with idiotic AT modem strings to make even a slow speed link. I did considerable investigation and found out that 70% of the modem chipsets sold were made by Rockwell, while the majority of the packaged modems sold were made by USR, containing a TI DSP implementation. But the USR modems accounted for only 30% of the market, so it was highly likely that the USR modem would be talking to a Rockwell chipset on the other end. The two beasts did not play well together.
We switched to a Rockwell based modem and all the problems went away.
That was a bit of reality that the marketing guys had difficulty understanding, especially sitting on a crate of USR modems that had to be returned or scrapped.
You must be an advocate of the practical, else you die.
Need other examples? How about the use of any new device or tool? Remember the Microchip DSPIC? I received marketing literature from them for years announcing the chip that was going to overtake all motor control and low end DSP apps. YEARS. But could I purchase the parts? NO! I got so sick of waiting that I wrote off the DSPIC and I have not used one since, nor will I. You just know that there were a hundred PCBs designed, fabbed and stuffed with all the parts except for the micro, just waiting for the first samples to be released ‘real soon now’.
If it’s new, I ignore it as a practical matter. I had another customer who was using an 8051 derivative in a product, and we had outgrown its capabilities. I suggested moving to another common processor with expandable memory. Shortly thereafter, a Hitachi rep visited the customer and convinced him to use an H8 variant, throwing in a free C compiler. So we used the H8, and lo and behold about two years later the part was discontinued. It worked great, but only as long as we could buy them.
You must be an advocate of the practical, else you die.
Another example. There was the customer who designed in a new Maxim supervisor IC. The samples worked great, but the Maxim wafer run apparently crashed, leaving my customer with thousands of PCBs sitting in trays waiting for one last part. Maxim eventually bonded out some old style die in the new pinout for them, but only after the schedule was wasted.
You must be an advocate of the practical, else you die.
Are you seeing a pattern here? The best embedded designer is not the person who knows the most electronics or software theory. The best designer is the person who understands how the world works, and all the theoretical stuff. That’s why we don’t see many companies run by PhD’s as they concentrate on one narrow slice of the universe and ignore economics and reality. Most products are designed by ordinary working slobs with BS degrees, but slobs who have been tempered by years of experience and wariness of hype. Unfortunately, their capes are virtual.
Can you be one of them?
There is much about the mindset of engineering in the article From Technician to Engineer. If you think your only job is to bang out working code, take a read, even if you already have the title ‘engineer’.
1The Alan Parsons Project. Gaudi. Money Talks. Arista, 1987.
Hank Wallace is the owner of Atlantic Quality Design, Inc., a consulting firm located in Fincastle, Virginia. He has experience in many areas of embedded software and hardware development, and system design. See www.aqdi.com for more information.