(C) 2009 Hank Wallace

PREVIOUS – Embedded System Design Library: Complexity
NEXT – Embedded System Design Library: Language Features

This series of articles concerns embedded systems design and programming, and how to do it with excellence, whether you are new to the discipline or a veteran. This article is about using open source code in your projects.

“A chain is only as strong as its weakest link.” That common maxim describes neatly the main issue with open source software. Don’t you know that most open source software is written by 20-something guys in their boxers in the wee hours of the morning (bunny slippers optional), while they snack on hot pockets and microwave popcorn? I’m sure there are government stats about that, if you’d like to look them up.

Unfortunately, sometimes you have to hold your nose and use open source code. Perhaps a prior programmer used such code, or your customer quoted a job thinking that 50% of the code could “just be downloaded off the net.” Or maybe you have taken the comfortable route to project completion, trolling newsgroups for links to open source blankies.

Whatever the reason, we’ll all be faced with skanky open source code at one time or another. What’s a programmer to do? Here are the first three rules of using someone else’s open source code:

1. Test.
2. Test.
3. Test.

There are higher order rules, but they follow this general pattern.

Many open source packages come with test routines written by the original boxer-clad coders. These are a good starting point, especially after a port to a new CPU is complete. Once you have the original programmer’s tests working, write your own. Not being close to the code and having the original programmer’s assumptions, it’s likely that you can write a simple test to break it.

Regarding portability, I have noticed that many open source packages are either 1) written to run on an obscure 1925 cellphone platform that does not exist today, or 2) written to run on every processor ever made or ever to be made, with ten times more lines of preprocessor code than actual source code. In the former case, you will have to update the code to run on your platform anyway. In the latter case, it may make sense to remove all the extraneous portability garbage from the program since most of it won’t be needed and you have no way to test it. It’s highly likely that much of the portability preprocessor code is untested or tested by only one user on one machine in a GNU cave in Siberia.

I use a source code formatter on all open source programs I am forced to use. That allows me to easily read the programs. Coding standards vary from anal to stoned, so that helps readability.

Once the code is formatted correctly, I scan the code for sloppy practices. For example, take this expression from a free AES implementation:

if (x > 6 && i % x == 4) {}

What the heck does that do? I’d have to download some ANSI C operator precedence rules to know for sure. It appears to me that the intent is this:

if ((x > 6) && ((i % x) == 4)) {}

but who knows? To determine if this is the case, a test program that encrypts and decrypts test vectors is required. The implementation did not come with test vectors, but I was able to find them as well from a NIST source.

I scan for use of *alloc() calls, and make sure they are matched with free() calls. Same for ‘new’ and ‘delete’ in C++. The Microsoft compilers implement their own variants. Memory leaks are time bombs, but can sometimes be killed by inspection.

I also scan for array usage goofs. The AES program I referred to declared an array of 250 bytes for key expansion. I instrumented the code and determined that the maximum usage of that array is 175 bytes. So there were 75 bytes of wasted RAM.

Much open source code contains no parameter checking in function calls. Pointers are used willy nilly, whether NULL or not.

Open source communications code is notorious for poor error checking. Message lengths and content are rarely verified, especially by implementers of UDP and TCP links. “Doesn’t the IP layer check all that stuff?” they ask. Yes, but the programmers who wrote that are just as sloppy.

Use of a LINT utility helps find nagging issues in open source code. Sometimes the warnings from LINT will prompt investigations that reveal gaping holes in open source designs, such as nonexistent thread safety coding.

It’s always helpful to scan the lean comments in open source programs for “this is unimplemented” holes. I was urged by a customer once to use a GNU ARM compiler for a particular micro. I was trying to debug some interrupt routines which appeared not to be running. After considerable investigation, I determined that there was no interrupt code being generated at all! I didn’t look, but I would expect the source for the GCC compiler contained somewhere the comment that “interrupts for X processor are unimplemented.” But that was not in the release notes.

Once you put that open source code in your product, it’s YOURS forever, so you will have to know how it works. Just start at the beginning of the project and learn how it works. But you ask, “If I have to do that, I could just as well write it myself.” Yep.

However, you WILL have to learn how that code works, whether now, before your product ships, or later, after the customer has stubbed several toes on your nonfunctional product. If you don’t gnow how it works, then it does gnot work. If it’s less than a few thousand lines of code, you might be better off writing it yourself, especially if you have to modify it to make it useful.

Another GNU issue I run into frequently is that open source programmers equate a proliferation of source files with code quality and readability. Download just about any program from Sourceforge and you will get dozens of source files, many of them only 50 lines long. I adapted an open source Modbus implementation but first had to condense the huge array of source files to one, which was then manageable.

Then there are the GNU debugging tools such as Eclipse. If Microsoft tools have the feel of having been written under the influence of pounding alt-rock in a dorm, GNU tools have the feel of having been written under the influence of Mr. Rogers in a day-care. It’s kind of a design-by-committee motif, except the committee never actually meets, and the members of the committee are self-appointed, foisting whatever miscellaneous and under-thought features they want on the user.

I was keeping a long list of Eclipse goofs and bugs, but I quit because I ran out of hard drive space. There comes a point when you have to just quit itemizing the errors and label the whole thing an error, with the expected manifestations. I won’t go through the entire list I compiled, but here are a few choice tidbits:

• Builds are undertaken by the make utility with apparently no regard for the timestamps on the source files. Sometimes it makes just the files that have been changed, and sometimes the entire 300 file project.
• When the program crashes, there is a box displayed with a load of Java detail, but there is no way to copy that data to the clipboard so that it may be easily forwarded to the developers.
• Important commands on menus have no shortcut keys, or the shortcuts do not conform to custom. This smells like a Java-hates-Microsoft culture clash.
• The Help files do not match the dialogs displayed in the user interface.
• The documentation is so poor that it takes hours to find out how to do the simplest things, like add another library to the linker input.
• Some project settings, like linker options, do not work as advertised in the documentation. Random experimentation is required to divine the actual function of such settings.
• Sometimes the Run to Line command works, sometimes not.
• Variables added to the Expression display in the debugger sometimes do not display the value of the variable at all, for no reason.
• Contents of context menus change when activated successively, when nothing about the state of the IDE or project has changed.
• After a successful debug session and closing of Eclipse, the next time Eclipse runs sometimes the project is not found or opened correctly, requiring manual reconfiguring of the entire project.

The common assertion that “thousands of programmers use free Eclipse” may be true, but it really makes me squirm. Poor tools discourage programmers to do thorough testing, especially when half of the schedule has been spent fighting with those tools. Choose your tools based on capability as well as cost. Programmers using Eclipse are paying a heavy price.

Sometimes we have no choice but to use preexisting GNU or public domain code. Just remember the three rules, above, and be as paranoid as possible in a free society.

2015 Update: The Open Source Bubble

There’s another more recent consequence to open source code. Today, most products are equipped with numerous complex communications and storage interfaces. What product is worth its weight in electrons if it does not have at least Ethernet, Bluetooth, USB, WiFi, and broadband interfaces, with a full TCP/IP stack and drivers for each port. Who writes that code? Do you have time to craft an Ethernet driver from scratch? Me neither.

What’s that you say? No budget for purchased drivers? Calm thy fluttering accounting heart! Enter open source code.

Just download all that mess. Download the whole OS. Download the hardware design while you’re at it. Then add your 5,000 lines of code to the 500,000 you just downloaded, and you have a product! The MBA’s will love you.

See the problem? Your product has all those features, including a bunch I did not mention, and you only wrote 5,000 lines of code to get it to market. That sounds like a solution, not a problem. Until your product breaks.

PPP link timed out? TCP links work only in one direction, sometimes? Have to plug and replug USB devices? (These are all problems I’ve had on real products.)

You should have no issue finding those bugs. After all, you have the source code. It’s all there in git. All 500,000 lines. What’s your problem? There is a pile of devices sitting in manufacturing, waiting for your code update. Get to work.

The consequence is that many, many modern devices are based on open source code. Poorly tested, horribly documented, open source code. That includes your company’s devices.

Open source code has fostered the creation of a gigantic, worldwide house of cards of products based on this weak foundation. And your product is on house of cards level 20.

All over the planet, a huge chunk of our new product development is being done with shabby open source code. And we’re worried about financial bubbles? What about the open source bubble?

Author Biography

Hank Wallace is the owner of Atlantic Quality Design, Inc., a consulting firm located in Fincastle, Virginia. He has experience in many areas of embedded software and hardware development, and system design. See www.aqdi.com for more information.