Visibility is all

For debugging, visibility is what matters most. For exampke, it is very hard to debug if you cannot use printf for timing reason or such.

When we cannot really see, we tend to

– Stop thinking, stop making forward progress.
– Mystify the symptom with the useless word “weird”.
– Think something else is wrong; such as, hardware or firmware.

In my experience of embedded system debugging, the chance of hardware failure is higher than PC programming; maybe about 50-50. But still, pinpointing where hardware is doing wrong is software engineer’s job. You cannot just say “this hardware is weird” and return it.

So preparing for visibility is a part of your job. There are many reasons why you cannot get visibility. I will talk about some reasons below.

1. You often cannot use debugger for timing reasons. Impossible to stop your program in the middle. This is getting more and more true because most of our application is becoming event-driven. I recommend to forget about debuggers altogether. Data logging and printf are much more useful because

– they can be used most of the time,
– they don’t harm timing of the application,
– they can show more information than debugger,
– there is no need to remember complex command,
– and you make less mistake.

I have been living without debuggers for more than 10 years.

2.  You don’t have a tool to see signals. As a software engineer, you tend to rely on register level information when you debug device drivers. But talk about the power of looking at the logic signals. You don’t have to fully interpret the meaning of the signals. Those signals often
provide a crucial information to split the OK-NG chart to make the binary search forward. As a software engineer, you don’t have to have an gorgeous HP scope. I have a very cheap LogicPort logic analyzer (

3. You don’t know important piece of information. For example, some automatic action happens associated with an exception, and you don’t know that yet.

I take it positive. I cannot read boring spec from cover to cover. I can read seriously only when it is really necessary. So when I sense I don’t know some important piece of information, I switch to study mode.

4. Information is hidden by moron, for example, who do not disclose source code or detailed spec of a device. Reverse engineering and cracking is an effective way to go around the gating moron. This sounds like absurd, and you might think you are wasting your time for the moron. But I take it positive. Cracking and reverse engineering is part of our job. After all, it is better than begging to the moron and receiving wrong info again and again.

For cracking, access dump is one effective way. For example, calling history with parameters or system call log. For device access, I sometimes make a dummy model of the device, and run the program on the simulator, and let the dummy device dump the access log.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s