RTEMS Shell as a Debug Aid

Until the 4.9 release series, the RTEMS Shell was very primitive and only a few commands existed. Virtually no one used it. But with 4.9, the RTEMS Shell took a major leap forward with Chris Johns and I doing a lot of work on it. We added commands, command line history, command line editing, and simple scripting. You can use the shell from a serial port or via telnet. There are now approximately 100 standard commands with some such as ls, mv, cp, ln, and dd ported over from NetBSD. In addition, there are RTEMS specific commands to look at CPU usage per thread, stack usage, and rate monotonic period statistics. There are commands to examine the state of most RTEMS OS objects.

Chris Johns used the standard file related commands to great benefit when developing and debugging the RTEMS File System (RFS). He would mount remote NFS volumes and copy great quantities of data to an IDE hard disk. This allowed him to place stress on his new file system and even turned up a bug in the NFS client code.

But the most useful capability for developing and debugging user applications is probably the capability to include custom commands. These allow you to write commands which are specific to your hardware configuration or application. I have used this to capability to write a set of commands for the Winsystems PCM-MIO-G multi-function I/O PC-104 module. (Kudos to Winsystems for relicensing their GNU/Linux driver to be compatible with RTEMS licensing requirements.) This board has the following features:

• Two 8-channel, 16-bit Analog-to-Digital (A/D)
• Two, 4-channel, 12-bit Digital-to-Analog (D/A)
• 48 Bidirectional I/O lines with interrupt support

For testing the driver for this board, I decided to write a series of custom RTEMS Shell commands to configure the analog inputs and perform various IO operations. These commands allowed direct interaction with the hardware and can be used during development as well as hardware checkout and integration. I implemented the following commands as well as providing shorter aliases for each command without the "pcmmio_" prefix:

• pcmmio_din - Read PCMMIO Discrete Inputs
• pcmmio_dout - Write PCMMIO Discrete Outputs
• pcmmio_adc - Read PCMMIO Analog Inputs
• pcmmio_adc_mode - Set PCMMIO Analog Input Modes
• pcmmio_dac - Write PCMMIO Analog Outputs
• pcmmio_irq - Wait for PCMMIO Interrupts
• pcmmio_bench - Benchmark PCMMIO Interrupts

All the test hardware I had available was a multimeter and a push button. Although minimal, this was sufficient for me to test nearly all of capabilities on this board. For example, I could use the pcmmio_din command to poll for the discrete inputs. When I pressed the button, the command reported the state changed.

[/] # pcmmio_din -i 10
Polling discrete inputs for 10 iterations with 1000 msec period
665:159912852 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
669:238111788 0000 0000 1000 0000 0000 0000 0000 0000 0000 0000 0000 0000
671:250111878 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000

The board cam be configured such that when a discrete input changes an interrupt is generated. The device driver's interrupt handler determines the pin which changed, its current value and timestamps it using the Time Stamp Counter (TSC) register. This information is placed in a message buffer and send via an RTEMS Classic API Message Queue to an application task which is blocked waiting. This allows the application to know as precisely as possible when an input has changed and process that change at the task level. An example output of the pcmmio_irq command when using the push button (which bounces) is below:

[/] # pcmmio_irq -d -i 5
Polling for DIN IRQ for 5 iterations with 1000 msec period
1000 DIN irq pin 8 @ b9eba88c0e (0 usecs since last)
2000 DIN irq pin 8 @ b9eba932b2 (42 usecs since last)
3000 DIN irq pin 8 @ b9eba9c52e (37 usecs since last)
4000 DIN irq pin 8 @ b9ebadec4e (272 usecs since last)
4 total interrupts from DIN in 5000 milliseconds

In the above example, the command looked for interrupts for 5 iterations of a loop with a delay of 1000 milliseconds between iterations. But the interrupts from pushing the button and it bouncing occurred over a 272 microsecond period. In real application code, you would not put a long delay in between each check but block forever or with a reasonable timeout.

I could test analog output (DAC) by simply entering a command to write a value to a particular DAC channel using the pcmmio_dac command and verifying that the proper voltage was written using my multimeter.

[/] # pcmmio_dac 0 5
Write 5.0000 to to dac 0

The pcmmio_dac command has an interesting feature where you can http://pc104.winsystems.comwrite a "step" pattern. This steps from a low voltage to a high voltage using the specified step voltage and time between steps. When it reaches the high voltage, the command begins to step down. The following example illustrates using the pcmmio_dac command to write a step pattern to DAC 0. The pattern ranges from -2.5V to 2.5V with a .5V change every 250 milliseconds for a total of 10,000 milliseconds. When the voltage reaches 2.5V, the step will change to -.5V.

[/] # pcmmio_dac 0 -2.5 2.5 .5 250 10000
Write -2.5000-2.5000 step=0.5000 stepTime=250 msecs dac=0 max=10000 msecs


When testing analog input (ADC), I attached one DAC output to one ADC input. Then I used the command pcmmio_dac to write a voltage and pcmmio_adc to read a voltage. Just as pcmmio_din can monitor the discrete inputs for changes, the pcmmio_adc command can monitor the ADCs for changes in input. The following commands illustrate using this command to read all ADCs or just a single ADC a single time.

[/] # pcmmio_adc
1117:232053 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
[/] # pcmmio_adc 0
1120:495519 0.0000

Note that in the above, the voltage read isn't the voltage that was written. There are a few potential reasons for this. First, I could have hooked things up wrong (but I checked and I didn't mess that up). Second, I could have gotten confused on the DAC and ADC channels I used. Yes, I did that a couple of times. But the final reason was that I forgot to initialize the channels for the input configuration I was using. This lead to the need for a command to configure an ADC channel.

Each ADC channel could be individually programmer for either single-ended or differential input, unipolar or bipolar voltage ranges and for 5V or 10V as the upper voltage in the range. I didn't want to enter sixteen commands by hand, so I added the feature where pcmmio_adc_mode can configure a contiguous range of ADC's to a particular setting. But this still could require multiple commands. With a flash of insight, I remembered that I could write a shell script to do this for me. This led to me writing the the following very simple shell script to configure the ADCs.

#! joel
echo "Setting all ADCs to +/-10V (bipolar) and single ended"
pcmmio_adc_mode 0 15

As mentioned earlier, to test the ADCs I had to write a voltage using the pcmmio_dac command and then read the voltage using the pcmmio_adc command. I wanted to run a series of voltages through the ADC but the step command didn't let me see the input between steps. I could have figured out a way to get access to the lines but I had used a pre-made jumper wire and didn't want to destroy it. So I wrote another simple shell script which repeated pcmmio_dac, sleep, pcmmio_adc commands. This allowed me to verify that a range of voltage could be written and read. The following is one set of the three commands. There were a lot more than this to have a script that ran for fifteen seconds.

pcmmio_dac 4 -2.0
sleep 1
pcmmio_adc 0

You might wonder how I got the shell scripts onto the target. Well I used another interesting feature of RTEMS. RTEMS has long has the In-Memory File System (IMFS) and the capability to load initial contents from a tar file image linked with the application. I simply wrote the scripts on my development machine and included them in the initial file system contents.

In using the RTEMS Shell and its ability to add custom commands, I was able to refactor the original GNU/Linux device driver, adapt it to RTEMS, add capabilities such as timeouts and timestamping input, and debug this device driver with very little difficulty. Plus these commands are now available for hardware integration and testing for this project and any other project that might use this device driver in the future.