Debugging Techniques
To Contents | To previous page | To next page

## Debugging Techniques

eCos applications and components can be debugged in traditional ways, with printing statements and debugger single-stepping, but there are situations in which these techniques cannot be used. One example of this is when a program is getting data at a high rate from a real-time source, and cannot be slowed down or interrupted.

eCos's infrastructure module provides a tracing formalism, allowing the kernel's tracing macros to be configured in many useful ways. eCos's kernel provides instrumentation buffers which also collect specific (configurable) data about the system's history and performance.

### Tracing

To use eCos's tracing facilities you must first configure your system to use tracing. You should enable the Asserts and Tracing component ( CYGPKG_INFRA_DEBUG ) and the Use tracing component within it ( CYGDBG_USE_TRACING ). These options can be enabled with the Configuration Tool or by editing the file BUILD_DIR `/pkgconf/infra.h` manually.

You should then examine all the tracing-related options in the Package: Infrastructure chapter of the eCos Reference Manual. One useful set of configuration options are: CYGDBG_INFRA_DEBUG_FUNCTION_REPORTS and CYGDBG_INFRA_DEBUG_TRACE_MESSAGE, which are both enabled by default when tracing is enabled.

The following "Hello world with tracing" shows the output from running the hello world program (from the programming tutorial in Getting Started with eCos ) that was built with tracing enabled:

Hello world with tracing

    $ mips-tx39-elf-run --board=jmr3904 hello

    Hello, eCos world!
    ASSERT FAIL: <2>cyg_trac.h          [ 623] Cyg_TraceFunction_Report_::set_exitvoid()                                                            exitvoid used in typed function
    TRACE: <1>mlqueue.cxx         [ 395] Cyg_ThreadQueue_Implementation::enqueue()                                                            {{enter
    TRACE: <1>mlqueue.cxx         [ 395] Cyg_ThreadQueue_Implementation::enqueue()                                                            }}RETURNING UNSET!
    TRACE: <1>mlqueue.cxx         [ 126] Cyg_Scheduler_Implementation::add_thread()                                                           }}RETURNING UNSET!
    TRACE: <1>thread.cxx          [ 654] Cyg_Thread::resume()                                                                                 }}return void
    TRACE: <1>cstartup.cxx        [ 160] cyg_iso_c_start()                                                                                    }}return void
    TRACE: <1>startup.cxx         [ 142] cyg_package_start()                                                                                  }}return void
    TRACE: <1>startup.cxx         [ 150] cyg_user_start()                                                                                     {{enter
    TRACE: <1>startup.cxx         [ 150] cyg_user_start()                                                                                     (((void)))
    TRACE: <1>startup.cxx         [ 153] cyg_user_start()                                                                                     'This is the system default cyg_user_start()'
    TRACE: <1>startup.cxx         [ 157] cyg_user_start()                                                                                     }}return void
    TRACE: <1>sched.cxx           [ 212] Cyg_Scheduler::start()                                                                               {{enter
    TRACE: <1>mlqueue.cxx         [ 102] Cyg_Scheduler_Implementation::schedule()                                                             {{enter
    TRACE: <1>mlqueue.cxx         [ 437] Cyg_ThreadQueue_Implementation::highpri()                                                            {{enter
    TRACE: <1>mlqueue.cxx         [ 437] Cyg_ThreadQueue_Implementation::highpri()                                                            }}RETURNING UNSET!
    TRACE: <1>mlqueue.cxx         [ 102] Cyg_Scheduler_Implementation::schedule()                                                             }}RETURNING UNSET!
    TRACE: <2>intr.cxx            [ 450] Cyg_Interrupt::enable_interrupts()                                                                   {{enter
    TRACE: <2>intr.cxx            [ 450] Cyg_Interrupt::enable_interrupts()                                                                   }}RETURNING UNSET!
    TRACE: <2>thread.cxx          [  69] Cyg_HardwareThread::thread_entry()                                                                   {{enter
    TRACE: <2>cstartup.cxx        [ 127] invoke_main()                                                                                        {{enter
    TRACE: <2>cstartup.cxx        [ 127] invoke_main()                                                                                        ((argument is ignored))
    TRACE: <2>dummyxxmain.cxx     [  60] __main()                                                                                             {{enter
    TRACE: <2>dummyxxmain.cxx     [  60] __main()                                                                                             (((void)))
    TRACE: <2>dummyxxmain.cxx     [  63] __main()                                                                                             'This is the system default __main()'
    TRACE: <2>dummyxxmain.cxx     [  67] __main()                                                                                             }}return void
    TRACE: <2>memcpy.c            [ 112] _memcpy()                                                                                            {{enter
    TRACE: <2>memcpy.c            [ 112] _memcpy()                                                                                            ((dst=80002804, src=BFC14E58, n=19))
    TRACE: <2>memcpy.c            [ 164] _memcpy()                                                                                            }}returning 80002804
    TRACE: <2>cstartup.cxx        [ 137] invoke_main()                                                                                        'main() has returned with code 0. Calling exit()'
    TRACE: <2>exit.cxx            [  71] __libc_exit()                                                                                        {{enter
    TRACE: <2>exit.cxx            [  71] __libc_exit()                                                                                        ((status=0 ))
    TRACE: <2>atexit.cxx          [  84] cyg_libc_invoke_atexit_handlers()                                                                    {{enter
    TRACE: <2>atexit.cxx          [  84] cyg_libc_invoke_atexit_handlers()                                                                      (((void)))

    Scheduler:

    Lock:                0
    Current Thread:      <null>

    Threads:

    Idle Thread          pri =  31 state = R      id =   1
                         stack base = 800021F0 ptr = 80002510 size = 00000400
                         sleep reason NONE     wake reason NONE
                         queue = 80000C54      wait info = 00000000

    <null>               pri =   0 state = R      id =   2
                         stack base = 80002A48 ptr = 8000A968 size = 00008000
                         sleep reason NONE     wake reason NONE
                         queue = 80000BD8      wait info = 00000000

### Kernel Instrumentation

Instrument buffers can be used to find out how many events of a given type happened in the kernel during execution of a program.

You can monitor a class of several types of events, or you can just look at individual events.

Examples of events that can be monitored are:

- scheduler events

- thread operations

- interrupts

- mutex operations

- binary semaphore operations

- counting semaphore operations

- clock ticks and interrupts

Examples of fine-grained scheduler event types are:

- scheduler lock

- scheduler unlock

- rescheduling

- time slicing

Information about the events is stored in an event record. The structure that defines this record has type `struct Instrument_Record` :

The list of records is stored in an array called `instrument_buffer` , which you can let the kernel provide or you can provide yourself by setting the configuration option CYGVAR_KERNEL_INSTRUMENT_EXTERNAL_BUFFER.

To write a program that examines the instrumentation buffers:

1. Enable instrumentation buffers in the eCos kernel configuration. The component macro is CYGPKG_KERNEL_INSTRUMENT.

2. To allocate the buffers yourself, enable the configuration option CYGVAR_KERNEL_INSTRUMENT_EXTERNAL_BUFFER.

3. Include the header file `cyg/kernel/instrmnt.h` .
   `#include <cyg/kernel/instrmnt.h>`

4. The Instrumentation_Record structure is not published in the kernel header file. In the future there will be a cleaner mechanism to access it, but for now you should paste into your code in the following lines:
   `struct Instrument_Record
   {
   CYG_WORD16 type; // record type
   CYG_WORD16 thread; // current thread id
   CYG_WORD timestamp; // 32 bit timestamp
   CYG_WORD arg1; // first arg
   CYG_WORD arg2; // second arg
   };`

5. Enable the events you want to record using `cyg_instrument_enable()` , and disable them later. Look at `cyg/kernel/instrmnt.h` and the examples below to see what events can be enabled.

6. Place the code you want to debug between the matching functions `cyg_instrument_enable()` and `cyg_instrument_disable()` .

7. Examine the buffer. For now you need to look at the data in there (the example program below shows how to do that), and future versions of eCos will include a host-side tool to help you understand the data.

Using instrument buffers

This program is also provided in the `examples` directory.

    /* this is a program which uses eCos instrumentation buffers; it needs
     to be linked with a kernel which was compiled with support for
     instrumentation */

    #include <stdio.h>
    #include <pkgconf/kernel.h>
    #include <cyg/kernel/instrmnt.h>
    #include <cyg/kernel/kapi.h>

    #ifndef CYGVAR_KERNEL_INSTRUMENT_EXTERNAL_BUFFER
    # error You must configure eCos with CYGVAR_KERNEL_INSTRUMENT_EXTERNAL_BUFFER
    #endif

    struct Instrument_Record
    {
     CYG_WORD16 type; // record type
     CYG_WORD16 thread; // current thread id
     CYG_WORD timestamp; // 32 bit timestamp
     CYG_WORD arg1; // first arg
     CYG_WORD arg2; // second arg
    };

    struct Instrument_Record instrument_buffer[20];
    cyg_uint32 instrument_buffer_size = 20;

    int main(void)
    {
     int i;

     cyg_instrument_enable(CYG_INSTRUMENT_CLASS_CLOCK, 0);
     cyg_instrument_enable(CYG_INSTRUMENT_CLASS_THREAD, 0);
     cyg_instrument_enable(CYG_INSTRUMENT_CLASS_ALARM, 0);

     printf("Program to play with instrumentation buffer\n");

     cyg_thread_delay(2);

     cyg_instrument_disable(CYG_INSTRUMENT_CLASS_CLOCK, 0);
     cyg_instrument_disable(CYG_INSTRUMENT_CLASS_THREAD, 0);
     cyg_instrument_disable(CYG_INSTRUMENT_CLASS_ALARM, 0);

     for (i = 0; i < instrument_buffer_size; ++i) {
     printf("Record %02d: type 0x%04x, thread %d, ",
    	i, instrument_buffer[i].type, instrument_buffer[i].thread);
     printf("time %5d, arg1 0x%08x, arg2 0x%08x\n",
    	instrument_buffer[i].timestamp, instrument_buffer[i].arg1,
    	instrument_buffer[i].arg2);
     }
     return 0;
    }

Here is how you could compile and run this program in the `examples` directory, using (for example) the MN10300 compiler:

    $ make XCC=mn10300-elf-gcc  PKG_INSTALL_DIR
    =/tmp/ecos-work-mn10300/install instrument-test
    mn10300-elf-gcc -c -o instrument-test.o -g -Wall -I/tmp/ecos-work-mn10300/install/include -ffunction-sections -fdata-sections instrument-test.c
    mn10300-elf-gcc -nostartfiles -L/tmp/ecos-work-mn10300/install/lib -W1,--gc-sections -o instrument-test instrument-test.o -Ttarget.ld -nostdlib
    $ mn10300-elf-run --board=stdeval1 instrument-test

Instrument buffer output

Here is the output of the `instrument-test` program. Notice that in little over 2 seconds, and with very little activity, and with few event types enabled, it gathered 17 records. In larger programs it will be necessary to select very few event types for debugging.

    Program to play with instrumentation buffer
    Record 00: type 0x0207, thread 2, time  6057, arg1 0x48001cd8, arg2 0x00000002
    Record 01: type 0x0202, thread 2, time  6153, arg1 0x48001cd8, arg2 0x00000000
    Record 02: type 0x0904, thread 2, time  6358, arg1 0x48001d24, arg2 0x00000000
    Record 03: type 0x0905, thread 2, time  6424, arg1 0x00000002, arg2 0x00000000
    Record 04: type 0x0906, thread 2, time  6490, arg1 0x00000000, arg2 0x00000000
    Record 05: type 0x0901, thread 2, time  6608, arg1 0x48009d74, arg2 0x48001d24
    Record 06: type 0x0201, thread 2, time  6804, arg1 0x48001cd8, arg2 0x480013e0
    Record 07: type 0x0803, thread 1, time    94, arg1 0x00000000, arg2 0x00000000
    Record 08: type 0x0801, thread 1, time   361, arg1 0x00000000, arg2 0x00000000
    Record 09: type 0x0802, thread 1, time   548, arg1 0x00000001, arg2 0x00000000
    Record 10: type 0x0803, thread 1, time    94, arg1 0x00000000, arg2 0x00000000
    Record 11: type 0x0801, thread 1, time   361, arg1 0x00000001, arg2 0x00000000
    Record 12: type 0x0903, thread 1, time   513, arg1 0x48009d74, arg2 0x48001d24
    Record 13: type 0x0208, thread 1, time   588, arg1 0x00000000, arg2 0x00000000
    Record 14: type 0x0203, thread 1, time   697, arg1 0x48001cd8, arg2 0x480013e0
    Record 15: type 0x0802, thread 1, time   946, arg1 0x00000002, arg2 0x00000000
    Record 16: type 0x0201, thread 1, time  1083, arg1 0x480013e0, arg2 0x48001cd8
    Record 17: type 0x0000, thread 0, time     0, arg1 0x00000000, arg2 0x00000000
    Record 18: type 0x0000, thread 0, time     0, arg1 0x00000000, arg2 0x00000000
    Record 19: type 0x0000, thread 0, time     0, arg1 0x00000000, arg2 0x00000000

Debugging Techniques
To Contents | To previous page | To next page