Mercurial > ecos
diff packages/services/memalloc/common/current/doc/notes.txt @ 115:6ed91473a1cd ecos-sw-2000-08-21
Merge from eCos master repository on 2000-08-21-22:40:54-BST
| author | jlarmour |
|---|---|
| date | Fri, 25 Aug 2000 17:32:38 +0000 |
| parents | |
| children | e0c0827131d1 |
line wrap: on
line diff
new file mode 100644 --- /dev/null +++ b/packages/services/memalloc/common/current/doc/notes.txt @@ -0,0 +1,352 @@ + Memory allocation package - Implementation Notes + ------------------------------------------------ + + + +Made with loving care by Jonathan Larmour (jlarmour@redhat.com) +Initial version: 2000-07-03 +Last updated: 2000-07-03 + + + +Meta +---- + +This document describes some interesting bits and pieces about the memory +allocation package - CYGPKG_MEMALLOC. It is intended as a guide to +developers, not users. This isn't (yet) in formal documentation format, +and probably should be. + + +Philosophy +---------- + +The object of this package is to provide everything required for dynamic +memory allocation, some sample implementations, the ability to plug in +more implementations, and a standard malloc() style interface to those +allocators. + +The classic Unix-style view of a heap is using brk()/sbrk() to extend the +data segment of the application. However this is inappropriate for an +embedded system because: + +- you may not have an MMU, which means memory may be disjoint, thus breaking + this paradigm + +- in a single process system there is no need to play tricks since there + is only the one address space and therefore heap area to use. + +Therefore instead, we base the heap on the idea of fixed size memory pools. +The size of each pool is known in advance. + + +Overview +-------- + +Most of the infrastructure this package provides is geared towards +supporting the ISO standard malloc() family of functions. A "standard" +eCos allocator should be able to plug in to this infrastructure and +transparently work. The interface is based on simple use of C++ - nothing +too advanced. + +The allocator to use is dictated by the +CYGBLD_MEMALLOC_MALLOC_IMPLEMENTATION_HEADER option. Choosing the +allocator can be done by ensuring the CDL for the new allocator +has a "requires" that sets the location of the header to use when that +allocator is enabled. New allocators should default to disabled, so they +don't have to worry about which one is the default, thus causing CDL +conflicts. When enabled the new allocator should also claim to implement +CYGINT_MEMALLOC_MALLOC_ALLOCATORS. + +The implementation header file that is set must have a special property +though - it may be included with __MALLOC_IMPL_WANTED defined. If this +is the case, then this means the infrastructure wants to find out the +name of the class that is implemented in this header file. This is done +by setting CYGCLS_MEMALLOC_MALLOC_IMPL. If __MALLOC_IMPL_WANTED is defined +then no non-preprocessor output should be generated, as this will be included +in a TCL script in due course. An existing example from this package would +be: + +#define CYGCLS_MEMALLOC_MALLOC_IMPL Cyg_Mempool_dlmalloc + +// if the implementation is all that's required, don't output anything else +#ifndef __MALLOC_IMPL_WANTED + +class Cyg_Mempool_dlmalloc +{ +[etc.] + +To meet the expectations of malloc, the class should have the following +public interfaces (for details it is best to look at some of the +examples in this package): + +- a constructor taking arguments of the form: + + ALLOCATORNAME( cyg_uint8 *base, cyg_int32 size ); + + If you want to be able to support other arguments for when accessing + the allocator directly you can add them, but give them default values, + or use overloading + +- a destructor + +- a try_alloc() function that returns new memory, or NULL on failure: + + cyg_uint8 * + try_alloc( cyg_int32 size ); + +- a free() function taking one pointer argument that returns a boolean + for success or failure: + + cyg_bool + free( cyg_uint8 *ptr ); + + Again, extra arguments can be added, as long as they are defaulted. + + +- resize_alloc() which is designed purely to support realloc(). It + has the prototype: + cyg_uint8 * + resize_alloc( cyg_uint8 *alloc_ptr, cyg_int32 newsize, + cyg_int32 *oldsize ); + + The idea is that if alloc_ptr can be adjusted to newsize, then it will + be. If oldsize is non-NULL the old size (possibly rounded) is placed + there. However what this *doesn't* do (unlike the real realloc()) is + fall back to doing a new malloc(). All it does is try to do tricks + inside the allocator. It's up to higher layers to call malloc(). + +- get_status() allows the retrieval of info from the allocator. The idea + is to pass in the bitmask OR of the flags defined in common.hxx, which + selects what information is requested. If the request is supported by + the allocator, the approriate structure fields are filled in; otherwise + unsupported fields will be left with the value -1. (The constructor for + Cyg_Mempool_Status initializes them to -1). If you want to reinitialize + the structure and deliberately lose the data in a Cyg_Mempool_Status + object, you need to invoke the init() method of the status object to + reinitialize it. + + void + get_status( cyg_mempool_status_flag_t flags, Cyg_Mempool_Status &status ); + + A subset of the available stats are exported via mallinfo() + + +Cyg_Mempolt2 template +--------------------- + +If using the eCos kernel with multiple threads accessing the allocators, +then obviously you need to be sure that the allocator is accessed in a +thread-safe way. The malloc() wrappers do not make any assumptions +about this. One helpful approach currently used by all the allocators +in this package is to (optionally) use a template (Cyg_Mempolt2) that +provides extra functions like a blocking alloc() that waits for memory +to be freed before returning, and a timed variant. Other calls are +generally passed straight through, but with the kernel scheduler locked +to prevent pre-emption. + +You don't have to use this facility to fit into the infrastructure though, +and thread safety is not a prerequisite for the rest of the infrastructure. +And indeed certain allocators will be able to do scheduling at a finer +granularity than just locking the scheduler every time. + +The odd name is because of an original desire to keep 8.3 filenames, which +was reflected in the class name to make it correspond to the filename. +There used to be an alternative Cyg_Mempoolt template, but that has fallen +into disuse and is no longer supported. + + +Automatic heap sizing +--------------------- + +This package contains infrastructure to allow the automatic definition +of memory pools that occupy all available memory. In order to do this +you must use the eCos Memory Layout Tool to define a user-defined section. +These sections *must* have the prefix "heap", for example "heap1", "heap2", +"heapdram" etc. otherwise they will be ignored. + +The user-defined section may be of fixed size, or of unknown size. If it +has unknown size then its size is dictated by either the location of +the next following section with an absolute address, or if there are +no following sections, the end of the memory region. The latter should +be the norm. + +If no user-defined sections starting with "heap" are found, a fallback +static array (i.e. allocated in the BSS) will be used, whose size can +be set in the configuration. + +It is also possible to define multiple heap sections. This is +necessary when you have multiple disjoint memory regions, and no MMU +to join it up into one contiguous memory space. In which case +a special wrapper allocator object is automatically used. This object +is an instantiation of the Cyg_Mempool_Joined template class, +defined in memjoin.hxx. It is instantiated with a list of every heap +section, which it then records. It's sole purpose is to act as a go +between to the underlying implementation, and does the right thing by +using pointer addresses to determine which memory pool the pointer +allocator, and therefore which memory pool instantiation to use. + +Obviously using the Cyg_Mempool_Joined class adds overhead, but if this +is a problem, then in that case you shouldn't define multiple disjoint +heaps! + + +Run-time heap sizing +-------------------- + +As a special case, some platforms support the addition of memory in the +field, in which case it is desirable to automatically make this +available to malloc. The mechanism for this is to define a macro in +the HAL, specifically, defined in hal_intr.h: + +HAL_MEM_REAL_REGION_TOP( cyg_uint8 *regionend ) + +This macro takes the address of the "normal" end of the region. This +corresponds with the size of the memory region in the MLT, and would +be end of the "unexpanded" region. This makes sense because the memory +region must be determined by the "worst case" of what memory will be +installed. + +This macro then returns a pointer which is the *real* region end, +as determined by the HAL at run-time. + +By having the macro in this form, it is therefore flexible enough to +work with multiple memory regions. + +There is an example in the ARM HAL - specifically the EBSA285. + + +How it works +------------ + +The MLT outputs macros providing information about user-defined sections +into a header file, available via system.h with the CYGHWR_MEMORY_LAYOUT_H +define. When the user-defined section has no known size, it determines +the size correctly relative to the end of the region, and sets the SIZE +macro accordingly. + +A custom build rule preprocesses src/heapgen.cpp to generate heapgeninc.tcl +This contains TCL "set"s to allow access to the values of various +bits of configuration data. heapgen.cpp also includes the malloc +implementation header (as defined by +CYGBLD_MEMALLOC_MALLOC_IMPLEMENTATION_HEADER) with __MALLOC_IMPL_WANTED +defined. This tells the header that it should define the macro +CYGCLS_MEMALLOC_MALLOC_IMPL to be the name of the actual class. This +is then also exported with a TCL "set". + +src/heapgen.tcl then includes heapgeninc.tcl which gives it access to +the configuration values. heapgen.tcl then searches the LDI file for +any sections beginning with "heap" (with possibly leading underscores). +It records each one it finds and then generates a file heaps.cxx in the +build tree to instantiate a memory pool object of the required class for +each heap. It also generates a list containing the addresses of each +pool that was instantiated. A header file heaps.hxx is then generated +that exports the number of pools, a reference to this list array and +includes the implementation header. + +Custom build rules then copy the heaps.hxx into the include/pkgconf +subdir of the install tree, and compile the heaps.cxx. + +To access the generated information, you must #include <pkgconf/heaps.hxx> +The number of heaps is given by the CYGMEM_HEAP_COUNT macro. The type of +the pools is given by CYGCLS_MEMALLOC_MALLOC_IMPL, and the array of +instantiated pools is available with cygmem_memalloc_heaps. For example, +here is a sample heaps.hxx: + +#ifndef CYGONCE_PKGCONF_HEAPS_HXX +#define CYGONCE_PKGCONF_HEAPS_HXX +/* <pkgconf/heaps.hxx> */ + +/* This is a generated file - do not edit! */ + +#define CYGMEM_HEAP_COUNT 1 +#include <cyg/memalloc/dlmalloc.hxx> + +extern Cyg_Mempool_dlmalloc *cygmem_memalloc_heaps[ 2 ]; + +#endif +/* EOF <pkgconf/heaps.hxx> */ + +The array has size 2 because it consists of one pool, plus a terminating +NULL. + +In future the addition of cdl_get() available from TCL scripts contained +within the CDL scripts will remove the need for a lot of this magic. + + +dlmalloc +-------- + +A port of dlmalloc is included. Far too many changes were required to make +it fit within the scheme above, so therefore there was no point +trying to preserve the layout to make it easier to merge in new versions. +However dlmalloc rarely changes any more - it is very stable. + +The version of dlmalloc used was a mixture of 2.6.6 and the dlmalloc from +newlib (based on 2.6.4). In the event, most of the patches merged were +of no consequence to the final version. + +For reference, the various versions examined are included in the +doc/dlmalloc subdirectory: dlmalloc-2.6.4.c, dlmalloc-2.6.6.c, +dlmalloc-newlib.c and dlmalloc-merged.c (which is the result of merging +the changes between 2.6.4 and the newlib version into 2.6.6). Note it +was not tested at that point. + + +Remaining issues +---------------- + +You should be allowed to have different allocators for different memory +regions. The biggest hurdle here is host tools support to express this. + +Currently the "joined" allocator wrapper simply treats each memory pool +as an equal. It doesn't understand that some memory pools may be faster +than others, and cannot make decisions about which pools (and therefore +regions and therefore possibly speeds of memory) to use on the basis +of allocation size. This should be (configurably) possible. + + +History +------- + + +A long, long time ago, in a galaxy far far away.... the situation used to +be that the kernel package contained the fixed block and simple variable +block memory allocators, and those were the only memory allocator +implementations. This was all a bit incongruous as it meant that any code +wanting dynamic memory allocation had to include the whole kernel, even +though the dependencies could be encapsulated. This was particularly silly +because the implementation of malloc() (etc.) in the C library didn't use +any of the features that *did* depend on the kernel, such as timed waits +while allocating memory, etc. + +The C library malloc was pretty naff then too. It used a static buffer +as the basis of the memory pool, with a hard-coded size, set in the +configuration. You couldn't make it fit into all of memory. + +Jifl +2000-07-03 + +//####COPYRIGHTBEGIN#### +// +// ------------------------------------------- +// The contents of this file are subject to the Red Hat eCos Public License +// Version 1.1 (the "License"); you may not use this file except in +// compliance with the License. You may obtain a copy of the License at +// http://www.redhat.com/ +// +// Software distributed under the License is distributed on an "AS IS" +// basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the +// License for the specific language governing rights and limitations under +// the License. +// +// The Original Code is eCos - Embedded Configurable Operating System, +// released September 30, 1998. +// +// The Initial Developer of the Original Code is Red Hat. +// Portions created by Red Hat are +// Copyright (C) 1998, 1999, 2000 Red Hat, Inc. +// All Rights Reserved. +// ------------------------------------------- +// +//####COPYRIGHTEND####
