https://www.eetimes.com/document.asp?doc_id=1255047

Successfully Using Linux and Open Software in an Embedded System Design
By Greg Rose, 06.10.02 

Fueled by time-to-market pressures and the need to reduce overall system cost,
many developers are migrating to Linux and open-systems software in their
embedded systems designs. This open-systems approach allows developers to
leverage freely available software, and spend their resources on their specific
area of expertise to add value to their products. Embedded systems present
unique design challenges such as: memory restrictions, booting from flash,
diskless operations, and requirements for deterministic operation. There are
some real challenges that face an embedded designer who wishes to use Linux in
his product. This article presents questions developers should ask while
planning embedded designs, how Linux stacks up, and how to overcome some of
those challenges.


Is the operating system available for the desired microprocessor?

There are several criteria used to evaluate and choose an operating system for
an embedded system or product: availability, resource usage, software
available, feature set, reliability, and performance. Simply whether the
operating system is available for the chosen embedded microprocessor can be the
major decision factor. Because of the short time-to-market of many of today's
embedded products, a port of an operating system to a new CPU or processor
board may take too long. Having an operating system that already runs on the
board saves the time and the expense of porting it.


Linux Support for Embedded Processors

The most popular version of Linux by far is the x86 version for PC-compatible
computers. PCs are inexpensive and can serve as both a development host and a
target. Most of the Linux developers around the world have PCs and develop for
it first. Many embedded systems use x86 processors. There are many variants x86
from very fast to very low power. A complete PC compatible computer can fit on
a single chip. For some embedded developers an x86 version of Linux is all that
is needed. But in some cases an x86 processor is not the best choice. In the
ultra low power range other processors such as the ARM or Hitachi SH processors
can provide the same performance level and use less power. For some
applications micro-controllers with PowerPC CPU cores, for instance, have the
right built in I/O and coprocessors to reduce hardware costs. For some
performance ranges there are MIPS processors that have a better prices
performance ratio than any x86 processor. So the embedded microprocessor market
is still fragmented. In fact it is as fragmented as it ever was and there is no
sign of consolidation.

The good news is that Linux has been ported to many different embedded
microprocessors including the PowerPC, MIPS, ARM, StrongArm, and SH. The bad
news is that these ports of Linux do not yet enjoy the amount of third-party
software that the x86 port of Linux has. Now that there are vendors for
embedded Linux making periodic releases of their Linux distributions for
non-x86 platforms, the situation should improve. A growing number of
third-party software vendors have pledged support for Linux on a variety of
embedded processors. But customers provide the strongest motivation. Inquiries
about the availability of an application, or protocol stack, for Linux on a
specific processor may speed the porting processes.


Can the operating system support headless and diskless operation?

Embedded systems rarely have a computer operator and most run as headless
systems, meaning that there is no operator interaction required to start and
run the system.


Linux in Headless and Diskless Operations

Linux was designed originally as a desktop operating system and it follows the
UNIX model. A text console is used to interface with the computer operator. The
console is used for controlling booting the operating system, giving the system
commands, and monitoring the systems activity. Embedded systems rarely have a
computer operator. So an embedded Linux must be setup not to rely on one. Linux
like all flavors of UNIX, uses the file system to store and locate executable
programs, and persistent data. Even system devices are referenced by special
files on in the file system. A file system on some type of disk device is an
integral part of the Linux execution environment.

Making a headless Linux system is not too hard. The console device can simply
be a null device. There is no operator interaction required if the startup
scripts simply start the application programs instead of login processes.
Monitoring can be done remotely if needed by setting a pseudo TTY as the
console once a remote login is established. Linux does support graphical user
interfaces with X-windows. But X is strictly optional and not required for the
operating system to run.

The problem of running without a disk can be solved in a few ways. There exist
Flash memory devices that emulate IDE disk drives. These devices can be
physically very small, not much bigger than the IDE connector itself. Linux
also includes a RAM disk driver. This is a device driver that uses some amount
of available system memory to emulate a disk drive. A RAM disk can be used for
data files that do not have to be persistent across a system reboot. It is also
possible to use the RAM disk driver to access a file system in ROM. If the
embedded system has Flash memory that does not emulate an IDE drive, a flash
file system can be used to treat the flash memory like a disk. A good flash
file system will use wear leveling to keep flash block erases to a minimum.


Can the OS Operate With Limited Memory?

All operating systems require some resource usage in order to operate. The main
resource required is computer memory. Some operating systems have a larger
memory "footprint" than others. In embedded systems where both RAM and ROM is a
precious resource, an operating system with too large a footprint must be ruled
out.


The Linux Memory Footprint

By default Linux has a big memory footprint, too big for most embedded systems.
The Linux distribution is hundreds of megabytes. A typical Linux kernel is 1.5
megabytes uncompressed. RAM use by the kernel for a default configuration is
over 4 megabytes. Linux also requires a file system for operation, and if the
embedded system does not have a disk this means more memory will be used in the
form of a RAM disk.

The solution to the memory footprint issue of Linux is through careful
configuration of the Linux kernel. The Linux kernel is modular, and to a
certain degree unused kernel facilities can be configured out. Also, kernel
resources can be configured such as the maximum number of processes or open
files on the system. By configuring the operating system modules and resource,
the size of the ROM and RAM footprint can be reduced. It is possible to
configure an "off-the-shelf" x86 Linux kernel to be 259K uncompressed. A
minimal ROM disk to provide a file system for this kernel can be as small as
102K. The total RAM usage can be under 4 Mbytes. A Linux capable of TCP/IP
networking is larger. The kernel with networking configured is about 370K, and
the required ROM disk is 740K (both uncompressed). RAM usage in this
configuration of Linux is under 8 Mbytes. This is not as small as an embedded
RTOS can achieve but small enough for many embedded devices.


Are there software development tools for embedding the operating system? Is the
operating system compatible with other application software needed?

When developing embedded applications, developers must use software tools such
as compilers and debuggers and these tools cannot commonly run on the embedded
target due to resource constraints. Embedded development requires running the
software development tools such as a compiler, assembler, and linker on a host
computer then downloading them to a target computer for execution. The host and
target may have different CPU types. In such case a compiler and assembler
intended for native use cannot be used as a cross compiler and assembler. A
cross compiler will have to be built or acquired that can run on the host and
create machine code for the target.

Embedded operating systems, like desktop and server operating systems, are many
times chosen not for their feature set but for the software available to run on
them. Many people run the Windows operating system on their desktop computer
because they can get the word processors or spreadsheet programs they need to
use for Windows. Embedded systems designers need software beyond the operating
system. They require protocol stacks, middleware, device drivers for special
hardware, and sometime the actual application software. Whether or not this
software is available for the embedded operating system they are considering
without the need of porting is a very strong factor in their operating system
choice. The feature set of an operating system becomes an important factor if
the application software is going to be developed for the embedded system and
is not already available. The availability of a certain feature may reduce the
software development time. Of course, software that needs to be ported has been
written assuming some set of features of the underlying operating system. If
those features are not available, it may mean a long and costly port.


Software-Development Tool Support for Linux

Most Linux software development is done natively. Programs are written and
executed as well as debugged all on the same Linux system. Once the program is
working its executable image or a "package" containing its executable image can
be copied to other Linux computers for execution. This method of software
development can be used for embedded systems as well providing the embedded
system has sufficient memory and disk space resources to run the development
tools. This typically is not the case, however. So cross development is
required.

For cross development, the popular GNU compilers can be used. If the host and
target CPU byte order is different such as an x86 host and a PowerPC target,
sometimes bugs will surface. When using cross development, some method of
downloading a kernel image to the target must be chosen. If the target can
access a disk drive (and can boot from disk), the drive can be mounted on the
host computer to write the kernel and file system and then moved to the target
for booting. If the target can boot an operating system over a network
interface, the host computer can simply run a boot server and a file server and
the target can do a network boot. However, if the target can only boot from ROM
or Flash memory, a special utility may be required to create a single load
image given a Linux kernel and a file tree.

The easiest way to get a workable cross development environment for Linux is to
pick up one of the Linux distributions designed for embedded use. These
distributions have pre-built cross tools for several embedded processor types.
Any byte order related bugs have usually been fixed. These distributions
typically include booting code for the target as well as tools to configure and
build Linux runtime images that can boot over the network or from Flash.

Cross development also requires cross debug support. In a cross debug
environment the debugger runs on the development host. The debugger included on
Linux distributions is GDB, the GNU debugger. GDB can run native or cross. In
native mode GDB executes or attaches to the executing application to be
debugged. GDB then can examine the variables and data structures of the
application program or library through the /proc interface. GDB can also trace
the program, be notified of signals being sent to the executing program, and
send signals itself. GDB reads the symbol information from the executable file
of the program and can also display the source from the programs source file.
In a cross debug environment the debugger executes on the host development
computer, and the application program runs on the target computer. The debugger
reads the source and executable file on the host to get source code and symbol
table information. The debugger communicates to an agent running on the target
in order to examine variables and data structures and to control the process
being debugged. The agent for GDB is the GDB Server and it is a lot smaller
than GDB itself. GDB communicates to the GDB server either through a TCP/IP
connection or through a raw serial connection.

The Linux kernel and device drivers cannot be debugged the same way as a user
application. Setting a breakpoint in the kernel or otherwise stopping execution
of a process while it is running in the kernel would cause the debugging agent
to stop. Complete debug ability for the Linux kernel and device drivers is not
supported in most desktop distributions of Linux. Some embedded distributions
do support full kernel debugging. In this case GDB attaches to a debugging
agent in the kernel usually through a low-level serial connection. This way the
kernel or a device driver can be single stepped or breakpoints used even in
interrupt routines. The only thing that cannot be debugged this way is the
kernel debug agent itself.


Third-Party Software Availability for Linux

There is more and more software available for Linux, not just open-source
software, but third-party application programs and protocol stacks as well. The
number of device drivers available for Linux is large and growing and the
source code for Linux device drivers is almost always free. Linux has a good
reputation for reliability on servers, and it has good network and file system
performance.

Not only is the source code to Linux free, but runtimes of Linux are royalty
free. For high volume embedded systems with tight per unit cost constraints,
having no royalty payments for the embedded operating system is very
attractive. It saves money that would go to royalty payments and eliminates the
overhead for keeping track of the number of runtimes shipped.

The programming interface for Linux is UNIX. A major reason Linux has so much
software available for it is that Linux uses the UNIX application programming
interface as well as UNIX object and executable file formats. Much of the
public domain and open source software of the last two decades has been written
for UNIX and is now available for Linux. Also, third-party software developers
have been providing products for flavors of UNIX for years such as AIX from
IBM, Solaris from Sun Microsystems, and HP/UX from Hewlett-Packard. It was a
simple matter for these software vendors to port their products to Linux. This
also means that there are programmers experienced on writing UNIX software that
already have the expertise to write Linux software.

Because of the popularity of Linux, there are already several software vendors
that provide releases of Linux aimed at embedded systems. These vendors
typically offer technical support programs for Linux that is geared toward
embedded system development. This support includes bug fixes for Linux versions
that may be years old and help with ports to embedded devices.


Is the operating system reliable enough and does it meet performance
requirements?

Reliability and performance are two features of an operating system that
usually cannot be augmented with application or library level software. Running
reliable application software on an unreliable operating system simply makes
the application unreliable. The reliability of an operating system is based on
how well it was conceived, designed, and coded. Most embedded systems require a
higher level of reliability than desktop systems. So choosing a reliable
operating system is important. Of course, all operating system vendors claim
that their operating systems are reliable, so an evaluation is often necessary
to discern the facts. Performance can also be benchmarked. It is usually
straightforward to determine if the operating system can meet the embedded
systems throughput requirements. What is harder to determine is will the
operating system meet the real-time requirements. Real-time performance can be
benchmarked, but usually the difficulty lies with determining the real-time
requirements of the embedded application. The real-time characteristics of
system software can vary greatly as the system is put under load. All too often
it is discovered that an operating system was the wrong choice because it could
not meet real-time requirements that are not discovered until near the end of
product development.


Linux Reliability and Performance

Linux is not a real-time operating system. The Linux kernel uses a fair share
scheduling algorithm designed for time sharing, not a strict priority
preemptive scheduler or some other scheduler suitable for real-time. But that
is the least of the linux real-time problems. The Linux kernel is not
preemptive nor reentrant by user processes. If one process is using a kernel
facility no other process can execute until the process finishes or waits on
something. This means the worst case response time of any real-time task
running under the Linux kernel is longer than the time it takes to execute the
longest stretch of kernel code before a wait. This time is unknown. The
worst-case response time is further increased by memory cycle stealing by DMA
devices and the CPU cycle stealing for the execution of interrupt handlers.
There is no bound on the amount of CPU time used by interrupt handlers under
Linux, and these run at a higher priority than any user process. Practically
speaking, a user processes running on a very fast Pentium computer that is
running a heavy load my take hundreds or even thousands of milliseconds to
respond.

There are several ways to deal with the problem that Linux has poor real-time
performance—ignore the problem, or run real-time applications under an RTOS
with Linux itself as a separate task as in the case of RT-Linux, or run a Linux
compatible RTOS kernel instead of the Linux kernel. The most popular approach
is to ignore the problem. In most cases embedded developers have a problem
identifying their real-time requirements at the beginning of a project anyway.
This method my work if the application has extremely loose real-time
requirements. If it is discovered that Linux is not meeting a real-time
requirement then one of the other approaches can always be taken.


RT-Linux

RT-Linux is a distribution of Linux that runs the Linux kernel as a task under
a small real-time executive. Real-time tasks are run directly under the
real-time executive and non-real-time tasks are run under Linux. This approach
is not new. Years ago vendors marketed products to use this technique with
commercial UNIX, and today several vendors use this approach to allow a
computer running Windows NT to also execute real-time tasks. One of the
advantages of the approach is that the real-time executive can be small and
simple making it easier to verify its real-time performance. Another advantage
is that the Linux kernel running non-real-time tasks is "standard" Linux so it
is compatible with other Linux distributions and can be updated to a new
release quickly. This approach also support hard real-time applications, ones
where deadlines must be met absolutely every time.

There is certainly a set of real-time applications that can benefit from the
RT-Linux approach. Machine control, process control, and instrumentation
applications where the real-time application code is a small part of the entire
application software should run under RT-Linux fine. Using RT-Linux is a much
better approach than ignoring real-time requirements.

Some real-time applications do not fit the RT-Linux model well. Data
acquisition to mass storage, telephone call setup or any requiring a real-time
database, any application where the real-time part of the software is large or
not easily separated from the code that needs a Linux environment. The problem
is that tasks running under the real-time executive do not have access to the
Linux facilities, device drivers, etc. And tasks running under the Linux kernel
and can access the facilities are not real-time. As programmers apply the
RT-Linux type dual kernel approach to applications where it is not a good fit
the number of facilities in the real-time kernel will undoubtedly increased. At
some point the real-time executive will no longer be small and simple, and as
it adopts more of the facilities of the non-real-time kernel the question of
whether the non-real-time kernel is required comes into doubt.


A Linux-Compatible Real-Time Kernel

Another approach to the problem of running real-time applications on Linux is
to replace the Linux kernel with a compatible kernel that has hard real-time
performance characteristics. This may sound like sacrilege, but even in the
RT-Linux case running real-time tasks under a Linux kernel is already
abandoned. Also, with the advent of the Linux Standard Base (LSB), a binary
interface standard for Linux, Linux becomes more than a technology but a
platform for running applications. Any operating system that can meet the
standard can run those Linux applications. The kernel is only a small part of a
whole operating system. If just the Linux kernel is replaced and the libraries,
utilities, and file structure remain the same, it's even easier to be
compatible and conforming to the standard. The real-time kernel would have to
support all the facilities of a Linux kernel while still remaining fully
preemptive and reentrant. It would have to have a real-time scheduler and have
bounded execution time for interrupt handlers.

The advantage of the replacement kernel approach is that there would not be a
limit on the size or complexity of the real-time application code. Third party
software not originally envisioned for real-time use could be invoked by
real-time tasks. Programmers who understand Linux would already be experts in
the real-time environment because the programming interface would be the same.


Other Considerations for Embedding Linux

Despite the good reasons for choosing Linux for an embedded system or device,
Linux does have some problems in the areas of licensing issues. Linux is not
public domain software, it is licensed according to the Gnu Public License,
which has a strict set of rules for use.


Legal Issues

Linux is licensed under the terms of the Gnu Public License (GPL). Source code
for any software under the GPL is free. Runtimes generated from source code
under the GPL must also be free of charge. Any modifications to software under
a GPL license automatically come under the GPL as well. You can't sell runtimes
of GPL code for more than the cost of the distribution media and cost of
distribution. If runtimes are distributed, the source code must be made public
and readily available.

What does GPL mean to the embedded Linux developer? If you take the Linux
kernel or any Linux utility and modify it, port it, or add features to it, you
must post your source to the Internet or ship the source to anyone who asks for
it. If you are not careful you may give up the rights to your proprietary
software unintentionally. It is best to consult your attorney on all
open-source copyright issues.

It is OK to ship Linux in a hardware device being sold. If any modifications to
the Linux kernel, libraries, or utilities were required, the sources for the
modified version must be made available free. Application programs that run
under Linux may remain proprietary. Just be sure no part of the application is
GPL (or other open source license) code. Device drivers can also be kept
proprietary. They must be shown to be separate and distinct from the Linux
kernel. This is most easily done if the device driver is written as a loadable
kernel module. If it is linked as part of the base kernel it is in the gray
area of whether it becomes GPL or not. If in doubt consult your intellectual
property attorney.


Conclusion

Although Linux was not designed with embedded systems in mind, through careful
configuration and by leveraging some of the work of the embedded Linux vendors,
Linux can be successfully embedded in hardware products. Linux is emerging as
more than just a technology but also a platform for embedded applications.
Linux does have some problems in the areas of licensing issues, memory
footprint, software development, headless and diskless operation, support for
embedded processors, and real-time characteristics. Linux is not public-domain
software, it is licensed according to the Gnu Public License, which has a
strict set of rules for use. But because of its popularity, Linux will be
considered for more and more embedded applications. When considering the main
criteria for choosing an embedded system—availability of hardware and
availability of third-party software, Linux has a lot going for it.

For more information about Linux, visit the LynuxWorks Web site.

All contents are Copyright © 2018 by AspenCore, Inc. All Rights Reserved.