2. Solaris Booting and Configuration

The Solaris (Intel Platform Edition) booting and configuration framework is best understood in the context of Solaris booting and configuration in general, since the features of the Solaris (Intel Platform Edition) mechanisms have been designed to service the needs of Solaris booting and initialization. The Solaris booting model makes several assumptions:

That the configuration (device types and assigned I/O resources) for all "boot-time devices" (for example, the boot device, the keyboard, and the console display) is knowable, so that a single set of modules can be used to boot any platform
That each boot-time device is supported by an adequate device driver (presumably in firmware), and that the device has already been initialized so that it is ready for use
That the bootstrap is capable of loading a couple Solaris kernel modules from the soon-to-be root device (which may be a UFS file system on disk, or the NFS environment accessed over a network, or some other format)
That once the basic Solaris kernel has been loaded, the bootstrap will continue to provide sufficient services to permit the fledgling kernel to find and load additional modules

2.1 Open Firmware Model

Virtually all computer systems include a minimal ROM-based operating system that performs power-on self-tests, initializes the system, and initiates the bootstrap process. Most of these systems also provide a few simple device drivers that a common operating system can initialize on a wide range of platforms (as opposed to having to know how to find, initialize, and use the particular set of devices on every possible system). On x86 based systems these services are provided by the ROM BIOS. On SPARC based systems these services are provided by Open Firmware.

The Open Firmware specification (defined by IEEE 1275 and a host of platform-specific bindings) describes a set of ROM services that very effectively hide most of the details of a particular platform:

There is a device information (devinfo) tree that hierarchically represents the buses, controllers, and devices on the system.
Properties associated with each node of the tree define the type of the device, where it is located, and what I/O resources (for example, interrupt levels, DMA levels, bus addresses) have been allocated to that device.
The standard services provided by 1275 firmware include methods for walking the tree and querying the properties associated with each node.
A firmware device driver is associated with each node, and can be used to read and write the associated device. These drivers are written in Forth (FCode) so that they should, in principle, be both platform and Instruction Set Architecture (ISA) independent.

For more detailed information, consult the 1275 Open Firmware specification or its processor-specific bindings.

        i86pc
            isa, instance #0
                asy, instance #0, chosen-interrupt=4, address=0x3f8, size=8
                asy, instance #1, chosen-interrupt=3, address=0x2f8, size=8
                fdc, instance #0, chosen-interrupt=6, address=0x3f0, size=6
                                                      address=0x3f6, size=1
                    fd, instance #0, unit=0
                    fd, instance #1, unit-1
                kd, instance #0,  chosen-interrupt=1
            pci, instance #0
                pci1000,4, instance #0, chosen-interrupt=9
                    cmdk, instance #0, lun=0, target=0
                    cmdk, instance #1, lun=0, target=1
                    cmtp, instance #2, lun=0, target=2

Figure 2-1 Sample Device Information Tree

As explained in subsequent chapters, x86 BIOS does not reliably provide any of these services, and much of the work in the Solaris (Intel Platform Edition) realmode environment goes into making a relatively chaotic PC work as if it had a good Open Firmware implementation.

2.2 Solaris Boot Process

The earliest portions of the Solaris boot process are completely platform dependent, but they somehow culminate in the loading of a second-stage bootstrap, that somehow knows what kernel is to be loaded from what file system. The second-stage bootstrap has access to sufficient device drivers to enable it to talk to all of the boot-time devices, and it knows how to interpret a variety of file system formats and how to access remote file systems via NFS.

The second-stage bootstrap reads a couple of modules into memory (a minimal UNIX kernel and the kernel runtime loader krtld) that can be used to load and link additional modules into the kernel. The bootstrap then sets up a virtual address space for the newly loaded modules and transfers control to them. The second-stage bootstrap also leaves behind a call vector that can be used (by the fledgling kernel) to allocate memory, traverse the device information tree, and read files from the root file system.

Armed with these services, the kernel examines the devinfo tree, reads configuration files, determines what device drivers and other modules it will need to load, allocates memory, reads them in, and binds them into the kernel. When the kernel has assembled all of the modules it needs to run, it makes a "quiesce" call into the bootstrap, after which the kernel takes over the handling of interrupts. At this point the booting process has completed, and Solaris is now running unassisted.

2.3 Device Configuration

The means by which the initial device information tree is assembled are completely platform specific. It may be hard-coded in ROM, automatically generated by ROM bus and device drivers, or (on x86 based systems) built up by the higher-level software. However this tree is built, it needs to exist by the time the Solaris kernel is loaded. It need not contain all of the devices on the system, but it must contain:

All boot-time devices
All top-level buses
All non-self-identifying devices

The boot-time devices must be known up-front because they will be used to load the system into memory, and because the Solaris software must have all of these drivers loaded into memory prior to quiescing the bootstrap.

The buses must be known so that the kernel can load the appropriate nexus drivers. These, in turn, can be used to enumerate the devices on those buses.

Sun requires that all non-self-identifying devices be represented in the initial devinfo tree because the alternative would be to have a succession of Solaris drivers probe through the I/O space in search of hidden devices. Such probing, while possible, is both slow and dangerous. It is slow because all possible drivers have to probe at all possible locations. It is dangerous because tweaking random bits on arbitrary devices can have a wide range of bad effects on a running system.

Devices that are not needed to bring up the basic system and that are located on self-identifying buses (for example, a PCI sound card) can be safely and efficiently discovered by the associated Solaris nexus drivers later during system initialization. As such, it is not required that these devices be noted in the initial devinfo tree.

If a bus supports hot-pluggable devices, the associated bus nexus driver must be able to recognize insertion events in real time, identifying and configuring new devices as they are inserted. Since the Solaris software will have to be able to identify those devices, there is no need for them to be represented in the initial devinfo tree.

2.4 Device Drivers and Name Spaces

Solaris device drivers exist and operate in a hierarchy. At the bottom of the hierarchy are the leaf node drivers that directly manage a particular device (for example, a serial port or a SCSI target). Above these are bus nexus drivers that manage bus resources (for example, interrupts and DMA) for the leaf node drivers. Nexus drivers may themselves be served by higher-level nexus drivers.

root, nexus, and leaf node drivers

Figure 2-2 Root, Nexus, and Leaf Drivers in Device Information Tree

It is worth noting that there are also "pseudo device drivers" that look and act like device drivers, but do not have any peripheral devices associated with them. An example might be a RAM disk driver that emulates a disk, but stores all of the data in wired-down memory.

Solaris applications access devices through "special files." Associated with each special file is a "major" and "minor" device number. The major device number selects one of the many leaf or nexus drivers that is loaded into the kernel. The minor device number selects one instance, from all of the units that are being controlled by that device driver. For more information on special files and Solaris device drivers, refer to Section 7 of the Solaris Reference Manual.

The association of major device numbers with drivers is system specific and controlled by the file /etc/name_to_major (a sample name_to_major file is included in this document). The allocation of major device numbers is described in add_drv(1M). The association of minor device numbers with instances is driver specific, and affected by the order in which the instances are discovered.

Solaris device drivers are dynamically loaded into the kernel when the associated special files are first opened. Because nexus drivers provide services for leaf node drivers, a leaf node driver cannot work unless all of the nexus drivers above it have already been loaded and initialized.

Each Solaris system has two sets of special files. The first, the "physical device tree" has the same nexus and leaf structure as the firmware-supplied device information tree, with subdirectories for each nexus driver, and special files for each leaf driver instance. The physical device tree can be found in /devices. The name of each directory or special file identifies the drivers and the resources associated with each unit. Consider the example: /devices/isa/ata@1f0,0/cmdk@0,0:a

The interpretation of this name is:

Under the root node is an ISA bus
On the ISA bus, at address 0x1f0 is an ATA disk controller
On that disk controller is a unit 0
That disk has multiple slices, and this special file refers to the "a" (first) slice

The second set of special files is called the "logical device tree." This collection of special files is less formally structured, and is organized by general device class rather than by physical hardware types and topology (for instance, with all of the disks grouped together, rather than all of the devices on the same bus grouped together).

The logical device tree can be found under the /dev directory. The same ATA disk partition described above might appear in the logical device tree as: /dev/dsk/c0d0s0. In this disk device name, c0 refers to the first controller, d0 to the first device, and s0 to the first slice.

The /devices tree reflects the physical structure of the hardware and the hierarchical relations among the various device drivers. The /dev tree presents a more user-centric view of devices (based on what kinds of device they are, rather than their electrical connectivity).

For detailed examples, see the sample /devices and /dev trees included in this white paper.

2.5 Special Booting Situations

With one interesting exception, the only way a Solaris device driver can be accessed is through associated special files. Since the set of special files describing the devices on a system are system specific, it is not possible to access any physical devices until the /devices and /dev trees have been initialized. There are some special system startup options to (re)create the physical and logical device hierarchies. Most of the interesting activity takes place long after the bootstrap has been quiesced, but these special booting situations impose some interesting requirements on the bootstrapping mechanisms.

The one exception to the rule that devices can only be accessed through special files is the device containing the root file system. The bootstrap passes a physical device path (for example, /isa/ata@1f0/cmdk@0,0:a) into the kernel, and the kernel walks the device information tree and hand-assembles the subset of the device driver tree (the "root stick") that is needed to access the root file system. Given access to the root file system (with all of its configuration files and drivers), it is possible to create all of the other special files.

2.5.1 Reconfiguration Reboots

The simpler special booting case is called a "reconfiguration reboot." UNIX applications access devices through special files. A device that does not have a special file cannot be used. Each system must have a /dev directory hierarchy with special files that properly correspond to each of its devices. The Solaris software automatically recognizes and deals with the insertion and removal of hot-pluggable devices by creating and deleting special files. There is, however, no automatic mechanism for updating the /dev hierarchy in response to changes in statically configured devices (for example, the addition or removal of a SCSI HBA).

A Solaris system can be booted (or rebooted) with the "-r" option, which specifies that this is a "reconfiguration reboot." In a reconfiguration reboot, before starting up any other user processes, a set of programs is run to analyze the devinfo tree in the kernel, and to construct a /devices hierarchy that mirrors it. After this has been done, a /dev hierarchy will be created to associate logical names with the physical devices.

Only a few special files (the root device and console) are needed to boot a system and reconstruct the /devices and /dev hierarchies. These files are created properly when the system is initially installed. Once a reconfiguration reboot has completed, all of the special files should be correct.

2.5.2 Installation Boots

When the Solaris environment is initially installed, it will be booted from a read-only NFS root image or from a CD-ROM. Clearly, the /dev and /devices hierarchies (and other configuration files ) on this read-only root image do not reflect the hardware on the machine being booted. A very early startup script on the installation mini-root creates and mounts a RAM-based temporary file system (tmpfs). All of the files and directories on the mini-root that would normally be writable on a Solaris root file system have been replaced with symbolic links into tmpfs.

The initial boot of the new client specifies the reconfiguration option, and so appropriate /dev and /devices hierarchies (that accurately describe the hardware on the new machine) are created (in tmpfs).

The entire installation is carried out under Solaris, running on an essentially read-only root file system, with a few writable and machine-specific files in tmpfs. This installation system will open the special files for the target disks, create the necessary file systems, and install all of the desired software onto them. Only after the installation has been completed will the machine reboot (this time from the newly installed root disk).

Solaris (Intel Platform Edition) Realmode Environment

Contents