Solaris 8 Memory Architecture

Print-friendly Version

August 2000

The Solaris 8 Operating Environment incorporates an improved memory architecture that improves performance for desktop systems and servers. Better yet, it does not require any tuning, it is easier to monitor, and it provides better memory usage reporting to help you decide if you need more memory.

Use this table to locate subjects in this article:

Where Has All My Memory Gone?

A common question pre-Solaris 8 users ask is "Where has all my memory gone"? The vmstat command, used to report virtual memory statistics, often reports that free memory (measured in Kbytes in the free column) is zero or close to zero on a pre-Solaris 8 system that has been up and running for a while.

Most likely, memory is being used to cache file system data, since the virtual memory system is shared by applications, data, the kernel, and file system data. By default, any free memory is used to cache data read from or written to the file system (including NFS). The size of the file system cache is dynamic -- it grows or shrinks depending on free memory.

The idea of this memory allocation scheme is to simultaneously enhance file system performance and optimize the use of an important system resource -- virtual memory. The two computing tasks of running applications and reading and writing data compete equally for system memory.

Generally, sharing a pool of memory is not an issue on small memory systems with low compute power, but with today's powerful desktop systems and servers, the file system cache can overwhelm the memory pool and make application performance suffer. Another drawback is that file system performance is tied to how quickly the virtual memory system can free memory.

Even worse, it is difficult to measure memory usage amongst the consumers of memory on the system. The vmstat command is often the first tool users run to examine virtual memory usage, but pre-Solaris 8 versions do a poor job of indicating why a system is paging (running an algorithm that moves data out from physical memory to disk, and back into physical memory from disk).

So, the question becomes: is it because the system is caching file system data, or is it because memory is a bottleneck and the system is struggling to keep up?

Back to Top

The Least Recently Used Paging Algorithm

To better understand how the virtual memory system works, let's examine the least recently used paging algorithm (LRU). The LRU paging algorithm is controlled by the amount of available primary memory. If free primary memory falls below an established watermark, the system attempts to reclaim memory by scanning pages and looking for pages that have not been accessed recently. Pages that meet this criteria are moved to the free list, meaning they can be paged out to the swap device, thus freeing the page in primary memory.

For example, here are the watermarks used by the LRU paging algorithm for a system with 128 Mbytes of primary memory:

Back to Top

Priority Paging

To alleviate the pressure on virtual memory induced by heavy file system activity, priority paging was introduced for the Solaris 2.6 and Solaris 7 operating environments.

Priority paging introduces a new watermark in the LRU paging algorithm, cachefree:

Priority paging helps performance on desktop systems with more than 64 Mbytes of primary memory, especially when a windowing system such as the Common Desktop Environment (CDE) is used.

It is also a boon for servers running online transaction processing (OLTP) applications. OLTP applications will see better response time, as will applications that use batch processing, such as high performance computing (HPC) environments. Also, servers running mixed applications (for example, NFS and database systems) will benefit from priority paging.

By default, priority paging is not enabled on systems running the Solaris 7 operating environment. To enable priority paging, set the following system variable in the /etc/system file and reboot:

   set priority_paging=1

Priority paging has been back-ported for systems running the Solaris 2.6 operating environment (you must first install kernel patch 105181-09 or higher) and the Solaris 2.5.1 operating environment (you must install kernel patch 103640-25 or higher). Use the same system variable specified previously and reboot to enable priority paging on these systems.

Back to Top

The Solaris 8 Cyclical Page Cache

Priority paging is not available in the Solaris 8 operating environment, for good reason. Solaris 8 introduces a new file system caching architecture, the cyclical page cache, that eliminates most of the problems with virtual memory resulting from file system cache activity.

The new file system caching architecture in Solaris 8 introduces a new free list, the file system free list. The file system free list is dedicated to caching file system data only -- other objects in virtual memory such as applications, uninitialized application data, the kernel, shared libraries, and so on are managed on a separate free list.

The result is a much cleaner architecture where the file system cache no longer competes for virtual memory resources, since it effectively pages against itself. Heavy file system activity does not force applications out of primary memory -- it now throws only other file system pages out.

Because of these changes, the tools used to report virtual memory statistics report significantly different values that you should take into account when monitoring memory usage on your system.

The most obvious changes are:

• Higher Page Reclaims
  This is considered normal during heavy file system activity.

• Higher Free Memory Values
  The amount of free memory will be higher, since the free memory count now includes a large component of the file system cache.

• Zero Scan Rates
  Scan rates will be almost zero, unless there is a shortage of system wide available memory. Scanning is no longer used to replace the free list during normal file system activity.

The pre-Solaris 8 problem of not being able to differentiate between a true memory shortage or simply heavy file system activity has been eliminated. The scan rate (sr) column in the vmstat command is now a true indication of a memory bottleneck, and any non-zero values appearing in the sr column mean you don't have enough primary memory to keep up with the demand on the system.

Also, the vmstat command has been updated to report virtual memory usage statistics based on the cyclical page cache. Use the vmstat -p command to report paging activity details for applications (executable), data (anonymous), and file system activity, as shown in the following example:

% vmstat -p 3
     memory           page          executable      anonymous      filesystem 
   swap  free  re  mf  fr  de  sr  epi  epo  epf  api  apo  apf  fpi  fpo  fpf
 473160 40080   1   2   1   0   0    0    0    0    0    0    0   11    0    0
 416920 45168   0   1   0   0   0    0    0    0    0    0    0    0    0    0
 416920 45168   0   0   0   0   0    0    0    0    0    0    0    0    0    0

Tuning virtual memory on systems running Solaris 8 is unnecessary -- the virtual memory system has been optimized for the best performance straight out of the box, providing you with a compelling reason to upgrade your computing environment (especially servers) to Solaris 8.

Related Links

• A Practical Guide to Adopting the Solaris 8 Operating Environment
  A white paper for Solaris or UNIX system administrators who are curious about the new Solaris 8 features.
  
  
• Solaris Product Line: Solaris 8 Binaries
  This page has information about the Free Solaris Binary License Program.

August 2000

Back to Top