Sun Java Solaris Communities My SDN Account Join SDN
 
Article

Target Port Group Support (TPGS) in the Solaris OS

 
By Scott Tracy, June 2007  

Connecting your new storage device to a server is simple in the physical sense. Cables have become fairly indestructible for the user, but getting the device to show up properly and have the server understand its unique characteristics is virtually impossible without breaking into the operating system code and writing a module to tap into these features. Target Port Group Support (TPGS) is a method to enable a new storage device to be automatically detected and to allow methodologies to understand different port characteristics and failover behaviors. TPGS does not solve all problems, but it is a step in the right direction. Best of all, TPGS is integrated into the Solaris 10 OS.

Contents

Overview

Logical units can be connected to a service delivery subsystem through multiple target ports. The SCSI Architecture Model - 3 (SAM-3) specification describes the behavior of logical units that are accessed by application clients through multiple target ports.

In configurations in which there are multiple paths to a logical unit, each path might have different bandwidth and latency characteristics. Due to these different characteristics, the target ports might need to indicate which path is most efficient. Also, if a failure occurs on a target port, the SCSI target device might change its internal configuration, causing a path to go offline.

TPGS provides a method for determining the access characteristics of a path to a logical unit through a target port.

The access characteristics of a path are defined by the target port's asymmetric access state, which is returned by the SCSI command REPORT TARGET PORT GROUPS (RTPG). Access states are set through the command SET TARGET PORT GROUPS (STPG) or by the target device.

Adding TPGS into an array also provides the ability to perform these functions:

  • Automatic discovery
  • Standard handling

For further information, see the latest draft of the specification for TPGS (section 5.8 of the SCSI Primary Commands - 3).

Asymmetric Logical Unit Access

Asymmetric logical unit access occurs when the access characteristics of one port differ from those of another port. SCSI target devices with target ports implemented in separate physical units might need to designate differing levels of access for the target ports associated with each logical unit. Commands and task management functions can be routed to a logical unit through any target port. However, performance might not be optimal, and the allowed command set might be less complete than when the same commands and task management functions are routed through a different target port.

When a failure on the path to one target port is detected, the SCSI target device might perform automatic internal reconfiguration to make a logical unit accessible from a different set of target ports. Or the target device might be instructed by the application client to make a logical unit accessible from a different set of target ports.

A target port characteristic named target port asymmetric access state defines the properties of a target port and the allowable command set for a logical unit when commands and task management functions are routed through the target port.

A target port group (TPG) is a set of target ports that are in the same target port asymmetric access state at all times. Therefore, a TPG asymmetric access state is defined as a target port asymmetric access state that is common to a set of target ports in a TPG. The grouping of target ports is vendor specific.

A logical unit can have commands and task management functions routed through multiple TPGs. Logical units support asymmetric logical unit access if different TPGs can be in different asymmetric access states.

An example of asymmetric logical unit access is a SCSI controller device with two separated controllers for which all target ports on one controller are in the same asymmetric access state with respect to a logical unit and are members of the same TPG. Target ports on the other controller are members of another TPG. The behavior of each TPG can be different with respect to a logical unit, but all members of a single TPG are always in the same asymmetric access state with respect to a logical unit.

Explicit and Implicit Asymmetric Logical Unit Access

Asymmetric access can be managed explicitly by an application client using the RTPG and STPG commands. Alternatively, asymmetric access can be managed implicitly by the SCSI target device.

Logical units can attempt to maintain full performance across the busiest TPGs that exhibit the most reliable performance, allowing other TPGs to select a lower performance asymmetric access state.

If both explicit and implicit asymmetric access management methods are implemented, the precedence of one over the other is vendor specific.

Discovery of Asymmetric Logical Unit Access Behavior

SCSI logical units with asymmetric access can be identified using the standard INQUIRY command. The value in the TPGS field indicates whether the logical unit supports asymmetric access, and if so, whether implicit or explicit management is supported.

The INQUIRY command returns a value as defined in the following table.

Value
Meaning
00b
No support or vendor-specific support
01b
Only implicit asymmetric logical unit access is supported
10b
Only explicit asymmetric logical unit access is supported
11b
Both explicit and implicit asymmetric access are supported
 
Target Port Asymmetric Access States

For all SCSI target devices that report asymmetric access support in the INQUIRY data, all target ports in a TPG are in the same target port asymmetric access state with respect to the ability to route information to a logical unit.

The following sections describe the target port asymmetric access states.

Active/Optimized State

When commands and task management functions are routed through a target port in the active/optimized asymmetric access state, the device server responds to commands as specified in the appropriate command standards. All target ports within a TPG should be capable of immediately accessing the logical unit.

The SCSI target device participates in all task management functions as defined in SAM-3 and as modified by the applicable SCSI transport protocol standards.

Active/Non-optimized State

When commands and task management functions are routed through a target port in the active/non-optimized asymmetric access state, the device server responds to commands as specified in the appropriate command standards.

The processing of some task management functions and commands, specifically those involving data transfer or caching, might operate with lower performance than if the target port were in the active/optimized state.

The SCSI target device participates in all task management functions as defined in SAM-3 and as modified by the applicable SCSI transport protocol standards.

Standby State

When accessed through a target port in the standby asymmetric access state, the device server supports the following commands, which are supported while in the active/optimized state:

  • INQUIRY
  • LOG SELECT
  • LOG SENSE
  • MODE SELECT
  • MODE SENSE
  • PERSISTENT RESERVE IN
  • PERSISTENT RESERVE OUT
  • READ BUFFER (echo buffer modes)
  • RECEIVE DIAGNOSTIC RESULTS
  • REQUEST SENSE
  • REPORT LUNS (for LUN 0)
  • RTPG
  • SEND DIAGNOSTIC
  • STPG
  • WRITE BUFFER (echo buffer modes)

The device server might also support other commands.

For unsupported commands, the device server terminates the command with a CHECK CONDITION status, the sense key set to NOT READY, and the additional sense code set to LOGICAL UNIT NOT ACCESSIBLE, TARGET PORT IN STANDBY STATE.

The SCSI target device participates in all task management functions as defined in SAM-3 and as modified by the applicable SCSI transport protocol standards.

Unavailable State

When accessed through a target port in the unavailable asymmetric access state, the device server accepts a limited set of commands. The unavailable state is intended for situations in which the target port accessibility to a logical unit might be restricted because of SCSI target device limitations, such as hardware errors. Therefore, it might not be possible to transition from the unavailable state to the active/optimized, active/non-optimized, or standby state. The unavailable state is also intended to minimize disruptions during the downloading microcode mode of the WRITE BUFFER command.

While in the unavailable state, the device server supports the following commands, which are supported while in the active/optimized state:

  • INQUIRY (the peripheral qualifier is set to 001b)
  • READ BUFFER (echo buffer modes)
  • REPORT LUNS (for LUN 0)
  • REQUEST SENSE
  • RTPG
  • STPG
  • WRITE BUFFER (echo buffer modes)
  • WRITE BUFFER (download microcode mode)

The device server might also support other commands.

For unsupported commands, the device server terminates the command with a CHECK CONDITION status, the sense key set to NOT READY, and the additional sense code set to LOGICAL UNIT NOT ACCESSIBLE, TARGET PORT IN UNAVAILABLE STATE.

The SCSI target device is not required to participate in all task management functions.

Transitions Between Target Port Asymmetric Access States

The movement from one asymmetric access state to another is called a transition. During a transition, the device server responds to commands using one of the following two methods:

  • If the requested logical unit is inaccessible, the transition is performed as a single indivisible event and the device server responds by returning either a BUSY or CHECK CONDITION status, with the sense key set to NOT READY, and the additional sense code set to LOGICAL UNIT NOT ACCESSIBLE, ASYMMETRIC ACCESS STATE TRANSITION.
  • If the requested logical unit is accessible, the device server supports the following commands, which are supported while in the active/optimized state:
    • INQUIRY
    • READ BUFFER (echo buffer modes)
    • REPORT LUNS (for LUN 0)
    • REQUEST SENSE
    • RTPG
    • WRITE BUFFER (echo buffer modes)

The device server might support other commands when the commands are routed though a target port that is transitioning between asymmetric access states.

For unsupported commands during a transition, the device server terminates the command with a CHECK CONDITION status, the sense key set to NOT READY, and the additional sense code set to LOGICAL UNIT NOT ACCESSIBLE, ASYMMETRIC ACCESS STATE TRANSITION.

The SCSI target device is not required to participate in all task management functions.

If a transition fails, one of the following occurs:

  • If the transition was explicit to a supported asymmetric access state, the command terminates with a CHECK CONDITION status, the sense key set to HARDWARE ERROR, and the additional sense code set to SET TARGET PORT GROUPS COMMAND FAILED. The TPG that encountered the error completes a transition to the unavailable state.
  • If an asymmetric access state change occurred as a result of the failed transition, the device server establishes a unit attention condition for the initiator port associated with every I_T nexus other than the I_T nexus on which the STPG command was received, with the additional sense code set to ASYMMETRIC ACCESS STATE CHANGED.
  • If the transition was implicit, the device server establishes a unit attention condition for the initiator port associated with every I_T nexus with the additional sense code set to IMPLICIT ASYMMETRIC ACCESS STATE TRANSITION FAILED.

An implicit CLEAR TASK SET task management function can be performed following a transition failure.

After a transition is completed, the new asymmetric access state can apply to some or all tasks entered into the task set before the completion of the transition. The new asymmetric access state applies to all tasks received by the device server after completion of a transition.

After an implicit asymmetric access state change, a device server establishes a unit attention condition for the initiator port associated with every I_T nexus with the additional sense code set to ASYMMETRIC ACCESS STATE CHANGED.

After an explicit asymmetric access state change, a device server establishes a unit attention condition with the additional sense code set to ASYMMETRIC ACCESS STATE CHANGED for the initiator port associated with every I_T nexus other than the I_T nexus on which the SET TARGET GROUPS command was received.

Preference Indicator

A device server might indicate that one or more TPGs is preferred for accessing a logical unit by setting the PREF bit to 1 in the TPG descriptor. The preference indication is independent of the asymmetric access state.

An application client can use the PREF bit value in the TPG descriptor to influence the path selected to a logical unit. (For example, a TPG in the standby state with the PREF bit set to 1 might be chosen over a TPG in the active/optimized state with the PREF bit set to 0.)

The value of the PREF bit for a TPG can change whenever an asymmetric access state changes.

Implicit Asymmetric Logical Unit Access Management

SCSI target devices with implicit asymmetric access management are capable of setting the asymmetric access state of each TPG using mechanisms other than the STPG command.

All logical units that report support for implicit asymmetric access by displaying 01b or 11b in the TPGS field in the standard INQUIRY data do the following:

  • Implement the INQUIRY command Device Identification Vital Product Data (VPD) page identifier types 4h and 5h
  • Support the RTPG command

Implicit asymmetric access state changes can be disabled with the IALUAE bit in the Control Extension mode page.

Explicit Asymmetric Logical Unit Access Management

All logical units that report support for explicit asymmetric access by displaying 10b or 11b in the TPGS field in the standard INQUIRY data do the following:

  • Implement the INQUIRY command Device Identification VPD page identifier types 4h and 5h
  • Support the RTPG command
  • Support the STPG command
Behavior After Power On, Hard Reset, Logical Unit Reset, and I_T Nexus Loss

For all SCSI target devices reporting support for only explicit asymmetric access by displaying 10b in the TPGS field in the standard INQUIRY data, the target port preserves the asymmetric access state during any power cycle, hard reset, logical unit reset, or I_T nexus loss.

Symmetric Logical Unit Access

A device server that provides symmetrical access to a logical unit can use a subset of the asymmetrical access features to indicate this ability to an application client, providing an application client a common set of commands to determine how to manage target port access to a logical unit.

Symmetrical access is represented as follows:

  • Implicit asymmetric access support is indicated in the TPGS field in the standard INQUIRY data.
  • The RTPG command is supported.
  • The RTPG parameter data indicates that the same state, the active/optimized state, is in effect for all TPGs.
Glossary

Term
Meaning
Explicit asymmetric logical unit access
Asymmetric logical unit access managed explicitly by an application client using the RTPG and STPG commands
Implicit asymmetric logical unit access
Asymmetric logical unit access managed implicitly by the SCSI target device
Logical unit
A device at one end of a communications session
RTPG
REPORT TARGET PORT GROUPS
STPG
SET TARGET PORT GROUPS
Target port
The location where data is sent or received on a device
Target port asymmetric access state
The properties of a target port and the allowable command set for a logical unit when commands and task management functions are routed through the port
Target port group
A set of target ports that are in the same target port asymmetric access state at all times
TPGS
Target port group support
 
Rate and Review
Tell us what you think of the content of this page.
Excellent   Good   Fair   Poor  
Comments:
Your email address (no reply is possible without an address):
Sun Privacy Policy

Note: We are not able to respond to all submitted comments.