DLight Tutorial

DLight offers a variety of instrumentation that takes advantage of the Dynamic Tracing (DTrace) debugging and performance analysis functionality in the Solaris Operating System.

• System Requirements
• Required User Privileges
• DLight Basic Workflow
• Ending and Examining an Experiment
• Manipulating the Display
• Creating an Instrument (dtracelet)

DLight runs on the Solaris 10 and OpenSolaris operating systems.

To use DLight with Java applications, the system on which you are running the tool must have a DTrace enabled Java SE Runtime Environment (JRE) 6 or later.

DLight also runs on Linux operating systems, but only the Clock Profiler and JAVA Ticker instruments are available in these environments.

To run DLight tool, your user account must have the dtrace_user, dtrace_proc, and dtrace_kernel privileges. Not all instruments require all privileges. If your user account lacks a privilege required by an instrument, that instrument is unavailable.

To check your user privileges, type /bin/ppriv -v $$ | grep dtrace at a shell prompt. If your account doesn't have the required privileges, consult your system administrator. If you have root access to your system, you can grant your user account the required privileges.

To permanently grant required DLight privileges to a user account:

1. Make sure the user account whose privileges you want to modify is logged out of the system.
2. Become superuser (root).
3. Type the following:

   $ usermod -K defaultpriv=basic, dtrace_user,dtrace_proc user-name

    
   

To temporarily grant required DLight privileges to a user account:

1. Type the following to determint the process ID of the shell's process.

   $ echo $$

    
   
2. Become superuser (root).
3. In another shell, type the following:

   $ ppriv -s I+dtrace_user,dtrace_proc,dtrace_kernel process-ID

    
   

   All commands typed in the shell with the process-id inherit the required privileges.

To start DLight tool, click the DLight button on the toolbar. The tool opens three windows in the IDE:

• The main DLight window on the upper right
• The Instruments window on the upper left
• The Event Details window on the lower left

The basic DLight workflow consists of:

• Choosing a target application for DLight instruments
• Choosing the instruments that the tool monitors as the target application runs
• Analyzing the data that the tool returns
• Using the returned data to refine the experiment

Repeat this cycle until the behavior of the application under analysis is clear.

Running the DLight Tool on a Sun Studio Project

To run the tool on a Sun Studio project:

1. Make sure that the project on which you want to run the tool is your main project (the main project is in bold type the Projects window). To make a project the main project, right-click the project node and choose Set as Main Project.
2. From the drop-down list on the DLight button on the Sun Studio toolbar, select DLight Main Project.
3. Choose the instruments you want DLight to use.
4. Click the Run button on the DLight toolbar to run the project and monitor the instruments that are in the main DLight window.

 

Running DLight on a Remote System

When you develop your code on a local system and run the application on a remote system, you can run DLight instrumentation on the remote system by configuring the project for remote development.

To add a remote host:

1. Choose Tools > Options.
2. Click C/C++ in the top pane of the Options window.
3. Click the Build Tools tab.

 

4. Click the Edit button to open the Development Hosts Manager dialog box.
5. To add a remote host, click Add. In the Add New Server dialog box, type the name of the host in the Development Server field. Change the default Login name if needed. Type your password in the Password field and click OK.

 

6. Click OK in the Development Hosts Manager dialog box.

To configure your project for remote development:

1. Right-click the project node in the Projects window and choose Properties.
2. In the Project Properties dialog box, select the Build node in the Categories list.
3. Select the remote host from the Development Host drop-down list.

 

4. Click OK.

Running DLight on a C, C++, or Fortran Executable

To run the tool on a C, C++, or Fortran executable:

1. From the drop-down list on the DLight button on the Sun Studio toolbar, select DLight Executable to browse for an executable.
2. In the Select Executable Target dialog box, type the path to the executable in the Executable field, or click the Browse button to browse for an executable.
3. Type any arguments to pass to the executable in the Arguments field.
4. Type the path to a working directory in the Working Directory field or click on the Browse button to browse for a directory.

 

5. Click OK.
6. Choose the instruments you want DLight to use.
7. Click the Run button in the DLight toolbar to run the application and monitor the instruments that are in the main DLight window.

Running the DLight Tool on a Java Executable

To run the tool on a Java executable:

1. From the drop-down list on the DLight button on the Sun Studio toolbar, choose DLight Java Executable to browse for a JAR file.
2. In the Select Java Application dialog box, make sure the JVM Path field contains the correct path to your Java Virtual Machine (JVM) installation, or click the Browse button to browse for a directory.
3. Type any options you want to pass to the JVM in the JVM Options field.
4. Type the path to the JAR file in the JAR File field, or click the Browse button to browse for a JAR file.
5. Type the path to a working directory in the Working Directory field or click on the Browse button to browse for a directory.
6. Select the main class from the Main Class drop-down list or type in a main class.

 

7. Click OK.
8. Choose the instruments you want DLight to use.
9. Click the Run button on the DLight toolbar to run the Java executable that you specified in the JAR File field and monitor the instruments that are in the main DLight window.

Note: To use the Java Development Kit (JDK) DTrace probes, pass -XX:+ExtendedDTraceProbes as an option to the JVM as an option in the JVM Options field.

Note: The terms "Java Virtual Machine" and "JVM" mean a Virtual Machine for the Java platform.

Choosing Instruments

DLight provides several instruments that you can use to analyze your application's behavior. You can select an instrument from the DLight Instruments window in the top left of the IDE.

To open the Instruments window, press Alt-L or click the Show DLight Instruments Panel button in the top left corner of the DLight tool's main window.

You can drag and drop individual instruments to the vertical blue bar at the left edge of the main DLight Tool window. You can also double-click on an instrument to select it.

When you run the target application, the data that the individual instruments return is displayed in real time. You can end the experiment by clicking the Stop button on the DLight toolbar or waiting for the application to finish.

To examine and interpret the application's behavior, navigate the individual timelines, examining events and finding correlations between events that happen simultaneously. Click on a event in the display to see detailed information about that event in the DLight Event Details window.

The information returned by the instruments is displayed in the main DLight window. You can manipulate this display in various ways:

• To zoom in on the data and fill the display window, click on the Fit to Screen button or press Alt-F.
• To zoom in on a specific region of the timeline, click and drag in the Timeline window to outline the desired region.
• Click the up and down buttons on the DLight toolbar to change the display to the timelines reported by different instruments. You can also press Alt-U (up) and Alt-W (down).
• To display the D-script code that is associated with a specific instrument, right-click the instrument icon and select D-Script from the drop-down list.

• To reset the timeline, click Reset Collected Data on the DLight toolbar or press Alt-R.

Note: You cannot make changes to the instruments in the main DLight window while the window is displaying existing data. You must clear the collected data before changing the instrumentation.

• To move an instrument's timeline up or down in the list, drag and drop the instrument's icon to a different location on the Instrument list.
• To change the color masks that are associated with the events reported on an instrument's timeline, click Show Color Masks Dialog on the DLight toolbar or press Alt-C.
• To display the source code that is associated with a particular monitored event, double-click any element of the stack in the instrument's timeline. For C and C++ source code, you must compile the application with the -g option.

If you have some knowledge of DTrace, you can create your own instrument (dtracelet) to observe your application or even your system. A dtracelet is an xml file that is used to collect the information that determines what data is displayed and how it is displayed. It includes the name, icon, and description to be displayed for the instrument; the D script source, the data description, and the data visualization attributes.

For a template for a dtracelet xml file, see Instrument (dtracelet) Template.

The data visualization section of the dtracelet is responsible for the data visualization in the main DLight window.

The <graph> element describes the bars that are displayed in the timeline. If a key attribute is specified, the display will include as many bars as there are unique values for the key field. For example, the THRID graph key is specified in the Clock Profiler dtracelet, so the Clock Profiler timeline includes as many bars as your application has threads.

For examples of <vis> settings for various types of displays, see Visualization Examples.

When you have created a new dtracelet, save it in /sun_studio_installation_directory/prod/lib/dlight/dtracelets. Click the Refresh button in the DLight Instruments window, and your new instrument will be included in the list of instruments.

For an example of a dtracelet file, see Instrument Example.

Instrument (dtracelet) Template

The following is a template for a dtracelet. The inline comments explain the sections
of the xml file. For more information on the data visualization <vis> section, see Visualization Examples.

<!--
   The name attribute of a dtracelet will verbally identify it in
   the DLight Instruments window.
-->
<dtracelet name="name">

<icon format="pathname">

<!-- The absolute pathname of the file containing the icon image.
     This icon will appear in the Library window and Instrument panel.
-->
</icon>
<description>
<!-- This description will appear in the tooltip of DLight Instruments window.
-->
</description>
<validation>
<!-- Use this section to place here some validation script.
If you use dtrace you can use the built in script as following
#  #<![CDATA[
#  #!/bin/sh
#  . ./common_validation.sh
#  validate_dtrace dtrace_kernel dtrace_proc ..<dtrace privileges>

#  #]]>


-->
</validation>
<script>
<!-- The actual d-script source is here. -->
</script>
<data>

<profpckt kind="kind" name="name">

    <!-- The order and types of the following fields correspond to actual 
    d-script output. Field names are unique in the scope of this packet.
    -->
    <field name="name" uname="user_name" type="type"/>
    <field name="name" uname="user_name" type="type"/>

</profpckt>
</data>
<vis type="Timeline">
  <timestamp value="name_of_field_which_represents_time"/>
  <graph key="key is optional parameter, if presented the field name pointed 
  here will be used to multiply timeline bars correspondent to this tool">
    <color value="field name which will be used as bar color"/>

    <height value="field name which will be used as bar height"/>
  </graph>
</vis>
</dtracelet> 

Visualization Examples

Bars

The Read/Write monitor uses this kind of visualization.

 <vis type="Timeline">
  <timestamp value="TSTAMP"/>
    <graph key="THRID">
      <color value="OPERTN"/>

      <height value="NBYTES"/>
    </graph>

  </vis>

Stacked Bars

The Clock Profiler uses this kind of visualization.

 <vis type="Timeline">
  <timestamp value="TSTAMP"/>
    <graph key="THRID">

      <color value="USTACK"/>
    </graph>
  </vis>

Events With Duration

The MySQL monitor uses this kind of visualization

<vis type="Timeline">

  <timestamp value="HRTIME"/>
  <graph>
     <color value="SQLCOMMAND"/>

     <duration value="DURATION"/>
  </graph>
</vis>

Continuous Timeline

The Microstate Accounting and Heap Monitors use this kind of visualization.

<vis type="TimelineContinuous">

    <timestamp value="TSTAMP" />
    <graph uname="Heap Monitor" mode="OVERLAY" scale="10000000">
      <plot uname="Memory in use"     value="TUSER" color="red"/>

      <plot uname="Data segment size" value="TSYS"  color="green"/>
    </graph>
</vis>

You can use the <plot> element (a child of the <graph> element), which is similar to a series. Using the mode attribute of the <graph> element, you can describe how data should be displayed.

In the Continuous Timeline example, STACKED mode would mean that if you have 100 as a value for graph "Memory in use" and 100 as a value for graph "Data segment size", two bars would be displayed one on top of the other for a total height of 200. If you specified OVERLAY mode, as the example shows, you would see just one of the graphs as the second one would be hidden. and the height would be 100.

Setting the scale attribute of the <plot> element to auto would give you the autoscale feature.

When you use the <plot> element, set the color attribute using standard names of colors.

Instrument (dtracelet) Example

The following is the dtracelet xml file for the Read/Write Monitor.

<dtracelet name="Read/Write Monitor" mode="pid">
  <icon name="fops.png"/>

  <description>
    Monitors application's IO done via calls to read(2) and write(2)
  </description>
  <validation>
    #<![CDATA[
    #!/bin/sh
    . ./common_validation.sh
    validate_dtrace dtrace_kernel dtrace_proc || exit 1
    #]]>
  </validation>

  <script>
    <![CDATA[
    #!/usr/sbin/dtrace -s
    #pragma D option quiet
    #pragma D option destructive
    
    self uint32_t stkIdx;
    
    
    <-- Copy the prob below if you want to collect stacks-->
    collector$1:::ustack
    {
        printf("0 %d %d %u\n", tid, self->stkIdx++, arg0<<32|(arg1&0xffffffff));
    <--printf in stdout is used to save data, the first number 
    is "kind" used in data/profpckt section-->
    }
    
    syscall::read:entry
    /pid == $1/
    {
        self->fd = arg0;
    }
    
    syscall::read:return
    /pid == $1/
    {
        fname = (self->fd == 0 ) ? "<stdin>" : fds[self->fd].fi_pathname;
        printf("1 %d %d %d %d \"%s\" %d %d\n",
               cpu,
               tid,
               timestamp,
               0,
               fname,
               arg1,
               self->stkIdx);
    <--Use self->stkIdx variable to save stack index-->

        raise(29);
    <--Raising this signal allows you to invoke SunStudio 
    collect ustack probe-->
    }
    /data
    /pid == $1/
    {
        self->fd = arg0;
    }
    
    syscall::write:return
    /pid == $1/
    {
        fname = (self->fd == 1 ) ? "<stdout>" : (self->fd == 2 ) ? "<stderr>" : fds[self->fd].fi_pathname;
        printf("1 %d %d %d %d \"%s\" %d %d\n",
               cpu,
               tid,
               timestamp,
               1,
               fname,
               arg1,
               self->stkIdx);
    <--Use self->stkIdx variable to save stack index-->

        raise(29);
    <--Raising this signal allows you to invoke SunStudio collect 
    ustack probe-->
    }
    ]]>
  </script>
  <data>
    <-- Copy this section if you want to collect stacks-->

    <profpckt kind="0" name="STKIDXMAP">

    <--Kind is the same number which is used as first printed 
    value in printf inside your script-->
        <field name="THRID" type="UINT32"/>
        <field name="STKIDX" type="UINT32"/>
        <field name="STKUID" type="UINT64"/>
    </profpckt>

    <profpckt kind="1" name="RW" uname="Read/Write Monitor">

    <--all data described in this section will be shown in 
    Event Details Window (unless you will use noshow="1" attribute-->
        <field name="CPUID" uname="CPU id" type="UINT32"/>
        <field name="THRID" uname="Thread number" type="UINT32"/>
        <field name="TSTAMP" uname="Time stamp" type="UINT64"/>
        <field name="OPERTN" uname="Operation" type="INT32">

            <state uname="Read" value="0"/>

            <state uname="Write" value="1"/>
        </field>
        <field name="FNAME" uname="File name" type="STRING"/>
        <field name="NBYTES" uname="Number of bytes" type="INT32" />
        <field name="STKIDX" type="UINT32" noshow="1"/>

    <--add this field to use stack saved-->

    </profpckt>
  </data>
Visualization part
  <vis type="Timeline">
  <timestamp value="TSTAMP"/>
    <graph key="THRID">

      <color value="OPERTN"/>

      <height value="NBYTES"/>
    </graph>
  </vis>
</dtracelet>