Debugging MIDP HTTP Requests

Print-friendly Version

Download the J2ME source.
Download the servlet source.

Applications written using the Mobile Information Device Profile (MIDP) can communicate with a web server, an application server, or any other service that understands the HTTP 1.1 protocol. HTTP is in fact the only protocol all MIDP-compliant devices must support, even if the underlying network does not support standard socket-based connections. To write portable network-enabled applications you'll need to understand HTTP connectivity.

Debugging the MIDP HTTP request-response cycle is one of the challenges that developers face, especially when working with devices on public wireless networks. While developers control the major aspects of an HTTP request, such as the URL and the request type, GET or POST, the underlying system and the network transport can modify requests and responses in unexpected ways. Communication problems may arise that are particularly hard to diagnose.

This article presents two tools you can use to debug MIDP HTTP requests. The first runs on the device as part of your application, the second runs in a servlet container like Tomcat or the Sun Java System Application Server.

These tools supplement those provided by your application development environment. NetBeans 4.0, for example, includes an HTTP monitor you can use to track requests made to a web server that supports the Servlet 2.3 specification.

Let's start with a quick recap of the HTTP support available in MIDP. For more detailed explanations, please refer to the article "MIDP Network Programming using HTTP and the Connection Framework".

MIDP's HTTP support is based on the Generic Connection Framework (GCF), defined by the Connected Limited Device Configuration (CLDC). When passed a URL that starts with http: - or https: in devices that also support HTTPS - Connector.open() returns an object that implements the HttpConnection interface. This object can be used to make a single HTTP request to the server at the given URL - a separate object is required for each HTTP request.

The methods of the HttpConnection interface are the only means by which an application can control HTTP communication. Many details of the HTTP request-response cycle are hidden from the application. For example, the device may share the underlying network connections for successive requests made to the same server, or it may use chunked encoding to stream data to the server. As long as HTTP 1.1 semantics are observed, the implementation is free to do what works best for the device and the network.

The HttpConnection object doesn't establish a connection to the server immediately. It passes through three states: setup, connected, and closed. In the setup state, your application can call the setRequestMethod() and setRequestProperty() methods to set request headers and the request type. The transition to the connected state occurs when you call one of the other methods. For POST requests, the connection is usually made when you obtain an output stream for writing data to the server. For GET requests, the connection is usually made when the application fetches a response code or opens an input stream to read the server's response. Once the application processes the response, it calls close() to free any network resources used by the object, which enters the closed state. The object is generally discarded at this point because it can't be used for further communication.

Although the specification makes no explicit recommendation, an application should always perform HTTP operations on a thread it creates. See "Understanding MIDP System Threads" for the reasons.

Download example.

The first tool is a wrapper for the HttpConnection interface, which acts as a proxy for the actual connection. HttpConnectionWrapper is fairly straightforward to build because it simply forwards most method calls to the original connection for processing. The wrapper logs calls and their results - including any exceptions thrown - as they happen.

Use the wrapper by constructing an instance of HttpConnectionWrapper immediately after obtaining an HttpConnection object from Connect.open(), as in this example:

...
HttpConnection conn = null;

try {
    conn = new HttpConnectionWrapper( (HttpConnection)
         Connector.open( "http://www.java.net" ) );
    ... // do something with the connection
}
catch( IOException e ){
    // handle the error
}
...

Because the HttpConnectionWrapper implements HttpConnection, the application can - and should - just treat it as the latter. Here's the kind of output you'll see:

[INFO] [HTTP 1] Wrapping HttpConnection http://www.java.net
[INFO] [HTTP 1] Entering setup state
[INFO] [HTTP 1] setRequestProperty( User-Agent, Profile/MIDP-1.0 Configuration/CLDC-1.0 )
[INFO] [HTTP 1] setRequestProperty( Connection, close )
[INFO] [HTTP 1] Entering connected state
[WARNING] [HTTP 1] getResponseCode throws exception javax.microedition.io.ConnectionNotFoundException: TCP open
[INFO] [HTTP 1] Entering closed state


For output, the wrapper uses a simple logging mechanism that's loosely based on the J2SE javax.logging package. By default the logger sends its output to System.out and to a record store. On a real device you'll want to disable the logging to System.out and rely on viewing what gets written to the record store.

The granularity of the logging is easy to adjust:

HttpConnectionWrapper.logger.setLevel( Level.FINE );

Notice the additional information you get at the FINE level:

[INFO] [HTTP 1] Wrapping HttpConnection http://www.java.net
[INFO] [HTTP 1] Entering setup state
[FINE] [HTTP 1] Invoking setRequestProperty
[INFO] [HTTP 1] setRequestProperty( User-Agent, Profile/MIDP-1.0 Configuration/CLDC-1.0 )
[FINE] [HTTP 1] Invoking setRequestProperty
[INFO] [HTTP 1] setRequestProperty( Connection, close )
[FINE] [HTTP 1] Invoking getResponseCode
[INFO] [HTTP 1] Entering connected state
[WARNING] [HTTP 1] getResponseCode throws exception javax.microedition.io.ConnectionNotFoundException: TCP open
[INFO] [HTTP 1] Entering closed state

The wrapper also tracks a connection's state transitions as defined by the MIDP specification. Some methods can be invoked only in certain states, so the wrapper throws exceptions whenever methods are invoked in the wrong context. Input and output streams are also tracked. You'll be warned if you close the connection without closing the streams. Technically, doing so is not an error, but problems may arise if you forget to close the streams when you're done with them.

When you're done debugging the MIDlet's HTTP requests, simply remove all references to HttpConnectionWrapper. If problems reappear, you can easily reinsert the wrapper to track them down.

Download example.

Logging what's happening on the device is a good first step, but it's not always enough to diagnose and solve a problem. Sometimes the problem doesn't lie in the way the application is making its HTTP requests, but rather in the way the device is formatting the requests or the way the server is processing them. You need diagnostic code at the receiving end as well as the sending end. Fortunately, servlet filters make server-side troubleshooting easy.

The Servlet 2.3 specification introduced filters as a way of intercepting HTTP requests before they were processed by other components of a web application, such as servlets or JavaServer Pages (JSPs). Responses can also be intercepted. The article "The Essentials of Filters" is a good introduction to filters. The article mentions "logging and auditing" as one example of what filters can be used for, and that's exactly what you're going to do: build a filter that logs details about the HTTP requests it receives. The filter will use the JavaMail APIs to mail the information instead of just appending it to a log file.

If you haven't used the JavaMail APIs before, don't be intimidated. They make sending mail very simple. The basic steps are described in the article "Sending Email From a Servlet." You'll need access to an SMTP server for sending the mail. If you don't have a server you can use for testing, download and install a copy of James - the open-source Apache Java Enterprise Mail Server, as I did to test my example code.

First, though, you have to write the filter. Here's the skeleton for one that does nothing but pass the request on to the next element in the filter chain, the set of components that can process an HTTP request:

package httpfilter;

import java.io.*;
import java.util.*;
import javax.servlet.*;
import javax.servlet.http.*;

/**
 * A basic filter implementation. Does nothing but forward
 * requests to the next filter or servlet in the chain.
 */

public class HttpLogger implements Filter {    
    public HttpLogger() {
    }
    
    public void init( FilterConfig config ) {
        _config = config;
    }
    
    public void doFilter( ServletRequest servletRequest,
                          ServletResponse servletResponse, 
                          FilterChain chain )
                  throws ServletException, IOException {
        chain.doFilter( servletRequest, servletResponse );
    }
    
    public void destroy() {
    }
    
    private FilterConfig _config;
}

As you can see, filters are really simple to define. Other than the initialization and deinitialization methods, init() and destroy(), the only method to implement is doFilter(), which in this case simply passes the request on. To activate the filter, supply a definition and a mapping for it in the web application's web.xml file:

    ...
    <filter>
        <filter-name>HttpLogger</filter-name>
             <filter-class>httpfilter.HttpLogger</filter-class>
    </filter>
    
    <filter-mapping>
        <filter-name>HttpLogger</filter-name>
        <url-pattern>/*</url-pattern>
    </filter-mapping>
    ...

For simplicity, the mapping uses the URL pattern /* to intercept all requests made to the web application, but you can easily change this to filter more selectively.

To send mail, you'll need to know the name of the SMTP server and the "From" and "To" addresses to use. Rather than hardcoding these values, use filter configuration properties, embedded in the <filter> tag , to define them:

    ...
    <filter>
        <filter-name>HttpLogger</filter-name>
        <filter-class>httpfilter.HttpLogger</filter-class>
        <init-param>
            <param-name>mail.smtp.host</param-name>
            <param-value>smtp.myisp.com</param-value>
        </init-param>
        <init-param>
            <param-name>from.address</param-name>
            <param-value>logger@mydomain.com</param-value>
        </init-param>
        <init-param>
            <param-name>to.address</param-name>
            <param-value>developer@mydomain.com</param-value>
        </init-param>
    </filter>
    ...

You'll need to change the dummy values here to actual values before running the filter.

Now you can flesh out the code for the filter itself. You'll see that doFilter() receives three parameters; a ServletRequest, a ServletResponse, and a FilterChain. The first two represent the request and the response, respectively, while the last represents the chain of web application components that are processing this request. The new code for doFilter() is quite simple:

// Does the actual filtering by wrapping the request
// and response and then sending mail.
    
public void doFilter( ServletRequest servletRequest, 
                      ServletResponse servletResponse, 
                      FilterChain chain )
              throws ServletException, IOException {
        
    int id = _lastID++;
    RequestLogger req = new RequestLogger( servletRequest );
    ResponseLogger res = new ResponseLogger( servletResponse);
    Throwable error = null;
        
    try {
        chain.doFilter( req, res );
    }
    catch( ServletException e ){
        error = e;
        throw e;
    }
    catch( IOException e ){
        error = e;
        throw e;
    }
    finally {
        report( id, req, res, error );
    }
}

Most of the work is done outside this method. The RequestLogger and ResponseLogger classes wrap the request and response to trap details about the request and response. These details and any exception that has occurred are passed along to report(), which generates a simple text-based report and sends it to the appropriate email address. See the source code for all the details.

The JavaMail APIs make emailing a plain-text message quite simple. You'll have to download both the JavaMail and JavaBeans Activation Framework APIs if you haven't already installed them. Then make sure that three particular JAR files are in the filter's classpath: mailapi.jar and smtp.jar from the JavaMail distribution, and activation.jar from the JavaBeans Activation Framework. A simple way to do so is to place them in the web application's WEB-INF/lib directory.

When the filter starts running, it initializes a few things, including the JavaMail session you need:

...
// Extracted from the filter startup

_session = Session.getDefaultInstance( _props, null );           
        
String faddr = getInitParameter( FROM_ADDRESS, 
                                 DEFAULT_FROM_ADDRESS );
String fname = getInitParameter( FROM_NAME, 
                                 DEFAULT_FROM_NAME );
        
_from = new InternetAddress( faddr, fname );
...

The filter defines sendMail() for sending the report:

// Send mail using basic JavaMail calls. Nothing fancy, just 
// send the report out as plain text.
    
private void sendMail( String to, String subject, 
                   String body ) throws MessagingException {
    Message msg = new MimeMessage( _session );
    msg.setFrom( _from );
    InternetAddress toAddr = new InternetAddress( to );
    msg.setRecipient( Message.RecipientType.TO, toAddr );
    msg.setSubject( subject );
    msg.setContent( body, "text/plain" );
    log( "Sending mail to " + to );
    Transport.send( msg );
}

As you can see, not complicated at all.

After installing the filter in a web application and making sure it's properly configured, deploy the web application to a servlet container like Tomcat. Using a web browser, browse to a page in the web application. If everything's working correctly, you'll receive an email message shortly thereafter. If not, check the servlet container's log - the filter does extensive logging for debugging purposes.

To make building and testing the examples easier, I've provided two NetBeans 4.0 projects with this article. To use them you'll need to install the J2ME Wireless Toolkit 2.2 and the NetBeans Mobility Pack if you haven't already done so. The first project, HttpWrapper, is a simple MIDP application that demonstrates the use of the HttpConnectionWrapper class. Unpack httpwrapper.zip and open the project with NetBeans. Now do the same with the second project, HttpFilter, a web application for testing the filter: Unpack httpfilter.zip and open it in NetBeans. Your NetBeans project window should look something like Figure 1:

Figure 1: The NetBeans Project Window

Navigate to the web.xml file in the HttpFilter project tree and insert appropriate host and address values for the filter initialization parameters. Then right-click on the HttpFilter node and choose the Run item. The IDE compiles the web application, then deploys it to the embedded Tomcat server that's embedded in NetBeans.

Now right-click on the HttpWrapper node and choose the Run item from its popup menu. Doing so compiles the MIDP application and deploys it to the J2ME Wireless Toolkit emulator. Once the emulator is up, launch HttpWrapperMIDlet and enter a URL to test. The MIDP application is very simple; it just takes the URL, connects to it, and attempts to read any data sent back. You should see logging output on the emulator's console.

The downloadable example includes all the files for a typical build, including sample .java source files and .jsp server pages, and a web.xml file for you to tailor to your needs. NetBeans generates most of the files for you, automatically.

These simple tools can help diagnose your HTTP connectivity problems. Be sure to take advantage of other tools you might have. For emulator-based testing, for example, the built-in NetBeans HTTP monitor provides more detailed information about the requests and responses than the filter currently provides. You can extend the filter to provide much of the same information - I'll leave that task as an exercise for you. Not all problems occur in an emulated environment, however: some show up only when you deploy the application to a real device and connect to a production web server. That's where tools like HttpWrapper and HttpFilter come in handy.

Eric Giguére is a software developer, and wrote the first book about Java 2 Micro Edition, as well as numerous articles on wireless and handheld programming. For more information about Eric, visit his web site. You can also contact him via email.