Calculating Bytes in a MIDP Input Stream

Print-friendly Version

Question

How do I calculate the number of bytes in a MIDP input stream over an HTTP connection?

I have to store these bytes in an array for storing the data in a RMS RecordStore. My current way is to allocate an array with 1024 bytes and fill it until no more data can be read. This means, I do something like this:

HttpConnection connection =
(HttpConnection)Connector.open("http://www.test.com");
InputStream inputStream = connection.openInputStream();
byte[] byteArray = new byte[1024];
int oneByte = inputStream.read();
int length = 0;
while (oneByte != -1) {
byteArray[length] = (byte)oneByte;
oneByte = inputStream.read();
length++;
}

That's not what I want. The biggest problem with this solution is that I am limited to the actual size of the array (1024 bytes in this case). Even when I can ensure that my byte stream never exceeds this limit, I still allocate 1024 bytes -- even when I only need 100 bytes.

The implementation of the available() method in InputStream always returns 0 (as the API documentation describes it). So that isn't a solution either.

Tip

Try the following:

HttpConnection connection = (HttpConnection)
Connector.open("http://www.test.com/servlet/readData");

InputStream inputStream = connection.openDataInputStream();

int length = connection.getLength();
byte[] byteArray;
if (length == -1) {
// Reading from an HTTP 1.0 server or a chunked HTTP 1.1
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int c ;
while (true) {
c = inputStream.read();
if (c == -1)
break;
baos.write(c);
}
byteArray = baos.getBytes();
} else {
// Reading a Content-Length labeled payload
byteArray = new byte[length];
inputStream.readFully(byteArray);
}

In HTTP you can test the Content-Length header. If the length is not provided, you may be dealing with an HTTP 1.0 server. A simple ByteArrayOuputStream can be used locally to accumulate the intermediate results.

If the request is delivered using HTTP 1.1 chunked payloads, then the bytes available() will reflect the current payload contents that can be read without blocking. This works because the chunked encoding is run-length encoded, for example, a length is sent and then the implementation sends that number of bytes to an internal buffer. Calls to read, simply move the bytes from the internal buffer into the user's buffer without actually talking to the socket connection.

However, if the web server does not set the Content-Length header and is not using chunked encoding, then the only way to calculate the number of bytes returned is to read bytes until the stream is closed. That's all you can do: if no length is set, then HTTP requires the server to send the bytes and to then close its end of the stream. So you have to read all the data, buffer it, and count the length once it's done reading.

There are two ways you can determine whether the web server does not set the Content-Length header. You can use getLength() and check for a value of -1; getLength() returns a -1 to indicate that the length is not known. Or you can get the "Content-Length" header and check for a null return value.

Note that a client application can force HTTP 1.0 behavior by including the "Connection: close" header. In this behavior, the server closes the socket when it is done with the request.

For applications that make multiple round trip requests to the same server, you can achieve the best performance by taking advantage of the HTTP 1.1 persistent connections. This is particularly true if the requests are made over real wireless networks where the bandwidth is slow and the latency is very high.

More Information

• HTTP 1.1 see:
  http://www.w3.org/Protocols/rfc2616/rfc2616.html
  
  
• Persistent connections, see:
  http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html#sec8.1
  
  
• Connection headers, see:
  http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.10
  
  
• Transfer-Encoding headers, see:
  http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.41

Acknowledgments

Many thanks to Eric Giguere for contributing to this answer.

		Note: If you have a question to which you need an answer, try the Wireless Forums. You can read through the existing topics, or with your free Wireless membership, you can post new messages or threads.