Databases and MIDP, Part 4: Filtering and Traversal Strategies

To read or write a record you need to know its record ID. In Part 1 you learned that a record ID is an integer that uniquely identifies a record within the record store - and that it is not an index into the record store. This difference has some vital implications.

If a store of N records were indexed, each would have an index in either the range 0 to N-1 or the range 1 to N, depending on whether the range started with 0 or 1. Each time a record was removed or inserted, indexes of records later in the store would change accordingly. The range would shrink or grow, but remain continuous.

Unlike an index, a record ID does not change, no matter how many other records are inserted or removed before it in the record store. The first record added to a record store is given a record ID of 1, the next a record ID of 2, and so on. If you delete a record, its record ID is invalidated and any attempt to access that record throws InvalidRecordIDException. Valid record IDs don't remain continuous.

Because they uniquely identify records, you can use record IDs to link two or more records together, by storing the ID of one record as a data value in another. You can also use record IDs to synchronize data with external applications, as you'll see in Part 6.

The main disadvantage of record IDs is that they complicate record store traversal; you can't just iterate through a set of indexes as you can through an array. You must use one of two traversal techniques: brute force or enumeration.

With the brute-force approach, you simply fetch records one by one, starting with the first record, skipping invalid records, and continuing until you've fetched all records:

... RecordStore rs = ... // an open record store int lastID = rs.getNextRecordID(); int numRecords = rs.getNumRecords(); int count = 0; for( int id = 1; id < lastID && count < numRecords; ++id ){ try { byte[] data = rs.getRecord( id ); ... // process the data ++count; } catch( InvalidRecordIDException e ){ // just ignore and move to the next record } catch( RecordStoreException e ){ // a more general error that should be handled // somehow break; } } ...

This code calls getNextRecordID() to discover the ID of the next record to be added to the store, and uses it as the upper bound for the possible record identifiers. The code increments a counter each time it reads a valid record so traversal can stop as soon as all records have been seen. Note that the record store is not locked during the traversal - you'll need to use a synchronized block to keep other threads from changing the record store if that's important to you.

The brute-force method is simple to understand and works well if there are few missing records, but the preferred approach is to use an enumeration.

Instead of checking each record ID to see which ones are valid, which is essentially what the brute-force approach does, you can ask the RMS to return you an enumeration of valid record IDs, using the RecordEnumeration interface. This interface does not extend the standard java.util.Enumeration interface, but it works in a similar fashion. In fact, it's actually a more capable interface: You can traverse a record enumeration backward as well as forward, and optionally use it to track changes to the record store as they occur.

You obtain an enumeration by calling the enumerateRecords() method of the record store you wish to traverse, as in this example:

... RecordStore rs = ... // an open record store RecordEnumeration enum = rs.enumerateRecords( null, null, false ); ... // use the enumeration here enum.destroy(); // always clean it up! ...

For simplicity, exception handling has been omitted from this code snippet, but be aware that enumerateRecords() can throw a RecordStoreNotOpenException.

The first two parameters to enumerateRecords() control how the records are to be filtered and sorted - we'll discuss them shortly. The third parameter controls whether the enumeration tracks changes to the record store. Tracking changes requires extra overhead and is not needed in most cases, so we'll set it to false in all our examples.

When you're done with an enumeration, you must invoke its destroy() method to free any resources the system has allocated to support it - remember that there's no object finalization in CLDC. Undestroyed enumerations will cause your application to leak memory.

Use hasNextElement() and nextRecordId() to move forward through the enumeration:

... RecordStore rs = ... RecordEnumeration enum = ... try { while( enum.hasNextElement() ){ int id = enum.nextRecordId(); byte[] data = rs.getRecord( id ); ... // do something here } } catch( RecordStoreException e ){ // handle the error here } ...

You can also move backward, using hasPreviousElement() and previousRecordId(). Note that both nextRecordId() and previousRecordId() throw InvalidRecordIDException if a record is deleted from a record store while the enumeration is active, if it's not tracking changes to the record store.

The order in which an unsorted enumeration returns its records is implementation-specific, so don't make any assumptions based on that order.

For convenience, you can use nextRecord() or previousRecord() to obtain the data of the next or previous record instead of its record ID:

... try { while( enum.hasNextElement() ){ byte[] data = enum.nextRecord(); ... // do something here } } catch( RecordStoreException e ){ // handle the error here } ...

You will not, however, know the ID of the record in question or be able to read the data directly into a byte array that you've allocated. If you're filtering or sorting the enumeration, however, the data may be cached in memory, so obtaining it from the enumeration instead of the underlying record store may be more efficient.

A call to numRecords() will give you the number of record IDs in the enumeration, but this call may cause the enumeration to materialize all at once instead of incrementally, and this operation might use up a lot of memory or cause a noticeable pause in processing.

Use reset() to reset an enumeration to its initial state, and rebuild() to force it to update itself based on the current state of the record store.

Use isKeptUpdated() to check whether an enumeration is tracking changes to the record store, and keepUpdated() to change its tracking status.

Interested in only a subset of the data in a record store? You can have an enumeration skip unwanted records by using a filter, an object that implements the RecordFilter interface. The filter's matches() method should indicate whether a record is to be included in the enumeration:

public class MyFilter implements RecordFilter { public boolean matches( byte[] recordData ){ ... // matching code here } }

Ideally, the information you need to do the filtering is at the beginning of the array - you want to determine a match as quickly as possible. To use a filter, pass it as the first parameter to enumerateRecords():

... enum = rs.enumerateRecords( new MyFilter(), null, false ); ...

The record ID is not passed to the filter, only the byte array containing the record data. If you absolutely must know which record ID is being matched, you'll need to store the ID in the record itself or use some other way to identify the record from its data. In Part 3, for example, we used the first record in a record store to hold information about the fields in the remaining records - obviously you'd want to filter this record out of any enumeration.

To guarantee that records are returned in a consistent, predictable order, you must sort the enumeration. You supply the enumeration with a comparator, an object that implements the RecordComparator interface. The comparator's compare() method returns whether a record precedes, is equal to, or follows another record:

public class MyComparator implements RecordComparator { public int compare( byte[] r1, byte[] r2 ){ int salary1 = getSalary( r1 ); int salary2 = getSalary( r2 ); if( salary1 < salary2 ){ return PRECEDES; } else if( salary1 == salary2 ){ return EQUIVALENT; } else { return FOLLOWS; } } private int getSalary( byte[] data ){ ... // code to get salary info } }

Note the use of the constants PRECEDES, EQUIVALENT, and FOLLOWS. These are defined by the RecordComparator interface. To use a comparator, pass it as the second argument to enumerateRecords():