Monthly Archives: February 2012

Siri binary data, how to process

Apple's new personal assistant Siri was cracked by Applidium few months ago. The data format was explained by them in detail.

Just for a personal interest, I try to make a java version of siriproxy based on the information they provided. Turns out there are much more details into it.

How iphone send voice packages
First of all, Siri convert voice data in to the codex Speex data format. Which is a public available library and there are java/c libraries that deals with it. For people who are interested in collecting the data and do some dictation on their own. They can simply write some code to build voice recognition models on a java proxy.(work the same as siri proxy).

Then, the speex data packets are packaged in to CFPropertyList. There are also c libraries to read binary data and convert into the property list too. I haven't find a java library, but if comes to that, I'll just write one myself.

Then this property list of binary data is compressed using zlib. and sent to the guzzoni.apple.com server.

When you write code to build a proxy to intercept the packages between iphone and apple server. You'll find that iphone first some few lines of headers of a "ace" request, not http request!.
Then it start to send a bunch of binary packages, each with various length.

How to read the data iphone sent out
before the binary data, there is this ACE header immediatly follows the 4 or 5 lines of ace header. which look like this

ACE /ace HTTP/1.0
Host: 192.168.0.1
User-Agent: Assistant(iPhone/iPhone3,1; iPhone OS/5.0.1/9A405) Ace/1.0
Content-Length: 2000000000
X-Ace-Host: xxxxxxxxxxxxxxxxxxxxxxx

Then there is an empty line, which means there is two bytes of carriage return and line feed, "/r/n" immediately after the X-Ace-Host:xxxxxx...

To unzip the binary data using zlib. One important thing to know is, the header of the first package is universal to all follow up packages. which means, in java, if you write something like this.

while (line= read from iphone){
decompressor d= new decompressor();
d.unzip(line);
d.end();s
}

it will most likely not work. you have to look all packages as a whole when you unzip it.

decompressor d= new decompressor();
while (line= read from iphone){
d.unzip(line);
}
d.end();

Then binary data starts. The binary data start with a 4 bytes of ace header. it should always look like 0xAACCEE, the 4th byte doesn't really mean anything. So to unzip the binary data, start from 5th byte. In java socket programming, a InflaterInputStream is a handy thing to use. It automatically convert the InputStream of iphone to InflaterInputStream, and anything it spits out is unzipped data.

There are two types of data in the unzipped data. One is a "ping" package, it doesn't mean anything except it's a small packages (usually few bytes) send to apple to keep connection alive.. The 5 bytes header for this is 0x030000xxxx(in hex), the xxxx means the length of the package follows the 5 bytes header.

Another type is Property List. The list contain the iphone 4S identification key, the voice packages and etc. The 5 bytes header for this is 0x020000xxxx.

One very important thing to know about the xxxx is that, it's only 2 bytes, but it's very easy to get confused when calculating the length. Because every packages are one after another, if the length of 1st package is wrong, then you won't find the 2nd package.

The LSD and MSD is here to play a big part. While your machine reads from left to right, in each byte, the representation is tricky.
For example: if you see a xxxx as 0x00EC, it does not mean that it's E(15x16)+C(13), it actually means the opposite. it means C(13x16)+E(15).Because the C is actually the most significant digit. It's very important to check if your machine is reading this way.

If you successfully convert the binary data into property list, you can extract the iphone 4S certifications and save for your iphone 4 just like siriproxy.

If you see 0x0123, it's not 1x256+2x16+3, it's actually 1x16^3+3x16+2. Of course, in many systems a simple convert is enough, but just to make sure you know that this might cause problem. In java a Short.parsebyte(0x0123) works nicely to know the length of the packages.

Using I/O, When to close

In java, when using classes such as BufferedReader, it create a i/o stream to reads from the source. But one important for this is that the stream is always remain open unless you close it. Even after you scope out of the method that you created the instance, the stream is still there and not disposed.

This stream, if not closed, will still be forced quit after the program exit. Therefore if the program only use few streams, the problem is not detectable when running it.

BUT when you have a loop outside of the reader, to read thousands of files, you will get some kind of "stream reach its limit" exception. Meaning that the number of stream that's open is exceeded the limit. This limit is usually configured based on OS, in linux it's usually 10000. Theoriaticall if you have

while(true){

BufferedReader br = new   BufferedReader(new InputStream);

br.read();

}

You will get that exceed error after 10000 runs.

Now it occurs to me, that disposable doesn't do anything to close the stream, then switching to a different reader shouldn't do it neither.

Which means:

Buffered br= new BufferedReader(some stream);

br=new BufferedReader(another stream);

When br, this reference is redirected to another instance of BufferedReader, The original one is still not closed.

To verify this, I have run a while(true) loop on both cases that were mentioned above. (with Ubuntu 11.10)The program throws FileNotFoundException after 12000~14000 open streams. But does not throw any exceptions if the BufferedReader is properly closed.

This experiment is just to remind people that, most of us remembered to close a stream after the read. But did not realize that we also need to close the br before redeclaring the variable to something else.

P.S. This bufferedReader is just an example, other streams or readers, such as PrintStream, DataInputStream also need to be closed the same way.

Linux bash trick/bugs

This is simply a post of some thing that is easily ignored when programming in linux. For a while I had this idea, now start collecting.

1. In sh commands, when you have a shell file to run some commands like this

java -jar test.jar -para1

jara -jar test.jar -para2

The shell command actually pass the cr (carriage return) at the end of first line into the java parameters. This normally does not cause problems but if, just if, in the test.jar you try to creat a folder named para1. Then in linux it will creat a folder named "para1^M" and you won't be able to access it by typing

cd para1

because of that ^M symbol. so in your java code, you have to do a replace("r","") and replace("n","") before you create the folder.