[mdlug] Reverse-engineering data protocols

[David McMillan]

On 3/1/2011 1:29 PM, Ingles, Raymond wrote:
>> From: David McMillan
>>       I'm *pretty* certain that this data isn't enciphered to prevent
>> third-party access, but it's not plaintext being pushed through a
>> Telnet-esque connection either.  So I'm a bit stuck.
> I see two approaches, one short-term, one long-term.
>
>   1) Short term - see if you can 'record' the data coming into Linux
> (WireShark, netcat, whatever) and then 'play it back' later to the
> Windows logger. This would allow you to do a bunch of tests, saving to
> different files, and then 'batch log' them to Windows later for
> processing.
     Unfortunately, so far I can't get the client to stream data to 
anything except the proprietary server.  There's some sort of initial 
handshake between the two that so far is beyond my (nearly-nonexistant) 
TCP/IP skills to figure out, much less duplicate.  I have determined to 
my satisfaction that just opening the correct port and listening on it 
isn't enough.
>   2) Long term - figure out the structure of the file. Is the data
> basically numbers? Then do a little work, taking chunks of the file and
> interpreting them as numbers. I'd guess they were either integers, or
> IEEE floating-point numbers. Are you familiar with the concept of
> 'endianness'? You may have to take that into account. But given the
> input and the plaintext file, it should be possible to figure out the
> protocol.
     Endian-ness, as in Motorola vs Intel?  Yeah, I'm familiar with that 
-- I have to juggle it on mismatched industrial systems often enough.
     The data is definitely floating-point.  I'm pretty certain its 
32bit (the source system is 32bit natively), and IEEE encoding seems a 
reasonable assumption.
     Yeah, I thought that, given the two files, it should be doable.  
The problem is, I put the two side-by-side and there's nothing visible 
that relates the two, no loose ends to grab onto.  I might as well be 
looking at two files in English and Japanese for all the connections I 
can draw between them.  The primary reason I approached the list was in 
the hopes that there might be some key insight to serve as a starting 
point.  Someone with more experienced eyes might look at this and see a 
pattern where all I see is white noise.