how to read AIS data from encapsulated NMEA VDO sentence

See this site:

http://www.navicon.dk/web/normal.php?pageid=95







ccorsani.TakeThisOut@gmail.com escreveu:

> Hi all, I am a navigation software developer from Italy.
>
> I need to understand how to read AIS data inside the "blob" contained
> in VDO NMEA sentece. I have the ITU-R M.1371-1 document that should let
> me understand such format. Unfortunately I really cannot understand how
> to read the bits.
>
> Someone, some times ago, posted such message:
>
> >The AIS sentences are in a strange format, encapsulated 6 >bits. One
> >example here:
>
> >'!AIVDO,1,1,,,13:r`R5P1orpG60JeHgRSj4l0000,0*56
> > ' 0 1 2 34 5 6
> > '1 Total number of sentences needed to transfer the message, 1 to 9
> > '2 Message sentence number, 1 to 9
> > '3 Sequential message identifier, 0 to 9
> > '4 AIS channel number
> > '5 Encapsulateled 6-bit radio message
> > '6 Number of fill-bits, 0 to 5
> >
> >
> > '14eG=ch021rp4FTJdTGRRR0605q4
> >
> >
> > 'VDO.Identifier : 1 000001
> > 'Repeat indicator : 0 00
> > 'MMSI : 316001711 >010010110101011100110110101111
> > 'VDO1.NavStatus : 0 0000
> > 'VDO1.ROT : 0 00000000
> > 'VDO1.SOG : 12.9 kns 0010000001
> > 'VDO1.PosAcc : 1 1
> > 'VDO1.Latitude : 46º 39.3182'N >1101011100000010001011010010
> > 'VDO1.Longitude : 71º 38.2703'W
> >0011010101100100100010111100010100010100
> > 'VDO1.COG : 065º 010000000000
> > 'VDO1.Heading : 064º 110000000
> > 'VDO1.TimeStamp : 3 sec. 000101
> > 'VDO1.NavStatus : 0 1
> > 'VDO1.CommStatus : 15:17 h 11001000100
>
> I really cannot understand how he found such information inside
> 14eG=ch021rp4FTJdTGRRR0605q4. Please Help
>
> Ciao
> Cristiano
 >> Stay informed about: how to read AIS data from encapsulated NMEA VDO sentence 

Pascal wrote:

> See this site:
>
> http://www.navicon.dk/web/normal.php?pageid=95
>
>
>
>
>
>
>
> ccorsani.DeleteThis@gmail.com escreveu:
>
>
>>Hi all, I am a navigation software developer from Italy.
>>
>>I need to understand how to read AIS data inside the "blob" contained
>>in VDO NMEA sentece. I have the ITU-R M.1371-1 document that should let
>>me understand such format. Unfortunately I really cannot understand how
>>to read the bits.
>>
>>Someone, some times ago, posted such message:
>>
>>
>>>The AIS sentences are in a strange format, encapsulated 6 >bits. One
>>>example here:
>>
>>>'!AIVDO,1,1,,,13:r`R5P1orpG60JeHgRSj4l0000,0*56
>>> ' 0 1 2 34 5 6
>>> '1 Total number of sentences needed to transfer the message, 1 to 9
>>> '2 Message sentence number, 1 to 9
>>> '3 Sequential message identifier, 0 to 9
>>> '4 AIS channel number
>>> '5 Encapsulateled 6-bit radio message
>>> '6 Number of fill-bits, 0 to 5
>>>
>>>
>>> '14eG=ch021rp4FTJdTGRRR0605q4
>>>
>>>
>>> 'VDO.Identifier : 1 000001
>>> 'Repeat indicator : 0 00
>>> 'MMSI : 316001711 >010010110101011100110110101111
>>> 'VDO1.NavStatus : 0 0000
>>> 'VDO1.ROT : 0 00000000
>>> 'VDO1.SOG : 12.9 kns 0010000001
>>> 'VDO1.PosAcc : 1 1
>>> 'VDO1.Latitude : 46º 39.3182'N >1101011100000010001011010010
>>> 'VDO1.Longitude : 71º 38.2703'W
>>>0011010101100100100010111100010100010100
>>> 'VDO1.COG : 065º 010000000000
>>> 'VDO1.Heading : 064º 110000000
>>> 'VDO1.TimeStamp : 3 sec. 000101
>>> 'VDO1.NavStatus : 0 1
>>> 'VDO1.CommStatus : 15:17 h 11001000100
>>
>>I really cannot understand how he found such information inside
>>14eG=ch021rp4FTJdTGRRR0605q4. Please Help
>>
>>Ciao
>>Cristiano
>
>
Disassembling message protocols is tedious by hand. Automated
equipment does it with logic circuitry, which accomplishes tedious
tasks easily and knows the secret code.

We humans must parse manually.

The data is actually send as a bitstream and the protocol for every
device is proprietary.

The data presented above is actually encoded as hexadecimal or
ascii encode binary characters, so it actually looks more like
111101100101101010101010101011011001100110101010101010111010101...,
etc. These bits are sent as analogue levels, data encoded in one
long serial string. 100 character might be 800 bits, or 1600 bits,
in number base 10 notation. It might use another numerical base,
likely octal or hexadecimal.

The ports have their own protocols, and take each byte of 7 or 8
bits (as one example, I am not certain what this one actually uses,
perhaps 16 bit bytes) as a character. Each group of bits represents
one character in the string given above. The alphabet used by the
system may have many more than the 26 in our human alphabet, and
some of those characters are specific to the device in use. The
spacing and syntax is combined with error correcting parity (even -
odd) codes and checksum numbers which must also be decoded in logic
if you are writing an interface. Each receiving device must be up to
date with whatever protocol is used by all of the transmitters if it
is not to become discombobulated and confused, losing track of
character boundaries, etc, and those protocols seem to change with
each new generation of devices.

Different portions of the data string may use different schemes, ie,
the first 6 bits might be all ones, as a preamble and sync. The
next 4 bits might be a code to identify the remainder as a specific
type of statement, ie, sender I.D. Sometimes the info is bitsliced,
so that each bit represents a switch setting in the receiving parser
to cause it to understand that it is radar, weather, whatever.

I once independantly decoded the data sent by telco in it's caller
id string, sent between the ring tones to your phone. There are
padding bits, a preamble, and then each character is sent twice. It
was painful for a while, as I had to scope out the data without
benefit of knowing the protocol, even as simple as it was, using a
capture buffer to read each character out in ascii values, a sofware
protocol analyzer I wrote myself, which spit out gibberish ascii
codes until I learned to read it. If it had not been straight ascii,
normal in telephone modems, I would still be propping up my eyelids
with toothpicks.

You seem to have a protocol guide or data map, which you must now
come to know imtimately. Reverse engineering software data is fun
for some masochists, for a while.

400 page logic diagrams is where I cut my teeth, back in the 60's.
It's one way to burn out your brain so far as computers is
concerned. I got fed up with them, and retired years ago. Wouldn't
touch it, now, without cash up front and a clearly defined mission
statement engraved in titanium.

Good luck.

Terry K -SofDevCo-

wagthedoggie.DeleteThis@theduodecaheptagon.mil
 >> Stay informed about: how to read AIS data from encapsulated NMEA VDO sentence 

Terry, you're making this sound alot more complicated than it really
is.

The 6-bit encoded part of a VDM sentence is simply a bitstream, packed
using a special code. Each character represents 6 bits of data from
0x00 ot 0x3F. It starts with the message ID (6 bits), repeat (2 bits),
MMSI (30 bits), etc. This is all defined in m.1371 and 61993-2, even if
it is a bit confusing at first.

The first 6 bits being a character and the message ID make it easy to
tell that the example string above is a message 1.
 >> Stay informed about: how to read AIS data from encapsulated NMEA VDO sentence 

coderpunk.TakeThisOut@gmail.com Wrote:
> Terry, you're making this sound alot more complicated than it really
> is.
>
> The 6-bit encoded part of a VDM sentence is simply a bitstream, packed
> using a special code. Each character represents 6 bits of data from
> 0x00 ot 0x3F. It starts with the message ID (6 bits), repeat (2 bits),
> MMSI (30 bits), etc. This is all defined in m.1371 and 61993-2, even
> if
> it is a bit confusing at first.
>
> The first 6 bits being a character and the message ID make it easy to
> tell that the example string above is a message 1.


Hi guys,

I'm actually working on a similar project right now except I have to do
the ENCODE part of the VDM message. At first I thought every character
represents 6-bit but that is not true. As you can see in the message
there are lowered case alphabet characters (i.e. 'e') and the ascii
value for 'e' is more than 6-bits. I'm a little stuck on my project
right now too, can someone give me more info. Perhaps an example in C
or C++. Thanks.


--
tcdang714
 >> Stay informed about: how to read AIS data from encapsulated NMEA VDO sentence 

"tcdang714" <tcdang714.28chyl RemoveThis @news.boatbanter.com> wrote in message
news:tcdang714.28chyl@news.boatbanter.com...
>
> coderpunk RemoveThis @gmail.com Wrote:
>> Terry, you're making this sound alot more complicated than it really
>> is.
>>
>> The 6-bit encoded part of a VDM sentence is simply a bitstream, packed
>> using a special code. Each character represents 6 bits of data from
>> 0x00 ot 0x3F. It starts with the message ID (6 bits), repeat (2 bits),
>> MMSI (30 bits), etc. This is all defined in m.1371 and 61993-2, even
>> if
>> it is a bit confusing at first.
>>
>> The first 6 bits being a character and the message ID make it easy to
>> tell that the example string above is a message 1.
>
>
> Hi guys,
>
> I'm actually working on a similar project right now except I have to do
> the ENCODE part of the VDM message. At first I thought every character
> represents 6-bit but that is not true. As you can see in the message
> there are lowered case alphabet characters (i.e. 'e') and the ascii
> value for 'e' is more than 6-bits. I'm a little stuck on my project
> right now too, can someone give me more info. Perhaps an example in C
> or C++. Thanks.
>
>
> --
> tcdang714

I realize that you've asked about *encoding*, but here's an example of a
decoder written in visual basic, from a program I wrote for my pocket PC.
Perhaps you will be able to reverse-engineer something from this.
------------------
Function AsciiToHex6(AsciiIn)
AsciiToHex6 = A2H6(Asc(AsciiIn))
End Function

Dim A2H6(128)
Sub InitA2H6
Dim i, h
For i = 0 To 127
h = i
If i < 48 Then
h = -1
ElseIf i < 88 Then
h = h - 48
ElseIf i > 119 Then
h = -1
ElseIf i < 96 Then
h = -1
Else h = h - 56
End If
A2H6(i) = h
Next
End Sub

InitA2H6
------------------
The array A2H6() contains the hex6 value of the ascii input character, using
the character as an index. I put "-1" in the illegal positions, and ought
to do a little more defensive error-checking (but I probably won't ever get
around to it). The function AsciiToHex6(AsciiIn) just looks up the code and
returns it.

The IEC documents that describe the messages and coding aren't free, but
while I was googling around I did find this one:
http://www.gicoms.go.kr/knowledge/download.asp?filename=IEC%20standard%2061993(class%20A%20AIS).pdf&filepath=D:%5CGICOMS_FILE%5Cupdown%5CIEC%20standard%2061993(class%20A%20AIS).pdf

This contains a decent description of the 6-bit ascii encoding, as well as
the message structure. Note that the AIS data elements are not necessarily
six-bits long, so there will be an arbitrary alignment of the various
parameters across the 6-bit-ascii character stream. Also, some of the AIS
messages are longer than allowed in the NMEA sentence, so the messages have
to be split into multiple NMEA sentence. I can post my code for decoding
the arbitrary data if you think it would be instructive.

Be aware that some of the earlier discussions of 6-bit ascii encoding for
AIS show a different encoding (don't ask me how long I struggled with that
issue!). The one in my code sample, and the document above seem to be the
encoding actually used.

Good Luck,
Paul
 >> Stay informed about: how to read AIS data from encapsulated NMEA VDO sentence 

I too have been struggling with this.

I convert every 6 bits to a byte and convert to the approriate 8 bit
ascii character.

It all works for message type 1.

So, I thought I'd look at message type 5, which is

id 6 bits
repeat 2
user id 30
version 2
imo 30
call sign 42
name 120

It works up until imo, then for the call sign I get gibberish.

It seems characters don't like my approach ...

Does anyone have any code or suggestions?

Thanks
 >> Stay informed about: how to read AIS data from encapsulated NMEA VDO sentence 

Actually, I re-read what I posted and it didn't really make sense.

1. Start with the array of ascii characters
2. Convert to the 6 bit binary value.
3. Convert this byte array to a 6 bit bitstream.
4. Then just pull out the bits yuo want using the various bit
operators. I tried to use a bit-field, but it didn't seem to work in
Windows.

As I said, it all works until I get to character arrays (call sign etc)
 >> Stay informed about: how to read AIS data from encapsulated NMEA VDO sentence 

Paul wrote:
> <nickw473.RemoveThis@hotmail.com> wrote in message
> news:1149652247.992435.169280@y43g2000cwc.googlegroups.com...
> > Actually, I re-read what I posted and it didn't really make sense.
> >
> > 1. Start with the array of ascii characters
> > 2. Convert to the 6 bit binary value.
> > 3. Convert this byte array to a 6 bit bitstream.
> > 4. Then just pull out the bits yuo want using the various bit
> > operators. I tried to use a bit-field, but it didn't seem to work in
> > Windows.
> >
> > As I said, it all works until I get to character arrays (call sign etc)
> >
>
> Here is a portion of the code i wrote to handle message type 5 (I'm only
> handling the "Ship Static and Voyage Related Data
> " variant for now). My comments follow the code. Pay attention to my
> function H6StrToAsc, and how I manipulate the ascii.
>

I'll assume Paul's code is right He's skipping the intermediate step
which may be where the confusion lies. There are several layers of
things happening here.

0. Bits over the air: 9600bps GMSK data.
1. Demodulated bitstream, eg. 168 bits for a message 1.
2. Actual message content like Message ID, MMSI, Ship Name.
3. Serial output of bitstream using 6-bit encoding.

The data you see in the VDM sentence is a 6-bit encoded representation
of the raw bitstream, first bit on the left, last bit on the right (as
God intended things!).

To get the ship name you need to either skip ahead like Paul did, or
convert the 6-bit back to its bitstream. Then you grab the bits in the
name position and apply the m.1371 ASCII encoding rules to those bits.
BUT! That conversion is DIFFERENT from the 6-bit encode/decode. That is
probably what was tripping you up.

..cp
 >> Stay informed about: how to read AIS data from encapsulated NMEA VDO sentence 

<coderpunk DeleteThis @gmail.com> wrote in message
news:1150254823.702754.271150@g10g2000cwb.googlegroups.com...
>
> Paul wrote:
>> <nickw473 DeleteThis @hotmail.com> wrote in message
>> news:1149652247.992435.169280@y43g2000cwc.googlegroups.com...
>> > Actually, I re-read what I posted and it didn't really make sense.
>> >
>> > 1. Start with the array of ascii characters
>> > 2. Convert to the 6 bit binary value.
>> > 3. Convert this byte array to a 6 bit bitstream.
>> > 4. Then just pull out the bits yuo want using the various bit
>> > operators. I tried to use a bit-field, but it didn't seem to work in
>> > Windows.
>> >
>> > As I said, it all works until I get to character arrays (call sign
>> > etc)
>> >
>>
>> Here is a portion of the code i wrote to handle message type 5 (I'm only
>> handling the "Ship Static and Voyage Related Data
>> " variant for now). My comments follow the code. Pay attention to my
>> function H6StrToAsc, and how I manipulate the ascii.
>>
>
> I'll assume Paul's code is right He's skipping the intermediate step
> which may be where the confusion lies. There are several layers of
> things happening here.
>
> 0. Bits over the air: 9600bps GMSK data.
> 1. Demodulated bitstream, eg. 168 bits for a message 1.
> 2. Actual message content like Message ID, MMSI, Ship Name.
> 3. Serial output of bitstream using 6-bit encoding.
>
> The data you see in the VDM sentence is a 6-bit encoded representation
> of the raw bitstream, first bit on the left, last bit on the right (as
> God intended things!).
>
> To get the ship name you need to either skip ahead like Paul did, or
> convert the 6-bit back to its bitstream. Then you grab the bits in the
> name position and apply the m.1371 ASCII encoding rules to those bits.
> BUT! That conversion is DIFFERENT from the 6-bit encode/decode. That is
> probably what was tripping you up.
>
> .cp

I hereby give you a full money-back guarantee that my code is correct. At
least it seems to usually kind of work. The only trouble I've seen is that
I get a spurious (or incorrectly encoded?) rate-of-turn value sometimes,
positive or negative full-scale, if I recall correctly. I haven't posted
that code, though.

As for the intermediate step of converting to a serial bitstream, there is
certainly nothing wrong with doing so, and it may indeed simplify some
operations. I found that with the lack of easy bit-field operators in
Visual Basic, it was just as simple to use the VDM message character string
directly. All the data is there, and I found no good reason to do an
additional conversion. If I were dong this in c, I probably would have gone
to the bitstream. And yes, the ASCII conversion is different than the 6-bit
encode/decode, which is why I directed the original poster to the H6StrToAsc
function in my code.

I was about to ask for help with my ROT problem, and in researching my
sources I just found a later spec than I had been using which shows that
+/-127 values have been taken over to indicate a heading-derived ROT, rather
than a turn-indicator-derived ROT (which uses the values between -126 and
+126). Problem solved! Now, I guarantee double your money back!

Regards,
Paul
 >> Stay informed about: how to read AIS data from encapsulated NMEA VDO sentence 

Dear
I am development any software of navigation in Peru for present data AIS in Chart electronic. I have the frame AIVDM decode, the ais SR162 only receive frame !AIVDM wich have info the MMSI, position, course, speed, others.. but i want get the name of ship, and more data related of voyage,... how program the receptor for get the name of ship ??? and how this encode, is equal to AIVDM (6 bits)..? Thanks..

Regards
 >> Stay informed about: how to read AIS data from encapsulated NMEA VDO sentence 

Kenny wrote:

>
> Dear
> I am development any software of navigation in Peru for present data
> AIS in Chart electronic. I have the frame AIVDM decode, the ais SR162
> only receive frame !AIVDM wich have info the MMSI, position, course,
> speed, others.. but i want get the name of ship, and more data related
> of voyage,... how program the receptor for get the name of ship ???
> and how this encode, is equal to AIVDM (6 bits)..? Thanks..

Ship static data like name, destination, eta are contained in Message
5. This is only sent every 6 minutes by Class A units. If you have a
single channel receiver or the target signal is not strong you may miss
this message.

If you need some AIS parser code contact me privately, I have a 'c'
library that is complete except for documentation.

..cp
 >> Stay informed about: how to read AIS data from encapsulated NMEA VDO sentence