Documenting the P2 varint format (read by RFVAR / RFVARS instructions)

Conga · 2017-05-05 08:53

It's not urgent, but I think it's worth doing.

'varint' = Varying-length Integer, or Variable-length Integer.

This is my current understanding of the format implemented by Chip (may be wrong).

Unsigned varint (RFVAR):

0xxx xxxx -> 0xxx xxxx (byte value less than 128)

1xxx xxxx 0yyy yyyy -> 00xx xxxx xyyy yyyy ? or 00yy yyyy yxxx xxxx

1xxx xxxx 1yyy yyyy 0zzz zzzz

1xxx xxxx 1yyy yyyy 1zzz zzzz vvvv vvvv

Signed varint (RFVARS):

0sxx xxxx -> ssxx xxxx

Where is the sign bit in the following?

1xxx xxxx 0yyy yyyy

1xxx xxxx 1yyy yyyy 0zzz zzzz

1xxx xxxx 1yyy yyyy 1zzz zzzz vvvv vvvv

cgracey · 2017-05-05 09:58

RFVAR reads from 1 to 4 bytes, lower bytes first, depending on the MSB's it sees in the coming byte sequence:

%0aaaaaaa                                    -->                             %aaaaaaa
%1aaaaaaa, %0bbbbbbb                         -->                     %bbbbbb_baaaaaaa
%1aaaaaaa, %1bbbbbbb, %0ccccccc              -->             %ccccc_ccbbbbbb_baaaaaaa
%1aaaaaaa, %1bbbbbbb, %1ccccccc, %dddddddd   -->    %ddddd_dddccccc_ccbbbbbb_baaaaaaa

In the case of RFVARS, the value is sign-extended to 32 bits from the MSB of the data.

For both RFVAR and RFVARS, C = MSB of the data and Z = data is zero.

These instructions take two clocks.

Rayman · 2017-05-05 14:28

Wonder if it'd be more useful to have C and Z give you a 2-bit count of the number of bytes read...

cgracey · 2017-05-05 20:08

You can always do a GETPTR to find out where you are.

Rayman · 2017-05-07 01:09

I'm guessing you must be encoding the number of bytes needed into the first byte, right?

I was just thinking that this might be useful as a simple way of compressing data, maybe something like RLE...

I guess using a couple bytes in the first byte to say how many bytes in the packet is OK.

But, would be faster if was transferred to C and Z flags...

cgracey · 2017-05-07 01:23

Look at the diagram I made above. It's the MSB of the byte(s) that says if there's more data, or not.

Rayman · 2017-05-07 01:33

Yes, the encoder would set those bits...

But, the decoder wouldn't know, right?

cgracey · 2017-05-07 01:37

The data tells the RFVAR circuit how much of it to grab, byte by byte.

cgracey · 2017-05-07 01:49

This is for quickly reading byte-efficient values from memory in a compose-once/read-many format. It's really great for bytecode interpreters.

Rayman · 2017-05-07 02:05

That's fine. I was just thinking about using this to compress a simple image.
Maybe 4-byte data would be a 24-bit color value and 1-byte data would be a count of how many pixels to paint with that color. Similar to RLE encoding...

ozpropdev · 2017-05-07 05:00

Thinking about the RFVAR instruction you could complement it with a MAKEVAR instruction.

wire [28:0] makevar = {s[28,21],
			|s[28:21],
			s[20,14],
			|s[28,14]],
			s[13:7],
			|s[28:7],
			s[6:0]};

For example

	makevar	myvar		'make variable length value

	topone	adj,myvar	'get magnitude
	shr	adj,#3
	add	adj,#1		'= number of bytes

	rep	@.loop,adj	'save new varaiable length value
	wrbyte	myvar,ptra++
	shr	myvar,#8
.loop

or even better a WVLONG instruction as well that works similar to WMLONG except only bytes below the highest zero byte are written.

	makevar	myvar		'make variable length value
	wvlong	myvar,ptra	'write new var (1-4 bytes)

This way the values are no longer "compose once" type.

Ducks for cover.....

MJB · 2017-05-07 10:04

cgracey wrote: »
RFVAR reads from 1 to 4 bytes, lower bytes first, depending on the MSB's it sees in the coming byte sequence:
%0aaaaaaa                                    -->                             %aaaaaaa
%1aaaaaaa, %0bbbbbbb                         -->                     %bbbbbb_baaaaaaa
%1aaaaaaa, %1bbbbbbb, %0ccccccc              -->             %ccccc_ccbbbbbb_baaaaaaa
%1aaaaaaa, %1bbbbbbb, %1ccccccc, %dddddddd   -->    %ddddd_dddccccc_ccbbbbbb_baaaaaaa
In the case of RFVARS, the value is sign-extended to 32 bits from the MSB of the data.

For both RFVAR and RFVARS, C = MSB of the data and Z = data is zero.

These instructions take two clocks.

when seeing this (and I did not follow all the discussion ...)
I wonder if there is a inverse WFVAR, which takes a long and write it so it can be read with RFVAR(S)?
Doing this manually is long.
Or will this be done by the assembler / compiler only and not used in runtime?
This would be nice for some data compression/uncompression ...

EDIT: OZ - you were faster ;-) ... but obviously the idea is in the 'air'

cgracey · 2017-05-07 17:40

I think it might be a waste to make a WFVAR, since it may not have much practical use (data would always have to be read sequentially and from the beginning), plus it would require a 32-bit negation circuit for WFVARS. Then, there's the problem that not all data would even be representable. RFVAR{S} is really just for run-time stuff.

evanh · 2017-05-07 22:44

Rayman wrote: »

Maybe 4-byte data would be a 24-bit color value and 1-byte data would be a count of how many pixels to paint with that color. Similar to RLE encoding...

I'd recommend using something like the Amiga's HAM mode. It was notably superior because it assumed constant variation in the tone. It's weak point was with the RGB colour space it used, which meant colourised artefacting ...

Documenting the P2 varint format (read by RFVAR / RFVARS instructions)

Comments