FlexC and (non-) packed structs
ManAtWork
Posts: 2,176
I have defined the following data structures:
typedef struct
{
uint16_t regNum; // Registernummer für CMD
uint32_t regVal; // Register schreiben
uint32_t nomVal; // Sollwert senden
uint32_t outputs;// Ausgangszustände senden
uint32_t resv1;
uint16_t resv2;
} slot_tx;
typedef struct
{
uint16_t error; // Status / Fehler
uint32_t regVal; // Register lesen
uint32_t actVal; // Istwert empfangen
uint32_t inputs; // Eingangszustände empfangen
uint32_t data1;
uint16_t data2;
} slot_rx;
typedef struct
{
uint16_t page; // Page-Adresse
uint8_t data[16];
uint16_t resv;
} slot_up;
typedef struct
{
uint8_t cmd;
uint8_t sequence;
union {
slot_tx tx; // MOSI senden
slot_rx rx; // MISO empfangen
slot_up up; // Update Spezialfall
};
uint16_t crc;
} slot_t;
typedef struct
{
slot_t slot[BUS_maxSlots];
} BUS_Buffer;
When I write to the buffer like this
wrBuf->slot[i].tx.regNum= 0xAAAA;
wrBuf->slot[i].tx.regVal= 0xBBBB;
wrBuf->slot[i].tx.nomVal= 0xCCCC;
wrBuf->slot[i].tx.outputs= 0xDDDD;
I would normally expect that the memory dump looks like this11 05 AA AA BB BB 00 00 CC CC 00 00 DD DD 00 00 00 00 00 00
but instead I get this11 05 00 00 AA AA 00 00 BB BB 00 00 CC CC 00 00 DD DD 00 00
This means that the compiler has decided to insert padding so that the 16 bit and 32 bit data begins at an address divisible by 4. This is sometimes a good idea as it is more (time) efficient. But when I have to transfer data over a connection with quite limited bandwidth I sometimes like to have the buffers in a "packed" format without any padding bytes. Sometimes I have the choice of rearranging data fields to fit better. Sometimes I don't because the packet format was defined by somebody else like for ethernet packets. The P2 supports accessing word and long data at any (odd) byte address. So it would be good if I could make use of that and still use C structs for the buffers. Having to calculate adresses manually and using pointers for peek/poke makes the code look ugly.
For GNU C I think there is a compiler directive "#pragma pack" that tells the compiler to use packed format. Is there anything similar for FlexC?
Comments
No, unfortunately FlexC has no way to declare a packed structure. This comes from its requirement to support the P1. In theory a P2 only packed struct feature could be added, but every time I change the rules for structure layouts a bunch of bugs get introduced, so I'm reluctant to do that.
So what are the rules? I could live with something like "words have to be at an address divisible by 2, longs have to start at an address divisible by 4". I think I could re-arrange the ordering to make best use of the rules in most cases.
In the example above, however, the
uint16_t regNumis put to offset 4 although it should perfectly fit on offset 2.Prop1 was everything on longword boundaries I believe.
I think every element in the struct has the size of the largest element.
The struct as a whole has to be longword aligned, so that elements within it can be properly aligned. In your example regNum is the first element of the struct, so it gets longword aligned because the struct as a whole is long aligned. (An exception is made for structs which are smaller than 4 bytes, but that's rare.)
Otherwise the alignment within the struct goes by the alignment required for the items: uint8_t can have any alignment, uint16_t must be word aligned, uint32_t, structs, floats, doubles, and uint64_t must be long aligned.
So for example in:
struct blah { uint8_t x; uint8_t y; uint16_t z; uint16_t w; uint32_t a; } x = { 0x11, 0x12, 0x23, 0x24, 0x48 };the generated structure looks like:
The only padding needed is after
wto makeafit on a long boundary.Ok, that sounds good. So the solution for my example would be to bring the union out to the top level to avoid (sub-) structs starting in the middle of the top struct. There are some rare cases where I can't avoid that a struct starts at a word address that is not divisible by 4 because it is part of an ethernet packet. But I can solve that by simply copying the whole data field of the packet to a long-aligned buffer. Fortunatelly, the P2 is pretty fast copying hub RAM.
Hmm, maybe I'm too naive but I can't see a real problem why this couldn't be implemented relatively easily. I don't know how FlexC works internally, but the way I would do it is to store the offset of each field of a struct in the identifier table. So the decision which data field goes to which adress offset has to be made only once after parsing the struct typedef. That decision could be modified based on the presence of the #pragma pack attribute (and of course, for the P2, only). This theoretically shouldn't break any existing code. Users that rely on a field being at a specific address haven't used the #pragma so far because it didn't exist. So nothing changes. And any code generated simply uses the offsets from the table. So if the #pragma is used that offsets are different, but again, it should work because the P2 doesn't require long aligned adresses.
But that's just a theoretical mind game. As I said, I should get along with the current rules if I adjust my struct definitions a bit.
I've changed the definitions to this
typedef struct { uint8_t cmd; uint8_t sequence; uint16_t regNum; // Registernummer für CMD uint32_t regVal; // Register schreiben uint32_t nomVal; // Sollwert senden uint32_t outputs;// Ausgangszustände senden uint32_t resv1; uint16_t resv2; uint16_t crc; } slot_tx; typedef struct { uint8_t cmd; uint8_t sequence; uint16_t error; // Status / Fehler uint32_t regVal; // Register lesen uint32_t actVal; // Istwert empfangen uint32_t inputs; // Eingangszustände empfangen uint32_t data1; uint16_t data2; uint16_t crc; } slot_rx; typedef struct { uint8_t cmd; uint8_t sequence; uint16_t page; // Page-Adresse uint8_t data[16]; uint16_t resv; uint16_t crc; } slot_up; typedef struct { union { slot_tx tx; // MOSI senden slot_rx rx; // MISO empfangen slot_up up; // Update Spezialfall }; } slot_t;and now it works as it's supposed to.
Thanks for the support!
Glad to see it's more packed than I thought!
Ok, now I have re-arranged all my structs so that 32 bit fields start at an offset divisible by 4 from the beginning of the struct and 16 bit fields start at an even offset. This way FlexC calculates the offsets as expected and it works as if the structs were declared "packed".
But what if the whole structure starts at an address not divisible by 4, for example because it is part of an ethernet packet and there are an odd number of words in the buffer before the struct?
If I do something like
then FlexC seems to enforce strptr to be longword aligned (treat it like offset were equal to 8 or 12) even though the P2 could perfectly access longwords at any byte address.
@ersmith Is there any trick to avoid this? I mean, how can I assign an odd word address to strptr? strptr+= 2 doesn't work because it would increment by 2 times sizeof(MyStruct). (MyStruct)((void)strptr)+2 also doesn't work because casting void* back to MyStruct* enforces long alignment, again.
memcpy() should work but is not very efficient. Maybe an inline assembler
movto fake the type cast?But what if the whole structure starts at an address not divisible by 4, for example because it is part of an ethernet packet and there are an odd number of words in the buffer before the struct?
No, it doesn't require pointers to be longword aligned. When you assign an address to a pointer FlexC does no checking for alignment, even on P1 (where things actually have to be aligned to work). For example, here's a sample program that shows it isn't long aligned:
#include <stdio.h> #include <stdint.h> typedef struct blah { uint32_t x; // 4 byte aligned uint8_t c; // 1 byte aligned } MyStruct; uint8_t buffer[256]; int testit() { int offset = 11; MyStruct *strptr = (MyStruct *)&buffer[offset]; return strptr->c; } void main() { for (unsigned i = 0; i < sizeof(buffer); i++) { buffer[i] = i; } printf("testit() returns %d (should return 15)\n", testit()); }OK, I think what you're getting confused with here is that the size of the structure is a multiple of 4 (that is, FlexC pads all structs out so that in arrays and such they will start on a longword boundary). If you really want the struct to be, say, 5 bytes long then instead of
strptr += 2you'll have to do something likestrptr = (MyStruct *)(2*REAL_STRUCT_SIZE + (char*)strptr).You're right, I was confused and now it works.
The problem is when I spend the whole day debugging nasty mistakes in my C code I start to doubt everything. This leads to wrong assumptions and blaming of the wrong things or people. My fault.
Thanks for your support.
Hi @ersmith, it's me. This issue is popping up, again.
Last time I could solve it by re-arranging the fields of my structs. That was possible because it was my own protocol so I had the choice to modify it in any way I wanted. This time it's different. The protocol is defined in a standard and I'm bound to the existing definitions.
typedef struct { uint8_t dest[6]; uint8_t src[6]; uint16_t ptype; // protocol type EC_EtherType unsigned size : 12; // sum of datagram sizes unsigned htype: 4; // header type } EC_FrameHeader;This is still one of the easiest. The total size is 16 bytes. The FlexC compiler places src at offset 8 instead of 6 but I can fix that by declaring and address-pair field of 12 bytes. But there are many more of those structures and most of them don't follow the alignment rules of FlexC.
typedef struct { uint8_t command; uint8_t index; uint32_t address; // logical address or LSW=offset, MSW=slave unsigned length: 14; unsigned cirqul: 1; // circulated flag unsigned moreFol: 1; // more datagrams follow flag uint16_t events; // event mask of all slaves ORed } EC_DatagramHeader;I have no idea for an easy fix for this at the moment. In Spin I can use DAT sections with any alignment I need but DAT uses fixed memory locations that then have to be copied to the destination buffer. That's not multitasking safe.
I never wanted to do packed structs because they would be such a pain to implement on P1. Doing a P2-only version probably wouldn't be too bad, but it would impact enough fiddly bits of the compiler that it will take a little while. I've just started a new job, so I don't know when I'll have cycles to get to this, sorry,
Don't worry. At the beginning I was very sceptical because I thought that this would make the code very inefficient and and would stop me from re-using existing code. It turned out that I do not port from other platforms anyway because it is to inefficient or I just don't like it. It means that I have to use some more shift or copy operations but it's not as bad as expected. We can do optimizations an clean up later.