Shop OBEX P1 Docs P2 Docs Learn Events
FlexC and (non-) packed structs — Parallax Forums

FlexC and (non-) packed structs

I have defined the following data structures:

typedef struct
{
        uint16_t regNum; // Registernummer für CMD
        uint32_t regVal; // Register schreiben
        uint32_t nomVal; // Sollwert senden
        uint32_t outputs;// Ausgangszustände senden
        uint32_t resv1;
        uint16_t resv2;
} slot_tx;

typedef struct
{
        uint16_t error;  // Status / Fehler
        uint32_t regVal; // Register lesen
        uint32_t actVal; // Istwert empfangen
        uint32_t inputs; // Eingangszustände empfangen
        uint32_t data1;
        uint16_t data2;
} slot_rx;

typedef struct
{
        uint16_t page;   // Page-Adresse
        uint8_t  data[16];
        uint16_t resv;
} slot_up;

typedef struct
{
        uint8_t cmd;
        uint8_t sequence;
        union {
                slot_tx tx; // MOSI senden
                slot_rx rx; // MISO empfangen
                slot_up up; // Update Spezialfall
        };
        uint16_t crc;
} slot_t;

typedef struct
{
        slot_t slot[BUS_maxSlots];
} BUS_Buffer;

When I write to the buffer like this

      wrBuf->slot[i].tx.regNum= 0xAAAA;
      wrBuf->slot[i].tx.regVal= 0xBBBB;
      wrBuf->slot[i].tx.nomVal= 0xCCCC;
      wrBuf->slot[i].tx.outputs= 0xDDDD;

I would normally expect that the memory dump looks like this
11 05 AA AA BB BB 00 00 CC CC 00 00 DD DD 00 00 00 00 00 00
but instead I get this
11 05 00 00 AA AA 00 00 BB BB 00 00 CC CC 00 00 DD DD 00 00
This means that the compiler has decided to insert padding so that the 16 bit and 32 bit data begins at an address divisible by 4. This is sometimes a good idea as it is more (time) efficient. But when I have to transfer data over a connection with quite limited bandwidth I sometimes like to have the buffers in a "packed" format without any padding bytes. Sometimes I have the choice of rearranging data fields to fit better. Sometimes I don't because the packet format was defined by somebody else like for ethernet packets. The P2 supports accessing word and long data at any (odd) byte address. So it would be good if I could make use of that and still use C structs for the buffers. Having to calculate adresses manually and using pointers for peek/poke makes the code look ugly.

For GNU C I think there is a compiler directive "#pragma pack" that tells the compiler to use packed format. Is there anything similar for FlexC?

Comments

  • No, unfortunately FlexC has no way to declare a packed structure. This comes from its requirement to support the P1. In theory a P2 only packed struct feature could be added, but every time I change the rules for structure layouts a bunch of bugs get introduced, so I'm reluctant to do that.

  • So what are the rules? I could live with something like "words have to be at an address divisible by 2, longs have to start at an address divisible by 4". I think I could re-arrange the ordering to make best use of the rules in most cases.

    In the example above, however, the uint16_t regNum is put to offset 4 although it should perfectly fit on offset 2.

  • evanhevanh Posts: 15,126

    Prop1 was everything on longword boundaries I believe.

  • RaymanRayman Posts: 13,798

    I think every element in the struct has the size of the largest element.

  • @ManAtWork said:
    So what are the rules? I could live with something like "words have to be at an address divisible by 2, longs have to start at an address divisible by 4". I think I could re-arrange the ordering to make best use of the rules in most cases.

    In the example above, however, the uint16_t regNum is put to offset 4 although it should perfectly fit on offset 2.

    The struct as a whole has to be longword aligned, so that elements within it can be properly aligned. In your example regNum is the first element of the struct, so it gets longword aligned because the struct as a whole is long aligned. (An exception is made for structs which are smaller than 4 bytes, but that's rare.)

    Otherwise the alignment within the struct goes by the alignment required for the items: uint8_t can have any alignment, uint16_t must be word aligned, uint32_t, structs, floats, doubles, and uint64_t must be long aligned.

    So for example in:

    struct blah {
        uint8_t x;
        uint8_t y;
        uint16_t z;
        uint16_t w;
        uint32_t a;
    } x = {
        0x11, 0x12, 0x23, 0x24, 0x48
    };
    

    the generated structure looks like:

    byte    $11, $12, $23, $00, $24, $00, $00, $00, $48, $00, $00, $00
    

    The only padding needed is after w to make a fit on a long boundary.

  • @ersmith said:
    The struct as a whole has to be longword aligned, so that elements within it can be properly aligned. In your example regNum is the first element of the struct, so it gets longword aligned because the struct as a whole is long aligned. (An exception is made for structs which are smaller than 4 bytes, but that's rare.)

    Otherwise the alignment within the struct goes by the alignment required for the items: uint8_t can have any alignment, uint16_t must be word aligned, uint32_t, structs, floats, doubles, and uint64_t must be long aligned.

    Ok, that sounds good. So the solution for my example would be to bring the union out to the top level to avoid (sub-) structs starting in the middle of the top struct. There are some rare cases where I can't avoid that a struct starts at a word address that is not divisible by 4 because it is part of an ethernet packet. But I can solve that by simply copying the whole data field of the packet to a long-aligned buffer. Fortunatelly, the P2 is pretty fast copying hub RAM.

  • ManAtWorkManAtWork Posts: 2,049
    edited 2022-12-30 09:35

    @ersmith said:
    No, unfortunately FlexC has no way to declare a packed structure. This comes from its requirement to support the P1. In theory a P2 only packed struct feature could be added, but every time I change the rules for structure layouts a bunch of bugs get introduced, so I'm reluctant to do that.

    Hmm, maybe I'm too naive but I can't see a real problem why this couldn't be implemented relatively easily. I don't know how FlexC works internally, but the way I would do it is to store the offset of each field of a struct in the identifier table. So the decision which data field goes to which adress offset has to be made only once after parsing the struct typedef. That decision could be modified based on the presence of the #pragma pack attribute (and of course, for the P2, only). This theoretically shouldn't break any existing code. Users that rely on a field being at a specific address haven't used the #pragma so far because it didn't exist. So nothing changes. And any code generated simply uses the offsets from the table. So if the #pragma is used that offsets are different, but again, it should work because the P2 doesn't require long aligned adresses.

    But that's just a theoretical mind game. As I said, I should get along with the current rules if I adjust my struct definitions a bit.

  • I've changed the definitions to this

    typedef struct
    {
      uint8_t cmd;
      uint8_t sequence;
      uint16_t regNum; // Registernummer für CMD
      uint32_t regVal; // Register schreiben
      uint32_t nomVal; // Sollwert senden
      uint32_t outputs;// Ausgangszustände senden
      uint32_t resv1;
      uint16_t resv2;
      uint16_t crc;
    } slot_tx;
    
    typedef struct
    {
      uint8_t cmd;
      uint8_t sequence;
      uint16_t error;  // Status / Fehler
      uint32_t regVal; // Register lesen
      uint32_t actVal; // Istwert empfangen
      uint32_t inputs; // Eingangszustände empfangen
      uint32_t data1;
      uint16_t data2;
      uint16_t crc;
    } slot_rx;
    
    typedef struct
    {
      uint8_t cmd;
      uint8_t sequence;
      uint16_t page;   // Page-Adresse
      uint8_t  data[16];
      uint16_t resv;
      uint16_t crc;
    } slot_up;
    
    typedef struct
    {
      union {
        slot_tx tx; // MOSI senden
        slot_rx rx; // MISO empfangen
        slot_up up; // Update Spezialfall
      };
    } slot_t;
    

    and now it works as it's supposed to.
    Thanks for the support!

  • RaymanRayman Posts: 13,798

    Glad to see it's more packed than I thought!

  • Ok, now I have re-arranged all my structs so that 32 bit fields start at an offset divisible by 4 from the beginning of the struct and 16 bit fields start at an even offset. This way FlexC calculates the offsets as expected and it works as if the structs were declared "packed".

    But what if the whole structure starts at an address not divisible by 4, for example because it is part of an ethernet packet and there are an odd number of words in the buffer before the struct?

    If I do something like

    uint8_t buffer[1024];
    int offset = 10;
    MyStruct* strptr= (MyStuct*)&buffer[offset];
    

    then FlexC seems to enforce strptr to be longword aligned (treat it like offset were equal to 8 or 12) even though the P2 could perfectly access longwords at any byte address.

    @ersmith Is there any trick to avoid this? I mean, how can I assign an odd word address to strptr? strptr+= 2 doesn't work because it would increment by 2 times sizeof(MyStruct). (MyStruct)((void)strptr)+2 also doesn't work because casting void* back to MyStruct* enforces long alignment, again.

    memcpy() should work but is not very efficient. Maybe an inline assembler mov to fake the type cast?

  • @ManAtWork said:

    But what if the whole structure starts at an address not divisible by 4, for example because it is part of an ethernet packet and there are an odd number of words in the buffer before the struct?

    If I do something like

    uint8_t buffer[1024];
    int offset = 10;
    MyStruct* strptr= (MyStuct*)&buffer[offset];
    

    then FlexC seems to enforce strptr to be longword aligned (treat it like offset were equal to 8 or 12) even though the P2 could perfectly access longwords at any byte address.

    No, it doesn't require pointers to be longword aligned. When you assign an address to a pointer FlexC does no checking for alignment, even on P1 (where things actually have to be aligned to work). For example, here's a sample program that shows it isn't long aligned:

    #include <stdio.h>
    #include <stdint.h>
    
    typedef struct blah {
        uint32_t x; // 4 byte aligned
        uint8_t  c; // 1 byte aligned
    } MyStruct;
    
    uint8_t buffer[256];
    
    int testit() {
        int offset = 11;
        MyStruct *strptr = (MyStruct *)&buffer[offset];
        return strptr->c;
    }
    
    void main() {
        for (unsigned i = 0; i < sizeof(buffer); i++) {
            buffer[i] = i;
        }
        printf("testit() returns %d (should return 15)\n", testit());
    }
    

    @ersmith Is there any trick to avoid this? I mean, how can I assign an odd word address to strptr? strptr+= 2 doesn't work because it would increment by 2 times sizeof(MyStruct). (MyStruct)((void)strptr)+2 also doesn't work because casting void* back to MyStruct* enforces long alignment, again.

    OK, I think what you're getting confused with here is that the size of the structure is a multiple of 4 (that is, FlexC pads all structs out so that in arrays and such they will start on a longword boundary). If you really want the struct to be, say, 5 bytes long then instead of strptr += 2 you'll have to do something like strptr = (MyStruct *)(2*REAL_STRUCT_SIZE + (char*)strptr).

  • You're right, I was confused and now it works.

    The problem is when I spend the whole day debugging nasty mistakes in my C code I start to doubt everything. This leads to wrong assumptions and blaming of the wrong things or people. My fault.

    Thanks for your support.

  • Hi @ersmith, it's me. This issue is popping up, again. :#

    Last time I could solve it by re-arranging the fields of my structs. That was possible because it was my own protocol so I had the choice to modify it in any way I wanted. This time it's different. The protocol is defined in a standard and I'm bound to the existing definitions.

    typedef struct
    {
        uint8_t dest[6];
        uint8_t src[6];
        uint16_t ptype;     // protocol type EC_EtherType
        unsigned size : 12; // sum of datagram sizes
        unsigned htype: 4;  // header type
    } EC_FrameHeader;
    

    This is still one of the easiest. The total size is 16 bytes. The FlexC compiler places src at offset 8 instead of 6 but I can fix that by declaring and address-pair field of 12 bytes. But there are many more of those structures and most of them don't follow the alignment rules of FlexC.

    typedef struct
    {
        uint8_t  command;
        uint8_t  index;
        uint32_t address; // logical address or LSW=offset, MSW=slave
        unsigned length: 14;
        unsigned cirqul:  1; // circulated flag
        unsigned moreFol: 1; // more datagrams follow flag
        uint16_t events;     // event mask of all slaves ORed
    } EC_DatagramHeader;
    

    I have no idea for an easy fix for this at the moment. In Spin I can use DAT sections with any alignment I need but DAT uses fixed memory locations that then have to be copied to the destination buffer. That's not multitasking safe.
    :'(

  • ersmithersmith Posts: 5,900

    I never wanted to do packed structs because they would be such a pain to implement on P1. Doing a P2-only version probably wouldn't be too bad, but it would impact enough fiddly bits of the compiler that it will take a little while. I've just started a new job, so I don't know when I'll have cycles to get to this, sorry,

  • Don't worry. At the beginning I was very sceptical because I thought that this would make the code very inefficient and and would stop me from re-using existing code. It turned out that I do not port from other platforms anyway because it is to inefficient or I just don't like it. It means that I have to use some more shift or copy operations but it's not as bad as expected. We can do optimizations an clean up later.

Sign In or Register to comment.