Shop OBEX P1 Docs P2 Docs Learn Events
Test Coverage Matrix or Summary? — Parallax Forums

Test Coverage Matrix or Summary?

marsman2020marsman2020 Posts: 71
in Propeller 2
Hopefully there will be ~100 new P2-Prototype boards going out to people soon.

Is there some kind of a list or matrix somewhere of what has been tested, and what is left to be tested? I believe there were some Google docs where people could collaborate when the FPGA images were being tested, but I've only seen forum threads on P2 prototypes.

If I get a board I'd like to be able to target my testing on the right areas to make sure we cover everything as a group before the respin happens.

Comments

  • TubularTubular Posts: 4,702
    This seems like a nobel cause, marsman2020. I don't have a lot of experience with current (<10 year old) testing methodologies, but I'm sure many here do. Apart from doing testing for a living, I wonder if anyone does it for fun?

    Do you have a sample or link to the thing you have in mind?

    Fwiw, a heap of testing has been completed and is often reported in the forums. Its not formal testing, not all of it will be on the the current incarnation but thats not to say its not useful. It could certainly start to fill out a test matrix.
  • marsman2020marsman2020 Posts: 71
    For the things I do in my day job (mechanical engineering) you'd have a verification and validation matrix with a list of requirements and state if you were verifying each requirement via analysis, test, similarity to heritage hardware, etc.

    For this effort, I was thinking more along the lines of a list of instructions that have been verified to work properly, and expected higher level features/functionality that need to be tested. For example the many analog pin modes. Maybe a column for people to put in their name and say they intend to test something and approximately when.

    If I get my hands on one of the early boards, I'd like to make sure that I can help check off as many boxes as possible.
  • jmgjmg Posts: 15,173
    marsman2020 wrote: »
    For this effort, I was thinking more along the lines of a list of instructions that have been verified to work properly..

    There was an opcode validation suite, that ran on FPGA and was run to give identical outcomes on Silicon.
    Instructions have been exercised for many months on FPGA.
    marsman2020 wrote: »
    ... and expected higher level features/functionality that need to be tested. For example the many analog pin modes.

    Analog pins have been confirmed as functional, as have DACs.
    However, the definition of all ADC characteristics is still somewhat open - but given that is a custom block, and hard to modify, the final silicon will be what it is in those areas.

    marsman2020 wrote: »
    For the things I do in my day job (mechanical engineering) you'd have a verification and validation matrix with a list of requirements and state if you were verifying each requirement via analysis, test, similarity to heritage hardware, etc.

    There is also simple 'miles on the clock' testing, aka making things work, which can uncover things not on any 'validation matrix', usually with a 'that's strange' moment..
    See the VGA thread for one example, and the interactions between SD and FLASH for another.
  • Ym2413aYm2413a Posts: 630
    My plan was to just jump in and when I found issues, report them here on the Forum. But if someone wanted to populate a matrix. I'd check off the things that related to what I was doing.

    Mostly DSP stuff, Analogue sampling, DAC out, timers and math operations.

Sign In or Register to comment.