True "HDMI" with data packets...
Wuerfel_21
Posts: 4,550
All of the ususal (?) "HDMI" drivers actually just generate DVI-spec pure video. It'd be nice to figure out how to insert audio packets, but for now I'm just trying to generate an AVInfoFrame packet to set the monitor into YCbCr mode, just because that's easier: It only needs one packet that always stays the same and the result should be immediately obvious when it starts working.
But problematically, it seems that I only have junky HDMI devices. One is extremely flakey and now doesn't want to do with tests that worked an hour ago and the other I suspect may actually just be a "dumb" DVI device (looks like the video guard band is intruding into the picture on the left edge...). Might have to try some more with the upstairs TV tomorrow.
Anyways, after futzing around with packet encoding and the poorly described error correction code (it's just CRC...) I think I'm doing at least something right. See also: https://web.archive.org/web/20190520020602/http://hamsterworks.co.nz/mediawiki/index.php/Minimal_HDMI
I'm creating this thread to maybe get some ideas tossing, but you may also try the attached code and see what it does for you. The intent is to display the picture with wrong colors (RGB re-interpreted as YCbCr), so that's what it should do. If the colors are normal, it's not parsing the Info Frame.
Comments
I was able to get some HDMI audio working a once while back with a P1V solution but there was still a bug and I had mixed results - only one cheapo LCD monitor tested produced sound as I recall and I needed to fix the bitstream output somewhere as my consumer A/V devices were being far stricter than the the budget monitor in what they would tolerate. When I counted cycles for core parts I know a P2 COG can certainly do it with the resources available, but don't really want to go there mainly due to HDMI licensing issues which is a shame really. It would be great if Parallax could obtain an HDMI license somehow and we could then get audio and other HDMI features on the P2.
The interesting part is wether I can manage to generate the audio packets (including resampling to a non-whacky rate when required) while simultaneously doing pixel repeat (for 320-wide lowres modes). In general that takes 6 cycles every 20 cycles, though with the streamer buffer you can probably fudge it a bit back and forth.
As far as licensing goes, the license sharks better come to my door and ask for it. Then I pull their human masks off or something and then they die immediately because they are sharks and need water to breathe. Sounds fake but is more real then trying to claim a forever license over particular de-facto universal transmission formats.
Am so very tempted to send you my P2 test code as I'm sure it would be extremely useful to you right now @Wuerfel_21 but don't think it would be a good idea. I can send you this github project link though which is what I learned from. A real treasure trove.
https://github.com/charcole/NeoGeoHDMI/tree/master
ps. in the issues list there you can see some of my comments/questions to Charlie the original creator when I was doing the FPGA stuff which mentioned some problems I encountered.
Would be extremely cool if we had DVI with Audio (just don't call it HDMI).
I've tried the test1 code on two TVs:
1) a little Phillips TV, shows nothing and mostly says no Signal.
2) a Hitachi TV. Shows Reimu but in normal colors (I think - how do I know? She has red clothes and sits on a black chair).
This TV says the picture is 642 x 471 pixels @ 60 Hz.
Andy
Also tried @Wuerfel_21 's test code on my Pioneer plasma over HDMI. No image synced at all. I was driving via a long 10m cable from another room, so to validate it does work from a P2, I ran my own DVI driver code over the same cable and got a 640x480 picture fine. But in terms of HDMI compliance I believe my HDTV is very strict so if something is wrong with the signal then it gets ignored completely.
Roger.
It can be achieved on a P2.
Yea, I think that's the trick. Not sure what they can really get you on if you avoid the trademarks (same as SD cards etc)
Not sure where it's getting 471 from, but 642 sounds like it's eating the guard band, too. Maybe the preamble needs to be exactly 8px? Idk, will need to go upstairs to the proper TV and fool around.
Btw, YUV mode should be totally obvious when it engages, something like this (maybe not exactly, I think the channels would be remapped differently)
Seems like I used 8 pixels in my own code.
And immediately I realize I had made a typo in the data guard character and of course everything starts working, even on the crappy little screen.
Yep, got an image now. Green background, red/pink clothing colours. Well done.
Yes, works now also here on both TVs
Okay, I think audio works. At least on the one proper TV that I can try it on. "soundtest.spin2" should give you a nice sine wave (and normal-colored image). (DO NOT COMPILE WITH DEBUG IT WILL BREAK THE TIMING)
(I like that the P2 can just pull up sine waves using CORDIC instructions)
I also included a modified version of the YUV test because the TV didn't like where the VSYNC edges were happening (I think????), so I think this is more compatible. The other files also have that change. There's also a "reclocktest" that sends just the audio clock packet. It only exists to dump the packet out over debug so I could paste it in the other file.
Working on small Lenco screen.
But: The sine was low frequency, so did not passthrough the small speakers.
With headset, I get it.
So perhaps for testing generate a 1Khz
Nice work !
That probably is a good idea, here's a version with a 1 kHz tone.
Yes, loud and clear now on small speakers !
This work on all my HDMI monitors, right colors and sound
Keep going
Well, the next step is figuring out how to fit it into the framework of how P2 audio normally works, which is a cog producing samples at some interval. I forget, can I set a pin that's being used as an output into P_REPOSITORY mode? Could use that as a mailbox for samples, the video cog just needs to check it somewhat frequently. The other thing is that the audio needs to be a normal rate (32k, 44.1k or 48k), so I'll have to figure out a good-enough polyphase resampling algorithm (the yamaha FM emulator cogs use widely off-spec rates around 55kHz). Wonder if 4-point sinc is sufficient?
Though if there's time for that, you can probably also generate simple audio directly in the video cog.
I also am not yet taking advantage of a pretty big optimization: When using normal audio rates (32kHz to 48kHz) and VGA/EDTV Hsync rates (>= 31kHz), there are at most 2 samples in a scanline, which means that the second two subpackets are never used and can be hardcoded to zero, which could speed up packet encoding quite a lot.
This looks awesome! Having sound over HDMI would be very nice!
Same problem I encountered with my Pioneer plasma with my own code. Audio didn't work for me.
It's very picky. This TV does support LPCM 32,44.1,48 kHz rates though according to the E-EDID report.
I'll need to look into your code but are you computing the proper channel status stuff as is in S/PDIF framing? I couldn't see it there at first glance. I think that may be required.
Yes this needs to be sorted. Would be very neat if you could put a pin into REPO mode and it'd still honour the streamer and bitDAC settings as an output...that needs to be tested. If so, you'd have up to 8 HDMI pins that could potentially act as repos for separate multichannel sound samples which would be very cool indeed, assuming there is enough processing time to encode them all. At this point however stereo is probably hard enough, and good enough.
I have the status word stuff, but not a lot of bits set in that:
incmod spdif_phase,#191 wc if_c alts tmp3,#20 if_c bith packet_header,#0-0 cmp spdif_phase,#32 wc if_c testb spdif_status,spdif_phase wc muxc tmp2,#%1100_1100 ' also set parity here [...] '|||| Sample rate here | Disable copy protection spdif_status long %00_00_0000_00000000_00000000_00000100Other thing I'm not doing is sending an audio infoframe. Apparently spec requires sending it.
Spent the rest of the evening writing up a utility to easily generate canned packets like that. Can probably add it in tomorrow. Though that's 3 packets now, Audio clock, AVI Info and audio info. That's 384 bytes when canned.
Yeah I sent these three packets. You can alternate two of them in a pair of frames but the timing one should go out much more frequently.
That github link I provided above used this as channel status for 32kHz LPCM. I was never able to modify it successfully for 44.1 or 48kHz even in the FPGA variant so I was sure I was doing something wrong, even though I couldn't pinpoint it.
localparam [191:0] channelStatus = 192'hc203004004; // 32KHz 16-bit LPCM audio
In my P2 version which never had working audio on my Pioneer TV but had some success on a cheapo monitor, the long term plan to solve this was to capture the TMDS differences between my (working) FPGA version @32kHz and the P2 version to narrow down what bit was wrong, but it was a lot of work to do and I sort of moved on to other things after that. As I recall I did make a tool that could capture the streamed TMDS code output directly from the P2 pins though back into HUB RAM using a separate COG which I could export for analysis later. I'll have to look for that.
Here's the decoder I wrote. I think I just had it analyze the captured 8 bit bytes at the HDMI pins on the P2 but can't recall everything it did as it was a while ago. You might be able to make use of it perhaps. YMMV.
I'll see if I can find the other PASM part that streamed the data back from the pins (might be lost, not sure), but that's probably something simple you'd be able to add to your own code if it's just a byte logger.
One thing I see is that the FPGA code you linked sends the clock regeneration packet more frequently. That might be it, allegedly it should be sent once per N period. Idk, you'd have to test that, it all just works for me.
Here's the capture/logging component which firstly spawned my HDMI cog plus the snippet from this HDMI cog that syncs to it so I can capture a line buffer for a given scanline for later analysis. This was written in the early days before tools had evolved and uses hard code addresses etc. Hopefully it still compiles, but you get the gist.
'************************************************** '* HDMI + AUDIO demo for P2-EVAL rev B * '************************************************** CON HDMI_BASE = 24 ' base P2 pin for HDMI (must be multiple of 8) HDMI_GROUP = (HDMI_BASE >> 3) FRAMEBUF_START = $2000 ' frame buffer image data (5:6:5 format) FRAMEBUF_SIZE = 640*480*2 ' size of image data in bytes AUDIO_FIFOSTART = $1000 ' start address of audio FIFO AUDIO_FIFOSIZE = 64 ' size of audio FIFO in longs FIFO_STATUS = $1800 ' address where audio FIFO status is written FRAME_STATUS = $1304 ' address where HDMI frame status is written HDMI_DATA_BUF = $2000 ' address where HDMI island pkt data is written BAUDRATE = 2000000 '115200 BITCLOCKS = PROP_FREQ / BAUDRATE HDMI_CAPTURE_LINE_ADDR = $1200 DAT org startup ' ----------------------------------------------------------------- ' configure PLL, 20MHz / 2 * 27 * 1 = 270MHz ' ----------------------------------------------------------------- hubset ##%1_000001_0000011010_1111_10_00 'select PLL parameters waitx ##25_000_000 / 100 'stable after 10ms hubset ##%1_000001_0000011010_1111_10_11 'switch to PLL ' OR .... {{ ' ----------------------------------------------------------------- ' configure PLL, 20MHz / 10 * 126 * 1 = 252MHz ' ----------------------------------------------------------------- hubset ##%1_001001_0001111101_1111_10_00 'select PLL parameters waitx ##25_000_000 / 100 'stable after 10ms hubset ##%1_001001_0001111101_1111_10_11 'switch to PLL }} {{ ' ----------------------------------------------------------------- 'configure DEBUG clock for 11.9 MHz operation ' ----------------------------------------------------------------- hubset ##%1_010011_0001110110_0100_10_00 'select PLL parameters waitx ##25_000_000 / 100 'stable after 10ms hubset ##%1_010011_0001110110_0100_10_11 'switch to PLL }} ' reverse image bitmap data mov bmploop, ##640*350*2 ' number of bytes in image mov addr1, ##FRAMEBUF_START ' address of image mov addr2, addr1 add addr2, bmploop sub addr2, #2 shr bmploop, #2 ' divide by 4 to get number of words to swap rep #6, bmploop ' reverse bitmap pixels to flip image rdword pix1, addr1 rdword pix2, addr2 wrword pix1, addr2 wrword pix2, addr1 add addr1, #2 sub addr2, #2 mov i, #0 wrlong i, ##HDMI_CAPTURE_LINE_ADDR ' restart as the HDMI COG setq paramaddr ' addr of COG parameter block is passed coginit #1, ##@hdmicog ' restart COG to begin running HDMI 'setup smartpin async tx serial dirl #62 wrpin ##%_01_11110_0, #62 wxpin uartclocks, #62 drvh #62 'setup smartpin async rx serial dirl #63 wrpin ##%_00_11111_0, #63 wxpin uartclocks, #63 dirh #63 dirl #6 wrpin ##%0111_0000_000_0000000000000_00_01110_0, #6 wxpin ##PROP_FREQ, #6 wypin #0, #6 dirh #6 call #delay main callpa promptmsg, #puts call #readline cmp linesize, #0 wz if_z jmp #main2 if_z jmp #main callpb #32, #putch callpa linesize, #printbyte call #crlf call #puts_lut call #crlf call #parsehex callpa val, #printlong call #crlf mov ptra, val callpa ptra, #capture mov ptra, ##HDMI_DATA_BUF callpa ##128, #dumphex jmp #main main2 mov ptra, ##HDMI_DATA_BUF callpa ##60000, #dumphex jmp #main getcount testp #6 wz if_z rdpin val, #6 if_z callpa val, #printlong jmp #main delay waitx ##150_000_000 ret readline mov linesize, #0 readloop call #getchar cmp char, #13 wz ' detect return key for ending the line if_z mov char, #0 ' transform to a null for end of string if_z wrlut char, linesize ' and save in buffer if_z ret cmp char, #8 wz ' support backspace if_nz cmp char, #127 wz ' and delete keys if_z mov char, #8 if_z callpb char, #putch if_z callpb #32, #putch ' rubout previous character if_z sub linesize, #1 wc if_z_and_c mov linesize, #0 ' don't delete past start of line if_z_and_nc callpb char, #putch if_z jmp #readloop cmp linesize, #79 wc ' check for 80 char line limit if_c cmpr char, #32 wc ' check for printable chars if_c wrlut char, linesize ' store char in LUT if_c add linesize, #1 if_c callpb char, #putch ' echo character jmp #readloop crlf callpb #10, #putch callpb #13, #putch ret puts_lut mov ptrb, #0 putsloop rdlut char, ptrb add ptrb, #1 test char wz if_z ret if_nz callpb char, #putch jmp #putsloop putch rqpin inb, #62 wc testp #62 wz if_nc_or_z wypin pb, #62 if_nc_or_z ret jmp #putch puts mov ptra, pa putloop rdbyte char, ptra++ test char wz if_z ret if_nz callpb char, #putch jmp #putloop getchar testp #63 wz if_z rdpin char, #63 if_z shr char, #32-8 if_nz jmp #getchar ret printlong mov l, pa rol l, #8 getbyte b, l, #0 callpa b, #printbyte rol l, #8 getbyte b, l, #0 callpa b, #printbyte rol l, #8 getbyte b, l, #0 callpa b, #printbyte rol l, #8 getbyte b, l, #0 callpa b, #printbyte ret printword getbyte b, w, #1 callpa b, #printbyte getbyte b, w, #0 callpa b, #printbyte ret printbyte getnib nib, pa, #1 cmp nib, #10 wc if_nc add nib, #65-10 if_c add nib, #48 callpb nib, #putch getnib nib, pa, #0 cmp nib, #10 wc if_nc add nib, #65-10 if_c add nib, #48 callpb nib, #putch ret parsehex mov ptrb, #0 mov val, #0 prshexloop rdlut b, ptrb wz add ptrb, #1 if_z ret cmp b, #96 wc if_nc sub b, #32 'convert to lower case cmpsub b, #48 wc 'subtract ASCII "0" if_nc ret 'exit if it was less than "0" cmp b, #10 wc 'check if a digit from 0-9 if_c rolnib val, b, #0 'and if so, use this nibble if_c jmp #prshexloop 'continue cmp b, #17 wc 'check if < "A" if_c ret sub b, #7 'bring back into 10-15 range cmp b, #16 wc if_c rolnib val, b, #0 'if A-F use this digit and... if_c jmp #prshexloop ' ...continue ret dumphex mov i, pa mov j, ptra and j, #15 dump1 callpa ptra, #printlong callpb #$3A, #putch callpb #32, #putch dump2 rdbyte char, ptra++ callpa char, #printbyte callpb #32, #putch sub i, #1 wz if_z ret incmod j, #15 wc if_c call #crlf if_c jmp #dump1 jmp #dump2 val long 0 bmploop long 0 addr1 long 0 addr2 long 0 pix1 long 0 pix2 long 0 paramaddr long hdmiparams uartclocks long (BITCLOCKS << 16) | 7 char long 0 i long 0 j long 0 l long 0 w long 0 nib long 0 promptmsg long prompt linesize long 0 capture ' rdlong pa,ptra wz 'wait for start command ' if_z jmp #capture_loop wrlong pa, ##HDMI_CAPTURE_LINE_ADDR waitatn setxfrq ##$8000_0000 'sysclk sample rate wrfast #0, ##FRAMEBUF_START xinit xcfg, #0 waitxfi 'wait for completion ret { mov xcfg2,#sample_bytes encod xcfg2 add xcfg2,#%1101 mov cfg,xcfg ' setnib cfg,xcfg2,#7 mov xcfg2,#base_pin shr xcfg2,#3 setnib cfg,xcfg2,#5 mov rep_count,##sample_length shr rep_count,#15 wz if_z jmp #single_pass mov cfgm,cfg setword cfgm,##$8000,#0 rep @multi_pass,rep_count xinit cfgm,cfg2 multi_pass } 'cogatn #|< maincog ' wrlong #0,ptra 'capture done ' drvh #16 'jmp #capture_loop xcfg long $e0b6_0000 | 60000 '8 bit capture ' ----------------------------------------------------------------------------- orgh prompt byte 13, 10, "ADDR> ", 0 hdmiparams long HDMI_GROUP ' which group of 8 pins is used by HDMI long FRAMEBUF_START ' addr of video frame buffer long FRAMEBUF_SIZE ' size of frame buffer long AUDIO_FIFOSTART ' audio FIFO start address long AUDIO_FIFOSIZE ' size of audio FIFO long FIFO_STATUS ' addr where audio FIFO pointer is updated long FRAME_STATUS ' addr where frame status will be written long HDMI_DATA_BUF ' addr where HDMI island data is written long TOKENS_PER_LINE ' number of audio tokens given per scanline long TOKENS_PER_SAMPLE ' number of tokens needed per sample '************************************************** '* HDMI + AUDIO 16bpp & 5:6:5 RGB framebuffer COG * '************************************************** CON ' ----------------------------------------------------------------------------- ' 720x480p60 resolution timing ' ----------------------------------------------------------------------------- <SNIP - sorry > ' Subroutines ' ' ----------------------------------------------------------------------------- ' subroutine to send a group of blank lines in the vertical blanking interval ' ----------------------------------------------------------------------------- do_blanks call #do_hsync 'generate horizontal sync for the line xcont m_pre, syncs 'send some dummy preamble sized data xcont m_guard, syncs 'send some dummy guard band sized data xcont m_blank, syncs 'do the blank portion of the line call #create_island 'create the island while we have time add scanline, #1 'increment the scanline number _ret_ djnz pa, #do_blanks 'loop ' ----------------------------------------------------------------------------- ' subroutine to send the horizontal blanking and hsync region of each scanline ' ----------------------------------------------------------------------------- do_hsync rdlong x, ##HDMI_CAPTURE_LINE_ADDR wz if_z jmp #skip3 cmp x, scanline wz if_z mov x, #0 if_z wrlong x, ##HDMI_CAPTURE_LINE_ADDR wz if_z cogatn #1 skip3 xcont m_fp_b4_pre, syncs 'horizontal front porch before preamble xcont m_pre, preamble_d 'send data preamble drvl #57 'line pulse for DEBUG xzero m_guard, guard_l 'send data guard band <SNIP - sorry>ps. by the way my code didn't use canned packets, it dynamically encoded them.
Yeah I found this comment in my own code...
' ----------------------------------------------------------------- ' we want an audio clock regeneration packet sent every 32 samples ' ----------------------------------------------------------------- incmod samplecnt, #31 wc 'check for another 32 samples produced if_c mov clkregen, #1 'and set the flag for the next islandIf you're doing all that and none of it makes it work, the solution is probably somewhere else. Or your TV might just suck.
TV most certainly doesn't suck. Kicks booty which is why I'm still using it. However I think it must be following the standard to the letter and is probably HDMI 1.0 only or something too. Normal A/V gear works fine with it.
Update: just checked, it's actually HDMI1.3 and HDCP1.1. So old yeah, but not 1.0 old.
Well, if you ever figure it out, I can add the fix, but I'd rather focus on getting something going first. If the other guy is to be believed, it works more often than it doesn't as-is.
Just tested the examples on a cheap FireTV (that I despise) and it works.
Also works with my main 65" OLED TV and also a DLP projector