Could there be (or is there already) a graphical front end menu that could launch any of the MegaYume and/or NeoYume games?
@pik33 has a menu that can launch P2 applications (and he just built individual emulator binaries hardcoded to launch a respective game). Making one that can launch emulated games "properly" should be easy, but there's two tricky parts: The first being to launch the emulator with some sort of parameter passed. I guess they can be passed in some format through the upper 16K (this would make this not work with debug builds, but it seems like the least hassle). Another potential tripping point with that is that I don't think flexspin can retrieve the 8.3 name for a file when LFS is enabled (as it is in pik's menu loader).
This is a simple binary loader. The selected binary is loaded into the PSRAM, then a dedicated cog other than #0 stops all the other cogs except the PSRAM driver (there is no HUB code executing after this), moves the binary to the HUB RAM, stops the PSRAM driver, coginit #0,#0 and stops itsef. As the NeoYume can be compiled with a predefined game to load, the MIcroDOS menu simply loads the proper binary.
After the binary is loaded, the loader no longer exists in the P2 memory, so its LFN file system has no effect on the loaded binary.
@pik33 said:
After the binary is loaded, the loader no longer exists in the P2 memory, so its LFN file system has no effect on the loaded binary.
Yea, but if you passed a file path into the loaded binary, it would matter, since it wouldn't be able to open /sd/longdirectory/longfilename.bin. You'd have to give it /sd/longdi~1/longfi~1.bin. I am looking into what it takes to convert LFN paths to short paths, should be straightforward.
@Wuerfel_21 said:
MegaYume can quit to it's own menu (Ctrl+Esc on Keyboard or holding Start+Down for 5 seconds on the controller) and this is all easily controlled from the high-level code, you can change it to reset the P2 instead (launching another binary is sadly not a thing flexspin VFS can really do). NeoYume is more prickly because the menu code and associated C runtime is kicked out of memory as soon as the game launches. If I refactor the Z80 to use the memory arbiter for high ROM (which would also fix Metal Slug 3's audio) I might(?) be able to cut free some 64k. If I could then reduce the menu code to that size (help @ersmith the nu backend is still busted), I could have menu quit in there, too.
The MegaYume upper code at least starts up and can print characters now (I fixed several issues in the nucode runtime that were blocking that). I don't have external memory to test with, so not sure if any games can actually start up.
@Wuerfel_21 said:
MegaYume can quit to it's own menu (Ctrl+Esc on Keyboard or holding Start+Down for 5 seconds on the controller) and this is all easily controlled from the high-level code, you can change it to reset the P2 instead (launching another binary is sadly not a thing flexspin VFS can really do). NeoYume is more prickly because the menu code and associated C runtime is kicked out of memory as soon as the game launches. If I refactor the Z80 to use the memory arbiter for high ROM (which would also fix Metal Slug 3's audio) I might(?) be able to cut free some 64k. If I could then reduce the menu code to that size (help @ersmith the nu backend is still busted), I could have menu quit in there, too.
The MegaYume upper code at least starts up and can print characters now (I fixed several issues in the nucode runtime that were blocking that). I don't have external memory to test with, so not sure if any games can actually start up.
Thanks very much for looking into those cases directly.
Still very obviously broken though. MegaYume's file chooser is very broken, which you should be able to see without any memory board (though @Rayman may be happy to hook you up with one of those 96MB ones). Just have to press a key on the keyboard (or comment out the code that wait for it after printing the opening screen.
Meanwhile NeoYume just does this:
More importantly, it seems that you broke the ASM backend!
Further test: hardcoding games with DIRECT_BOOT in config doesn't work for either emulator. Seems that the file library is properly broken.
Oh, @ersmith, it seems going up to the root directory (where the sd mount point lives) in megayume's menu crashes now (could be older bug, I don't normally do this), both on nu and asm backends. The Version 6.0.0-beta-v5.9.28-47-g18bd553b Compiled on: Feb 14 2023 I have installed on my PATH doesn't have this bug. Might bisect later.
Yep root directory crash is recent bug, introduced somewhere between where native output got broken (I think 149c4dbbd46f7f2b266caaa79c69af6b3b7ae6e8) and where it got fixed ([61e59818fc05b26c7a89ebc769f2bd8d5bcb81e3] fix problem in native code with labels and -H)
@Rayman : Thanks for the kind offer. I hope that this bug, at least, we can resolve without extra hardware, but I'd love to take a look at external RAM at some point.
@Wuerfel_21 : I'm not completely sure this is a flexspin bug, although it probably is. The C library does sometimes need locks internally, and it looks like MegaYume is grabbing all of them; why is that?
@ersmith said:
@Wuerfel_21 : I'm not completely sure this is a flexspin bug, although it probably is. The C library does sometimes need locks internally, and it looks like MegaYume is grabbing all of them; why is that?
The lower code needs certain locks with hardcoded numbers, so I made it allocate all locks and then return the ones that aren't needed. MegaYume uses locks 7 through 10. Mostly for their event-signalling ability. There is no special significance to the numbers used, I just picked them to be in the middle to avoid conflicts (debugger uses 15 - allocated through a similar method)
@ermsith e9dc5570a083242a30cec48775208c67fd130b09 has working USB again. Also no more root directory issues. MegaYume on Nu can load and execute games, but the filechooser is constantly refreshing for some reason. Also the version info at the top is missing.
NeoYume on nu is very broken still and gets stuck when trying to load anything.
I think something is going wrong with sprintf calls in both cases.
@Wuerfel_21 : thanks for the feedback. Varargs functions were broken in nucode when called from Spin (they need special handling which was happening in CheckTypes, but that function doesn't happen in Spin). It should be fixed now.
@ersmith said:
@Wuerfel_21 : thanks for the feedback. Varargs functions were broken in nucode when called from Spin (they need special handling which was happening in CheckTypes, but that function doesn't happen in Spin). It should be fixed now.
Very cool and good. MegaYume works now. NeoYume fails to build with neoyume_upper.spin2:617: error: incompatible types in parameter passing: expected const any unknown type but got string pointer to const byte.
Line in question is c.sprintf(tmpstr,@"WAIT=%2d DELAY=%2d CLK=%-5s DAT=%-5s",PSRAM_WAIT,PSRAM_DELAY,PSRAM_SYNC_CLOCK?@"SYNC":@"ASYNC",PSRAM_SYNC_DATA?@"SYNC":@"ASYNC")
Also, can we make sure that if the build errs, the output binary is deleted?
@Wuerfel_21 : sprintf should be working now even for NeoYume (the type checking code was messed up, it should be fixed). As for output binary being deleted, it should be doing that already, at least it did in the samples I tried.
@ersmith actually, scratch that, was using the wrong script. Owie. sprintf is fixed, but loading is still broken in NeoYume (seems to commonly hang on LOAD_SROM or randomly crash...).
Ok, the time has come, I will look into USB multiplayer now (and perhaps related issues such as fixing hotplug detection(?) and increasing poll rate). By which I mean tomorrow, because it's late already. But I did a quick readthrough of @macca 's hub driver and oh god the spaghett got even worse. (And there's still a bunch of RevA silicon bug workarounds in there!!!). I don't think it has support for any device other than gamepads on the hub? Also the mouse support got lost along the way? Might make a more general release of a yet more improved driver.
Not sure how I'll handle button mapping. macca wrote a nice HID descriptor parser, but that on its own is mostly worthless since there's no consistent mapping of button numbers to button positions (though we can make an educated assumption about the directional inputs). Though I guess most 4 button controllers do follow a layout similar to these old speedlink generic pads and 9/10 as select/start is semi-standard, so maybe that's an ok default. (I'd still obviously have overrides for individual VID/PID. Maybe make it load these from a file?)
Then again, mapping real buttons to virtual ones is an inexact science to begin with. How do you map a normal 4-button layout to a SEGA controller?
Also, just to mess with us, there's multiple layouts of NeoGeo controller...
Which are both inadequate at matching the 4-buttons-in-a-row joystick layout (imagine pressing B and C simultaneously. This is infact needed for some moves in The Last Blade)
Actually, I realized that it just uses POLLCT3 to figure out when it's time to poll the devices. Could probably use a scanline trigger from the video driver instead, which should give the most consistent response time possible with the lowest bus usage possible. Would need to determine how long the polling process with four controllers actually takes, so as to trigger it far enough up that the new data always arrives ahead of the VBlank interrupt. Maybe trigger it twice per frame, which would be more similar to the current 125 Hz rate and avoid scenarios where weird poll timings in the game code would incur a full whopping 16ms of lag.
To get around the various USB controller button differences maybe you could have a setup screen that lets people press buttons for different directions etc and you could scan the reports and figure out which report bit fields change and associate them accordingly. I think that is generally what emulator environments like MAME do, right?
I expect this is quite a bit more high level software to develop though and ideally it would also need state saving somewhere to avoid doing this each time you run the emulator. But some of that code could be reusable in other future projects at least. It would be a handy component.
By the way it's great that you are looking into this host stuff using Macca's code as a starting point... Ideally we could eventually get a keyboard, mouse and a couple of USB controllers' inputs all from a single COG. You'd just need a cheap 4 port USB hub to attach to a P2 then.
@Wuerfel_21 said:
Ok, the time has come, I will look into USB multiplayer now (and perhaps related issues such as fixing hotplug detection(?) and increasing poll rate). By which I mean tomorrow, because it's late already. But I did a quick readthrough of @macca 's hub driver and oh god the spaghett got even worse. (And there's still a bunch of RevA silicon bug workarounds in there!!!). I don't think it has support for any device other than gamepads on the hub? Also the mouse support got lost along the way? Might make a more general release of a yet more improved driver.
It's been a long time since I last reviewed the code, however it should support both low and high speed devices on the hub, including keyboards. Mouse is not supported, but I think it can be eaisly added.
Hot-plug is supported however it may happen that something breaks and devices stops responding. It should work when all devices are connected at startup but I found some problems even in that case.
@pik33 said:
I hope I can connect a RPi dedicated keyboard. It has a hub inside and 3 USB ports.
Yeah that'd be quite a neat and tidy setup for a small P2 emulation/game system with the inbuilt hub. Good idea. Hopefully its USB hub should be compatible in the end...
The problems with having a control configuration GUI in the emulator itself are:
You can mess up the configuration and lock yourself out.
Needs additional interfaces to the USB driver to see raw pad data and reload mappings (problematic in NeoYume where input code is over in a parallel universe).
To save configuration, full read-write FS is neccessary (currently disabled in NeoYume, though for no particular reason).
The "press buttons in order" method of configuration doesn't really have a way to not map a button (if your controller physically doesn't have enough to map all the optional ones)
Sometimes you also want two physical buttons to map to the same virtual button (as in how the SEGA-style controllers are mapped in NeoYume currently)
By making it just load configs from a file I can make it someone else's problem, as @pik33 suggested.
Interesting first results just playing with the code:
StrikeFX gamepad is now detected in PS3 mode? It works and all the buttons are there, but now I can't use it as an XInput tester because it locks out the mode switch. I didn't even know it could do that, lol.
Similarly, the retro-bit controllers refuse to switch into XInput mode via button combination
Xinput does work with the RetroReceiver though (really need to get a proper Xbox360 pad for testing)
The XInput translation doesn't put the buttons in NESW order (easy to fix though!)
HID controller only decodes properly on port 1 (unless you plug it into port 1 and then back into another - descriptor buffer issue?)
I think there's some sort of issue with axis normalization?
Regular keyboard does work on a hub port, but my beloved PS/2 adapter causes it to die...
It seems to not re-init the hub after an error unless I manually re-plug it.
EDIT figured out the issue with the HID device decoding. Below .hub_devices there's a copy-paste error. PR submitted.
Other weird quandary: On hubs with a hardwired device (like the RPi keyboard @pik33 mentioned), it might be hogging port 1, which would prevent connecting a game controller for player 1. Dynamically assigning player ids to connected devices in order would also kinda suck I guess, but not as much? (Because maybe you want one player on the keyboard!).
If anyone has such a device laying around, perhaps investigate how it's wired. On windows, open device manager, set view to "Devices by connection" and find the keyboard. Open the device one level up (or two levels if you have a weirdo board that shows up as a composite device - basically the direct child of the hub device) from the actual keyboard endpoint. It should have a location like Port_#0003.Hub_#0003. The first number is the port number (1-based).
Shitpost status: flexspin can no longer compile DEBUG if due to conditional assembly, which I was trying to apply to making it easy to enable/disable the activity LED (which I always used to rip out of the USB code because the blinking is obnoxious), so I updated my main flexspin install and now the whole thing doesn't build.
Aaand I just noticed that it now suffers from having horrible update rate with the neo geo mini controller. I could've sworn it didn't do this yesterday, maybe I was just thick.
Eitherhow, the previous Spin iteration of @macca 's HID decoder had the same issue.
WTF? Going back to the original code without any of my edits, the poll rate is bugged, too. It turns out yesterday I was using Propeller Tool, where this bug does not happen? (Again, only affects this one particular gamepad).
Why does an ASM-only program behave differently on flexspin? aaaaa I'm feeling it.
EDIT some more: trying to compare binaries is fruitless due to flexspin and proptool having different opinions on when to use relative/absolute addressing modes. Classy.
EDIT 3: Issue has nothing to do with HID stuff, the flexspin version for some reason gets a lot of NAK responses? WHY?
EDIT 4: Things tried fruitlessly: loading proptool binary with loadp2 (still works), changing clkfreq (no difference), inserting waits at startup (no difference)
Okay, too much brainhurt for now. Am away over the weekend and hoped to make some inroads before that. No luck.
I did order some parts to hopefully build a crappy USB interception device (read: some breakouts that I will use to tap my bootleg logic analyzer into the data lines), those should be there by monday, too. oh well.
EDIT: Thinking about it, maybe it'd be slightly smarter to port the hub support portions of the code to the older spin driver instead?
Comments
This is a simple binary loader. The selected binary is loaded into the PSRAM, then a dedicated cog other than #0 stops all the other cogs except the PSRAM driver (there is no HUB code executing after this), moves the binary to the HUB RAM, stops the PSRAM driver, coginit #0,#0 and stops itsef. As the NeoYume can be compiled with a predefined game to load, the MIcroDOS menu simply loads the proper binary.
After the binary is loaded, the loader no longer exists in the P2 memory, so its LFN file system has no effect on the loaded binary.
Yea, but if you passed a file path into the loaded binary, it would matter, since it wouldn't be able to open
/sd/longdirectory/longfilename.bin
. You'd have to give it/sd/longdi~1/longfi~1.bin
. I am looking into what it takes to convert LFN paths to short paths, should be straightforward.Okay, here is a function to that extent...
In C because it has to be, spin can't access structs.
The MegaYume upper code at least starts up and can print characters now (I fixed several issues in the nucode runtime that were blocking that). I don't have external memory to test with, so not sure if any games can actually start up.
Thanks very much for looking into those cases directly.
Still very obviously broken though. MegaYume's file chooser is very broken, which you should be able to see without any memory board (though @Rayman may be happy to hook you up with one of those 96MB ones). Just have to press a key on the keyboard (or comment out the code that wait for it after printing the opening screen.
Meanwhile NeoYume just does this:
More importantly, it seems that you broke the ASM backend!
Further test: hardcoding games with DIRECT_BOOT in config doesn't work for either emulator. Seems that the file library is properly broken.
@ersmith would be happy to send you a care package
Oh, @ersmith, it seems going up to the root directory (where the
sd
mount point lives) in megayume's menu crashes now (could be older bug, I don't normally do this), both on nu and asm backends. TheVersion 6.0.0-beta-v5.9.28-47-g18bd553b Compiled on: Feb 14 2023
I have installed on my PATH doesn't have this bug. Might bisect later.Yep root directory crash is recent bug, introduced somewhere between where native output got broken (I think
149c4dbbd46f7f2b266caaa79c69af6b3b7ae6e8
) and where it got fixed ([61e59818fc05b26c7a89ebc769f2bd8d5bcb81e3] fix problem in native code with labels and -H
)@Rayman : Thanks for the kind offer. I hope that this bug, at least, we can resolve without extra hardware, but I'd love to take a look at external RAM at some point.
@Wuerfel_21 : I'm not completely sure this is a flexspin bug, although it probably is. The C library does sometimes need locks internally, and it looks like MegaYume is grabbing all of them; why is that?
The lower code needs certain locks with hardcoded numbers, so I made it allocate all locks and then return the ones that aren't needed. MegaYume uses locks 7 through 10. Mostly for their event-signalling ability. There is no special significance to the numbers used, I just picked them to be in the middle to avoid conflicts (debugger uses 15 - allocated through a similar method)
@ermsith Just tried bleeding edge
3c567bf7f7ab92cf22b2145355f91b30fac26261
, USB is now broken (both ASM and nu)@ermsith
e9dc5570a083242a30cec48775208c67fd130b09
has working USB again. Also no more root directory issues. MegaYume on Nu can load and execute games, but the filechooser is constantly refreshing for some reason. Also the version info at the top is missing.NeoYume on nu is very broken still and gets stuck when trying to load anything.
I think something is going wrong with
sprintf
calls in both cases.@Wuerfel_21 : thanks for the feedback. Varargs functions were broken in nucode when called from Spin (they need special handling which was happening in CheckTypes, but that function doesn't happen in Spin). It should be fixed now.
Very cool and good. MegaYume works now. NeoYume fails to build with
neoyume_upper.spin2:617: error: incompatible types in parameter passing: expected const any unknown type but got string pointer to const byte
.Line in question is
c.sprintf(tmpstr,@"WAIT=%2d DELAY=%2d CLK=%-5s DAT=%-5s",PSRAM_WAIT,PSRAM_DELAY,PSRAM_SYNC_CLOCK?@"SYNC":@"ASYNC",PSRAM_SYNC_DATA?@"SYNC":@"ASYNC")
Also, can we make sure that if the build errs, the output binary is deleted?
@Wuerfel_21 : sprintf should be working now even for NeoYume (the type checking code was messed up, it should be fixed). As for output binary being deleted, it should be doing that already, at least it did in the samples I tried.
Yea, it's all working now. Load speeds don't seem to be significantly worse, which is also cool.
And no, there's plenty of cases where the output file isn't deleted on error. The sprintf type thing was one of those cases.
@ersmith actually, scratch that, was using the wrong script. Owie. sprintf is fixed, but loading is still broken in NeoYume (seems to commonly hang on LOAD_SROM or randomly crash...).
Ok, the time has come, I will look into USB multiplayer now (and perhaps related issues such as fixing hotplug detection(?) and increasing poll rate). By which I mean tomorrow, because it's late already. But I did a quick readthrough of @macca 's hub driver and oh god the spaghett got even worse. (And there's still a bunch of RevA silicon bug workarounds in there!!!). I don't think it has support for any device other than gamepads on the hub? Also the mouse support got lost along the way? Might make a more general release of a yet more improved driver.
Not sure how I'll handle button mapping. macca wrote a nice HID descriptor parser, but that on its own is mostly worthless since there's no consistent mapping of button numbers to button positions (though we can make an educated assumption about the directional inputs). Though I guess most 4 button controllers do follow a layout similar to these old speedlink generic pads and 9/10 as select/start is semi-standard, so maybe that's an ok default. (I'd still obviously have overrides for individual VID/PID. Maybe make it load these from a file?)
Then again, mapping real buttons to virtual ones is an inexact science to begin with. How do you map a normal 4-button layout to a SEGA controller?
Also, just to mess with us, there's multiple layouts of NeoGeo controller...
Which are both inadequate at matching the 4-buttons-in-a-row joystick layout (imagine pressing B and C simultaneously. This is infact needed for some moves in The Last Blade)
Actually, I realized that it just uses POLLCT3 to figure out when it's time to poll the devices. Could probably use a scanline trigger from the video driver instead, which should give the most consistent response time possible with the lowest bus usage possible. Would need to determine how long the polling process with four controllers actually takes, so as to trigger it far enough up that the new data always arrives ahead of the VBlank interrupt. Maybe trigger it twice per frame, which would be more similar to the current 125 Hz rate and avoid scenarios where weird poll timings in the game code would incur a full whopping 16ms of lag.
To get around the various USB controller button differences maybe you could have a setup screen that lets people press buttons for different directions etc and you could scan the reports and figure out which report bit fields change and associate them accordingly. I think that is generally what emulator environments like MAME do, right?
I expect this is quite a bit more high level software to develop though and ideally it would also need state saving somewhere to avoid doing this each time you run the emulator. But some of that code could be reusable in other future projects at least. It would be a handy component.
By the way it's great that you are looking into this host stuff using Macca's code as a starting point... Ideally we could eventually get a keyboard, mouse and a couple of USB controllers' inputs all from a single COG. You'd just need a cheap 4 port USB hub to attach to a P2 then.
The configurator can be a standalone program that allows this and saves the config files and then starts *yume.
I hope I can connect a RPi dedicated keyboard. It has a hub inside and 3 USB ports.
It's been a long time since I last reviewed the code, however it should support both low and high speed devices on the hub, including keyboards. Mouse is not supported, but I think it can be eaisly added.
Hot-plug is supported however it may happen that something breaks and devices stops responding. It should work when all devices are connected at startup but I found some problems even in that case.
The code is really a mess, I know.
Yeah that'd be quite a neat and tidy setup for a small P2 emulation/game system with the inbuilt hub. Good idea. Hopefully its USB hub should be compatible in the end...
The problems with having a control configuration GUI in the emulator itself are:
By making it just load configs from a file I can make it someone else's problem, as @pik33 suggested.
Interesting first results just playing with the code:
EDIT figured out the issue with the HID device decoding. Below
.hub_devices
there's a copy-paste error. PR submitted.Other weird quandary: On hubs with a hardwired device (like the RPi keyboard @pik33 mentioned), it might be hogging port 1, which would prevent connecting a game controller for player 1. Dynamically assigning player ids to connected devices in order would also kinda suck I guess, but not as much? (Because maybe you want one player on the keyboard!).
If anyone has such a device laying around, perhaps investigate how it's wired. On windows, open device manager, set view to "Devices by connection" and find the keyboard. Open the device one level up (or two levels if you have a weirdo board that shows up as a composite device - basically the direct child of the hub device) from the actual keyboard endpoint. It should have a location like
Port_#0003.Hub_#0003
. The first number is the port number (1-based).Shitpost status: flexspin can no longer compile DEBUG
if
due to conditional assembly, which I was trying to apply to making it easy to enable/disable the activity LED (which I always used to rip out of the USB code because the blinking is obnoxious), so I updated my main flexspin install and now the whole thing doesn't build.Shitpost status: Flexspin got fixed.
Next obnoxious issue: macca converted the whole thing into a standalone ASM blob, but I want it to be a Spin object again.
Of course it stops working once I replace the ASM start stub
with a spin start stub
Flexspin should have absolute addressing in DAT sections, so who knows what the heck goes on here.
And this is why I put this task off so long.
Aaand I just noticed that it now suffers from having horrible update rate with the neo geo mini controller. I could've sworn it didn't do this yesterday, maybe I was just thick.
Eitherhow, the previous Spin iteration of @macca 's HID decoder had the same issue.
The descriptor is kinda weird...
EDIT
WTF? Going back to the original code without any of my edits, the poll rate is bugged, too. It turns out yesterday I was using Propeller Tool, where this bug does not happen? (Again, only affects this one particular gamepad).
Why does an ASM-only program behave differently on flexspin? aaaaa I'm feeling it.
EDIT some more: trying to compare binaries is fruitless due to flexspin and proptool having different opinions on when to use relative/absolute addressing modes. Classy.
EDIT 3: Issue has nothing to do with HID stuff, the flexspin version for some reason gets a lot of NAK responses? WHY?
EDIT 4: Things tried fruitlessly: loading proptool binary with loadp2 (still works), changing clkfreq (no difference), inserting waits at startup (no difference)
Okay, too much brainhurt for now. Am away over the weekend and hoped to make some inroads before that. No luck.
I did order some parts to hopefully build a crappy USB interception device (read: some breakouts that I will use to tap my bootleg logic analyzer into the data lines), those should be there by monday, too. oh well.
EDIT: Thinking about it, maybe it'd be slightly smarter to port the hub support portions of the code to the older spin driver instead?