Feedback from HOTAS

[JNOV]

I am in the planning stage of implementing an application that runs in the background and provides feedback from CH HOTAS devices. There are any number of situations in when I have wanted such an application in the past. For example, in a WWII flight sim, I might program modes for different combinations of guns, cannons, rockets, bombs, etc. I might create a program that changes the stick sensitivity on-the-fly (e.g., one setting for landing, one for taxiing, one for cruise, one for dogfighting, etc.). My program might allow me to cycle through a number (e.g., 10) zoom levels. In any of those cases, it might be nice to be able to see what "mode" the HOTAS is currently in. Or you might just like a real-time display of the values that DirectX is receiving for the various axes that you are using to control the game. There are myriad possibilities. (For a number of reasons, the three LEDs on the Fighterstick and Pro Throttle are inadequate. For starters, they are hardwired to the forefinger button and ministick button, respectively, and are thus totally inflexible.)

Due to the wonderful flexibility of Control Manager and CMS, it is a simple manner to devise a scheme that allows the HOTAS to communicate with the background application without impinging on or interfering with the HOTAS's primary mission--communicating with the game application. Moreover, the information that the HOTAS can send to the background application is limited only by the HOTAS's memory.

There are two general classes of communications from the HOTAS to the background application that I am planning right now. The first is simply a command code, to which the background application responds in a predefined way. There can be any number of these command codes, each of which represents a different HOTAS state and prompts a separate feedback indication from the background application. The other class of communication is strings. The HOTAS simply sends a text string (i.e., "80% Zoom") that the background application relays to the user. Depending on the form of feedback, the text string may have embedded control characters to control positioning, font, etc.

The trickiest part is deciding on a means for the background application to provide feedback to the user. The slickest solution would be if everyone had a dual-monitor set-up and all game applications allowed use of the second monitor. Sadly, I don't think that this solution is practical right now. Nonetheless, there are several options that I an considering, each of which has advantages and disadvantages.

1. Sound. The user would create .wav files that would provide useful feedback. For example, the user could simply record his own voice saying things like "Zoom 10%," "Gear down," "Stick sensitivity dogfight," etc. The background application would then be configured to play the appropriate .wav files upon receiving the corresponding command codes from the HOTAS.

2. Display overlay. The background application would create a feedback display (similar to the FRAPs frame-rate display) in some unotrusive location on the game display. The feedback display could be responsive to either command codes or text strings from the HOTAS. This is more flexible than the sound option and offers persistence, although the dispaly area will be quite limited.

3. Separate display. This option probably affords the most flexibility, but it requires an extra bit of hardware. There are a number of inexpensive USB/serial LCD displays available from companies like Crystalfontz (www.crystalfontz.com). The background application could send feedback information to such a display, responsive to either or both of command codes or text strings. Depending on the type of LCD, graphics would be available.

Anyway, I am interested in any feedback, be it criticism (i.e., that is a stupid idea), suggestions, or comments.

Thanks!

- JNOV

[Revvin]

Thats a great idea! while I'm not really electronically gifted or able to program I could test and application you wrote to breaking point :D

[MichaelCHProd]

Now that is hard core.

[JNOV]

I've got a very rough version of the feedback system completed. So far, I've got the audio feedback implemented, albeit hardcoded and probably not flexible enough to be easily usable by anyone else. It can recognize an arbitrary number of commands from the HOTAS and play corresponding .wav files. I've tried it out in Aces High and it works well. It's pretty wild to hear myself say "Cannons", "bombs", or "rockets" while I'm playing the game :)

Once nice feature of the program is that the way in which the HOTAS communicates with it doesn't rely on sending characters, so there is no delay waiting for the HOTAS to send one or more characters to the background application (i.e., it doesn't tie up your HOTAS, preventing you from sending key commands or button presses to the game). The communication scheme uses just one cms axis (e.g., cms.a1), which the user simply assigns to a DirectX axis that he or she is not otherwise using. You can even stick it on its own CM Device (e.g., Control Manager Device 2, X axis) if you want.

I'm able to send text to the game screen, but it flickers. The (as-far-as-I-can-tell-unavoidable) reason for the flicker is that I'm using Windows GDI to get the screen's device context (GetDC(0)). Thus, I am able to write directly to the active video memory, but when the game switches in its backbuffer (e.g., executes a "present" in D3D), it writes over my text--hence, the flicker. As far as I can tell, there is no way to get a device context for the backbuffer(s) that the game is using. If anyone has any insight into setting an API hook into a remote process's D3D calls, I'd be thankful for your help :)

I'm pretty happy with the program as a proof-of-concept piece, and I think I'm going to take the next step and pick up an LCD unit from Crystalfontz and try that out. I think it will be pretty slick.

- JNOV

[JNOV]

I'm still working on the feedback appliation. I'm using it as a vehicle to learn Windows MFC programming, so I'm playing around with a lot of non-essential stuff like Windows controls. It works now, but requires specifying modes, signals, etc. in a text file. I'm trying to build GUI that makes specifying the feedback that you want easy (I hope).

One interesting development is that I stumbled across Microsoft's speech SDK. This allows, among other things, using synthesized speech in applications. Thus, as an alternative to specifying a *.wav file with audio feedback, you can specify text that Microsoft's speech engine will turn into a spoken phrase. It's quick and easy, if you don't want to take the time to create custom *.wav files.

- JNOV

[Revvin]

That would be cool, especially if you could record your own so that you could click to set engine RPM to maximum and set it so it sounded like your co-pilot saying "roger max rpm sir" as you executed the command. Let me know if you want a guinea pig JNOV I'll help in whatever way I can.

[JNOV]

You can do that sort of thing, Revvin. Basically, you can provide feedback for any event that you can detect in your CMS script.

I'll post the app along with a test profile as soon as I get something remotely acceptable for public consumption.

Thanks - JNOV

[JNOV]

After my last post, I made quite a bit of progress. I built a decent GUI and had fully implemented audial feedback, in the form of *.wav files and voice synthesized text. But what I really wanted was persistent, VISUAL feedback. I toyed with the idea of buying some sort of visual accessory, like a USB LCD panel and interfacing to that. I opted not to do that, however, because it was too expensive (most people wouldn't opt for this) and because what I really wanted was to be able to see my feedback on the screen. So, what I've been doing on and off for the last month or so is trying to figure out how to write to the screen of a DirectX game. Well, suffice it to say that it is no trivial matter for someone with my level of Windows programming background (read: not much). But I think I've finally cracked it, at least for games that use D3D8. For earlier games and D3D9 games, adapting the system should be a simple matter.

Here are a couple of screens in LOMAC, where I've displayed a simple HUD (at the top center of the screen) reflecting two different states. The HUD is simple, as it displays just NAV/A-A/A-G mode and whether the throttle is in afterburner mode or not (I've programmed mine so that when it is not in afterburner mode, max throttle is just under the point at which the burners kick in).

http://home.austin.rr.com/kandeburgess/CH_Hangar/HUD_1.jpg

http://home.austin.rr.com/kandeburgess/CH_Hangar/HUD_2.jpg

The system is totally flexible in terms of how the HUD appears-- you can use any combination of text and graphics--and you can freely alter where the HUD is located on the screen and selectively turn it on and off. If you can create a bmp, jpg, or png image, you can display it. I just created these quickly in Paint Shop Pro. I'm thinking of adding the ability to just render text so that if you want a quick and dirty solution, there's no need to go to the trouble of creating a graphic. I also plan to add transparency, so a larger display might not be too intrusive (particularly if you just flash it up when you need to look at it).

The graphics patch is written almost entirely in assembly language, so it is fast. As far as I can tell, it has no appreciable affect on frame rates.

Anyway, there's still quite a bit of work to do to before this feedback system is integrated and ready for public consumption, but I thought that these results were interesting, so I thought I'd share.

Best regards,

JNOV

[Bob Church]

Now that is really cool! Congratulations, JNOV!

- Bob

The StickWorks
http://www.stickworks.com

[MichaelCHProd]

Good work Brother, that is pretty damn cool. :thumbsup:

[Revvin]

Nice one JNOV! if you need any help hosting files drop me a line :salute:

[JNOV]

Thanks, fellas! It's been fun, but has seriously cut into (read: completely eaten away) my simming time! :)

- JNOV