How to upload midi files to your CME U-Key keyboard from linux

The CME U-key is a cheap midi keyboard but is very convenient :

• works on his own with the included speakers and amplifyer
• light, you may tansport it easily from one room to another
• small in size, you can cope with your pc keyboard and the U-key keyboard both in front of you or store it in a drawer
• both Midi DIN and USB connection
• store up to 16 midi songs, max 56Ko, each of them may define up to 8 loop for play back
• decent keyboard
• my children and I use it to help when singing
• you can use the stored midi songs as an accompaniment when practising your instrument (I play the cornet :-), easily selecting the tempo and the channels you want.
• I supply a copy of the documentation you may find on the CME site

Unfortunately the tool named U-Brain to upload midi files to the keyboard is only available for windows.
I asked CME several time for information about the upload protocol with no response.
I had two solutions : to install a Windows machine or to hack some linux stuff to do the job. I of course selected to hack :-)

You can't upload a midi file directly to the U-Key as the midi file must be of a special format :

• a special Midi header with special midi markers and a special Midi SYSEX command
• you must insert up to 8 marker pairs to locate the loops
• midi format must be Midi0, only one track is defined in the midi file
• a DOC file describing this format is available in the CME download pages, I supply a pdf copy of it
• you may also look at the example midi files also copied from the CME download pages
• I also provide a online readable csv version of the Reggae.mid

Prepare your file to get a UK compatible midi file

• get some midi file from the internet. In this example I used The Crunge a song by Led Zeppelin, downloaded from www.mididb.com. The files at this site are said to be free, but don't ask me if it's legal to get freely some copyrighted material from Led Zeppelin…
• edit it with RoseGarden. Muse is also convenient to do the job.
• for each track assign the channel and the instrument to use on the U-Key, this will also determine which button will control the output level :

  Channel #	Channel Type /U-Key control button
  8	Melody
  9	Bass
  10	Drums
  11 & 12	Chord
  13 & 14	Pad
  15 & 16	Phrase

  
• To do this with RoseGarden proceed as it follows in the Track Parameters panel
  • ensure the Special Parameter panel is shown (otherwise fire it from the Settings dropdown menu)
  • select General MIDI Device as the device
  • then select a channel in the Instrument list just below.
  Don't forget that you must leave the drums on the #10 channel.
• You must also configure the midi instrument for each track. Proceed like indicated below in the Instrument Parameters panel
  • check Bank then select General Midi
  • check Program and select the instrument of your choice
  • set up the controls (Pan,Volume, Sustain, Chorus, Reverb and Expression if necessary



• Add some MIDI Markers to use the U-key's loop pads. Only put LoopStart<n> markers the tool I provide will automatically create LoopEnd<n> markers. Using RoseGarden or Muse won't allow you to put the markers in the correct order at the same place ie LoopEnd2 then LoopStart3
  Yes! The Markers do are case sensitive Expect some amazing effect if you replace LoopStart3 with Loopstart3
  
• To create a marker with RoseGarden, right-click while on the bar count row and select Insert Marker. A Manage Marker windows is also available when you click Edit Markers in the Composition dropdown menu.
  
• once you are done with positioning the markers, I suggest to align them to a bar. Use the Edit Markers tool to do this. Doing this will allow you to move smoothly from a loop to another when pressing the loop buttons on the U-Key.
• once you are done, save your file as a RoseGarden file TheCrunge.rg and export it as a midi file TheCrunge.uk.mid
• close RoseGarden and launch a terminal
• go to whatever directory you did put the .uk.mid file
• format your mid file for the U-key with the ukeyify command,
  ukeyify TheCrunge.uk.mid > TheCrunge.ukcsv
  to create the TheCrunge.ukcsv file.
• I prefer to store the U-key special midi format as a .ukcsv file because this way it's very hackable with a simple gvim or any command line text processing tools, awk, perl etc.
• wrap your .ukcsv with the U-key special MIDI sysex into a syx file with the uksyxify command,
  cat TheCrunge.ukcsv | csvmidi | uksyxify 2 > TheCrunge.syx
  The 2 parameter means you'll upload the midi file to song #2 on the U-key.
• at last upload the syx file with
  cat TheCrunge.syx | ukupload hw:1
  The hw:1 correspond to the alsa raw midi device to use as output.
  I suggest you determine which device to use with a amidi -l command
• you may of course make it more natural with
  ukeyify TheCrunge.uk.mid | csvmidi | uksyxify 2 | ukupload hw:1
• select the song number on th U-Key. If the song number is flashing then get back to the previously selected song, it means you midi file was incorrectly formatted. It usually deals with incorrect LoopStart/LoopEnd
• you may have to switch off/on your keyboard for the midi song to wrok properly.

• Muse, RoseGarden and midi tools (for amidi) should be available as packages in any decent linux distro.
• get also ALSA developpement package to compile ukupload the libasound library and the alsa/asoundlib.h C header
• get, compile and install MIDICSV in a ${PATH}ed directory
• download ukeyify and uksyxify and put them in the same ${PATH}ed directory
• download ukeyify.awk and put it in the same directory you did put uksyxify
• download ukupload.c, compile it with gcc -oukupload -lasound ukupload.c an put the resulting ukupload executable in the {PATH}ed directory
• download uk7bits.c, compile it with gcc -ouk7bits uk7bits.c an put the resulting uk7bits executable in the {PATH}ed directory
• you are done

The following sysex codes are used to send a file to the U-Key :

Header

appears only once at the beginning of the file transfer.

F0 00 20 63 00 01 00 02 00	song#	00	total block count	89 F7
F0 is the midi code for Sysex, 20 63 is CME manufacturer number	song number between 01 and 0F		number of blocks that will be sent. Blocks are 1024 bytes long, except the last one which may be less	F7 is for sysex end. Note that the 89 code used by CME is illegal if you consider the midi specification : SysEx content should be 7bits contents only, thus data bytes should be between 00 and 7F. This is a real problem with ALSA, see below.

Data block

Send as many Data blocks as they are 1024 bytes block in the 7bit encoded midi file.

F0 00 20 63 00 01 00 7F 00	song#	00	block#	89	data, up to 1024 bytes value maybe 00 to 7F	F7
F0 is sysex, 20 63 is CME manufacturer number	song number between 1 and 0F, should math the previously header		current block number, first block# is 0, last block is block count-1	The 89 illegal code	The midi file may contain value greater than 7F so it has to be 7 bits encoded prior to be sent. Find the way it is 7bit encoded below.	End of sysex

The following procedure is used to 7bit encode the midi file

Let's explain this with an example, consider we want to send 8F 0F F7 70 70 FF 00.

I even had to bring back home some Windows XP laptop from work to do the job, that was very disgusting, I had to hide from my children ;-)

One major problem was to find out why the upload process failed even using the exact CME SysEx commands. The reason was usbmidi.c in ALSA source tree : when processing SysEx commands the usbmidi.c automat resets the sending loop when encountering an invalid hex value (between 80 and FF). Thus, when receiving the incorrect 89 hex value defined by CME, it ends abruptly the SysEx command, crashing the upload process.
To bypass this I send a 09 value instead, the U-Key keyboard interprets it correctly.
Windows doesn't care sending an invalid character in SysEx commands.

Windows software used

• U-Key Brain. Get it for free from CME download section.
• IDA Pro debugger/disassembler. Excellent. Get the free version at HexRays
• MIDI-OX Jamie O'Connell's Midi utility available for free at www.midiox.com/. May trap any sysex under Windows. Unfortunately it has been trapped by the incorrect CME sysex syntax. They're also a HP16C emulator available for a few dollars on this page, lemme dream one day I will own the real stuff ;-)

Linux Software used

• Gnome GHex to view hex file
• The very excellent MidiCSV by John Walker available at www.fourmilab.ch. This one saved my life ! Thank you for such a good job John :-)
• Muse available at muse-sequencer.org
• RoseGarden available at www.rosegardenmusic.com.