Why I'm not porting Polymeter

Milkii Brewster on GitHub asks "Linux version? Plugin version? Possibly maybe in the future?" and here's my answer:

The Polymeter source currently consists of 47,100 lines of C++ code. Of that, 80% is UI. How do I know that? To facilitate automated testing, I created a command line version of Polymeter, which can open and play a .PLM (Polymeter) document, and optionally export it to a standard MIDI file. After all of the UI is eliminated, 9,447 lines of code remain, and if you were planning to port the project, that would be the right place to start. The UI code is so dependent on MFC (Microsoft Foundation Classes) that it would almost certainly be easier to throw it away and start from scratch.

The sequencer core also depends on MFC, though far less so: it only depends on CString and the CArray and CMap container classes. It’s on my to-do list to make a fork of the command line version that uses the standard C++ library only. That means creating wrappers that emulate the interfaces of CString, CArray, and CMap, and that’s a much bigger job than it sounds. You might think someone would have already done that work, and perhaps they have, but if so they didn’t advertise it very well.

Even supposing those wrappers already existed and worked reliably, my application still depends on the Windows API. In particular, the sequencer is designed around the MIDI Stream API. Equivalent APIs may exist in Linux or MacOS, but even seemingly small differences between those hypothetical APIs and the Windows API could present huge obstacles. I surmise that it’s possible, because my program works correctly under Wine. I’ve never studied the Wine MIDI Stream code, but I wouldn’t be surprised to learn that it’s messy. It’s solving a hard problem that involves multithreading and device drivers.

But these are just technical issues. I’ve done plenty of porting, for example during my professional career I ported software from Windows to embedded processors, so I have a good idea what’s involved. The real issue isn’t technical, it’s temporal. I’m old! I’ve been working on the modern (Windows) version of this project for three years already, and I started the original version in 1998 under MS-DOS. I probably don’t have enough time left in my life to learn UI programming for MacOS or Linux or WxWidgets or Qt, certainly not at the same level of expertise I already have with Windows and MFC.

I didn’t write a polymeter sequencer out of curiosity, I did so because I wanted to compose in complex polymeter. I had hoped to avoid spending so much time writing software. I tried many commercial products before undertaking the Windows version of Polymeter, but sadly none of them had significant support for complex polymeter, and certainly not for polymeter modulation, many types of which I have—as far as I know—invented. I prefer to spend my remaining years actually using my sequencer, which I’ve paid so dearly for, to compose novel and aesthetically pleasing music. Other than composing persuasive examples of complex polymeter, probably the best use of my time would be to write a paper about all this. That’s also on my to-do list.

But even if I had another life to lead, why would I spend it on porting when perfectly good solutions already exist? There’s no significant obstacle to you or anyone else running Polymeter. You could simply obtain a Windows computer. It needn’t be anything fancy. A used PC would set you back a few hundred bucks, surely insignificant compared to thousands of hours of development time. Or you could use PlayOnMac or Crossover or Wine itself to run Polymeter under Linux or MacOS. Sure there are a few issues running in Wine, but 99% of the program works fine. Or you could run a VM, or do dual boot. So in short, the answer is, are you volunteering to port the project? If so, great. Otherwise, it is what it is.

Polymeter Phase diagram with Convergences

Version 1.0.4 of the Polymeter MIDI Sequencer features a couple of cool enhancements to the Phase bar, both accessible via its context menu. The first is that the phase diagram can now indicate convergences slightly before and after they occur. The participating orbits are increasingly highlighted as the convergence approaches, and gradually return to normal as the convergence recedes. This is useful because the planets often move fast enough that it can be hard to tell when they’re lined up. Also the planets can appear lined up when they’re actually not, unlike the convergence indicator which is foolproof. And, because the highlighting changes both the width and the color of each orbit in proportion to its proximity to convergence, it’s very satisfying to watch. The second feature is that the Phase bar can now go full screen, without caption bar or borders, which is similarly visually pleasing. Music: "Atunwi" from "Polymeter" (Mental Groove). More info: https://victimofleisure.github.io/

 

 

Polymeter 0.0.31 adds offset modulation and voice raising

Version 0.0.31 of Polymeter introduces a new feature called Offset modulation. This new modulation type allows a track's time offset to be changed by another track. This is useful for making the track periodically slow down and then catch up again, which helps emulate the feel of a live performer. Offset modulation is customizable per track, unlike tempo modulation which is global and affects all tracks the same. An offset modulator track's step values are interpreted as unsigned delays from 0 to 127 MIDI ticks. Note that by default this scheme only allows events to be delayed, not made earlier. To make events earlier, set the target track's Offset parameter to a negative value, e.g. -64, which is roughly an eighth note using the default time division of 120 PPQ. Because offset modulation is added to the track offset, this shifts the offset modulation range to -64 to +63 ticks, so that the target track's events can be made up to an eighth note early or late. For offset modulation to work properly you may need to set the "Note Overlaps" Master property to "Prevent."

Version 0.0.31 also enhances voicing modulation. In previous versions, voicing modulation could only lower voices an octave, but not raise them an octave. This was sufficient for common chord voicings such as drop 2 and drop 3, but more esoteric chord voicings exist that require some voices to be dropped and others raised. Consequently the voicing value can now be negative, in which case the specified voice is raised an octave instead of lowered an octave. Yes, it's inverted which is a bit annoying, but since lowering is by far the more common case it gets to be positive. In either case the numbering is still one-origin relative to the top voice, i.e. the voice with the highest pitch. An example for a tetrachord:

Voicing modulation value	Result
2	2nd voice from the top is lowered an octave
3	3rd voice from the top is lowered an octave
-2	2nd voice from the top is raised an octave
-3	3rd voice from the top is raised an octave

Lowering or raising multiple voices is achieved by assigning multiple voicing modulators to the target track.

Offset modulation demo files are available here.

Optimizations in sequencer core

There's an updated version, 0.0.30.001, which will process complex modulation setups significantly faster, due to some sorely-needed optimizations in the sequencer core. If you occasionally get the error "Sequencer Callback took too long" this version may help. Basically the sequencer was doing three dumb things:

1. Muted tracks were processing their modulators, pointlessly.
   
2. Modulator tracks were processing their modulators, pointlessly.
   
3. If a mute modulator evaluated true, the track's remaining modulators were evaluated, pointlessly.
   

All three of these were wasting precious time in the callback thread for no good reason. Large projects often develop subtle flaws like this, due to iterative design over long periods of time. Periodic performance checks and code reviews are a must.

Polymeter 0.0.30 adds synchronization via MIDI clock

Polymeter 0.0.30 optionally transmits MIDI clock, allowing you to synchronize your DAW or hardware sequencer with Polymeter. This was by far the most-requested new feature. The Start, Continue, Stop and Song Position messages are also sent as usual. Note that this feature must be enabled, via Options/Send MIDI Clock.

MIDI clock is old school, but it’s widely supported and works pretty well. There are some limitations, all involving the starting position. 1) Playback must start on a sixteenth note boundary; if you try to start from a position that isn’t evenly divisible by a sixteenth note, you’ll get a warning message and your starting position will be quantized to the nearest sixteenth note. 2) Playback can’t start from a negative position; this is MIDI’s limitation, not mine! 3) Similarly, playback can’t start from a position later than 16,383 sixteenth notes; again, not my fault. And 4) once playback is started, skipping forward or backward in the sequence will cause loss of synchronization; that’s because the MIDI clock specification doesn’t support position change during playback, alas.

This version adds some other handy features. For one, the ability to assign multiple modulators at once (via Insert Group on the Modulation bar’s context menu). This is especially useful with scale and chord modulation. And, the Step Values bar now supports multi-step editing; if the edited step is within a contiguous range of selected items, the track’s corresponding steps are filled with the new value. This is an easier way to set all of a track’s steps to some value, as opposed to using the Velocity pane.

Finally, this version supports assigning custom colors of tracks in the Track view. The color picker is in the main toolbar. This feature is disabled by default, but you can enable it via Options/Show Track Colors.

Polymeter 0.0.29 adds Chord Modulation and mapping to steps

Polymeter 0.0.29 introduces a major new feature called Chord Modulation, explained below. There are many other additions and improvements, including the ability to remotely control each individual step of a track. The release notes are here.

Chord Modulation

A scale is a subset of the chromatic scale. Each scale tone is a note, expressed as an offset in semitones relative to some base note. Assuming a base note of C, zero is C, one is Db, two is D, three is Eb, and so forth.

A chord is a subset of a scale. A chord can contain any number of tones, up to and including all of the tones of the parent scale. Chords are created by modulating a note track with a combination of scale, chord, and index modulators. Voicing modulators may optionally be added also.

Without a chord, index modulation accesses the tones of the current scale. The index modulation picks a scale tone, and the scale tone maps to a note. For example, suppose a note track has seven scale modulators, constituting a C major scale: C, D, E, F, G, A, B. The index modulation is interpreted as a zero-based index into that scale, ranging from zero to six. In other words, if the index modulation is zero the C is picked, if it’s one the D is picked, if it’s two the E is picked, and so on.

With a chord, index modulation accesses the tones of the current chord. This adds an extra level of mapping. The index modulation picks a chord tone, the chord tone map to a scale tone, and the scale tone maps to a note. Suppose the track has four chord modulators, which output 0, 2, 4, and 6 respectively. These correspond to the first, third, fifth, and seventh notes of the current scale. So if the scale is C major as before, the resulting chord is C major seventh.

Pitches:

Index	0	1	2	3	4	5	6	7	8	9	10	11
Note	C	Db	D	Eb	E	F	Gb	G	Ab	A	Bb	B

Scale:

Index	0	1	2	3	4	5	6
Scale Tone	0	2	4	5	7	9	11
Note	C	D	E	F	G	A	B

Chord:

Index	0	1	2	3
Chord Tone	0	2	4	6
Scale Tone	0	4	7	11
Note	C	E	G	B

In this example, when accessing the scale, indices 0, 1, 2 map to the notes C, D, E but when accessing the chord, the same indices map to the notes C E G. The chord gives the indices a different meaning.

Notice that three things are being distinguished: the subset of notes (the scale), the subset of scale tones (the chord), and which tone is picked (the index). All three of these entities are distinguished from the rhythm, which is determined by the note track. So we have an orthogonal system in which each axis (scale, chord, ordering, and rhythm) is independent of the others.

A demo file that makes use of chord modulation is attached to this post. The demo file uses chord modulation to implement the "Numbers" system of tonal palettes popularized by my ex-teacher, noted tenor saxophonist Jerry Bergonzi. This system was also implemented in my previous project ChordEase. The Numbers system is explained here.

Mapping to Track Steps

Why would you want to want to map a controller to a track step? I’m glad you asked that question. For one thing, it means you can control the tempo. Here’s how.

Create a track, change its type to Tempo, set its Length to one, and set its Duration to 100. The duration in this case is repurposed as the range of tempo scaling, as a percentage. The 100 means the tempo control ranges from half time to double time. Create a mapping for whatever controller you’re using. Within the mapping, set the Output Event to Step, the Output Control to zero, and the Track to your tempo track. Note that for Step mapping, Output Control is the zero-based index of the target step.

Start the sequencer. Your controller should now control the tempo. Zero gives you half time, 127 gives you double time, and at 64 the base tempo is unaffected.

There’s lots of other fun stuff you can do with step mapping. For example, you could make an empty track, and then map each of its steps to a key on a keyboard. Pressing and holding a note on the keyboard sets the corresponding step in your pattern. How hard you hit the key determines the step’s velocity. Playing a chord on the keyboard gives you a rhythm.
Try doing this for two different tracks, each with a different prime length. For example one could have a length of five and the other a length of seven. Create two sets of step mappings, one for each track, on different ranges of the keyboard. Now you can dynamically create two rhythms, one with each hand, and they’re in polymeter. Pretty cool!

Step mapping will most likely be useful for controlling the behavior modulator tracks, for example range modulation, voicing modulation, and so on.

Docking bars to tabbed panes: visible but inactive

Re the issue with not being able to dock Piano and MIDI bars to a tabbed pane: I looked into that some more. It's a non-trivial issue with pros and cons that need consideration. But the short form is this: Currently, keeping many of the bars in tabbed panes may make the UI sluggish. This is especially true of the Step Values bar!

When a bar is docked to a tabbed pane, Windows considers it "visible" even if it's not the active tab, in which case it's not actually visible to the user. In my current implementation, such "visible but inactive" bars receive document updates (affecting performance), though they don't get "painted".

It seems reasonable that hidden bars shouldn't consume any CPU cycles. When a bar gets "shown" my code assumes the bar is "dirty" (meaning it wasn't being updated while it was "hidden"), and updates it. But there's a catch to that work-saving strategy: bars that show real-time events (e.g. the Piano and MIDI event bars) will miss events that occur while they're hidden. This is why when you show the Piano bar, it doesn't display notes that were already in progress by the time you showed the bar; it missed those events.

It's also why I had to disable tabbed docking for Piano and MIDI event bars. Those bars have big impact, because they oblige the sequencer to send a "monitoring" copy of the entire output MIDI sequence to the main thread. If those bars were allowed to tab-dock (or auto-hide), they would drag performance even though they might not be visible to the user. This seemed wrong to me, so I disallowed it.

It's a messy situation with competing design goals. On the one hand, it's nice if bars don't consume resources while they're "hidden". On the other hand, it's annoying that when you show the Piano or MIDI event bars, events that occurred while they were hidden are missing.