The watchmaker

To get special results you need special tools, and the more sophisticated the tools, the more sophisticated the result can be. Most people are using technology to do things that humans can already do, just much faster. I'm interested in things that computers can do, that I can't do easily or at all, that let me extend my creativity into areas that I couldn't otherwise access or even fully visualize. It's not unlike building a bathysphere to visit the ocean depths. What's down there? How can we be sure if we can't get there?

I find people are surprisingly resistant to the idea of co-creating with AI, but I'm convinced it's the future of art. I see it as just another stage in our evolution into cyborgs, as we extend our ability to manipulate our environment by building increasingly powerful and sophisticated tools. Musicians sometimes get upset when I say that music has co-evolved with technology, but it's obvious: a piano is a machine, and couldn't have been properly manufactured before the 18th century because the necessary machining skills didn't exist yet. Same with modern brass instruments, the valves and curves of which require ferociously complex math and engineering skill that evolved with steam engine technology.

Even though I've sometimes used it, I'm always suspicious of randomness as a method of generating variety in art and music, because the essence of artistic expression is intention, and true randomness is inherently unintentional. The beauty of polymeter as a means of generating variety is that it's non-random, i.e. completely intentional. It also captures a key aspect of our biological reality, right down to the mechanics of cells and enzymes: myriad clocks, all cycling while maintaining precise phase relationships with each other. Wheels within wheels within wheels. When I'm composing polymeter, I sometimes imagine myself as a watchmaker: not Richard Dawkin's blind watchmaker, but the Victorian kind, staring through a huge magnifying glass to turn tiny screws that would otherwise be invisible. Victorian watchmakers needed special tools, and I do too.

Progress report, some background, and the road ahead

I continue to be astonished by the complexity and rich feature set of the original DOS software the Polymeter project is based on, considering the harsh limitations it contended with: 16-bit pointers, chronic memory shortage (640K max), no GUI support from the OS (I had to write my own menu and window manager), and time-critical MIDI stuff running bare-metal as ISRs (Interrupt Service Routines) painstakingly hand-crafted around obscure details of a particular ISA MIDI card, the manufacture of which vanished without a trace years ago.

I'm trying to save Polymeter from bit-rot while preserving the look and feel of the original, except in cases where doing so would be obviously detrimental, or where an original design decision was a kludge necessitated by the limitations of 1990s software. Many of the improvements are under the hood, e.g. the original was limited to 384 tracks, each consisting of at most 384 steps, whereas the new version supports up to 65,536 tracks, each containing up to millions of steps.

The original consisted of three separate applications, which I'm hoping to consolidate into one: Bongo, Jock, and Stencil. They correspond to the three stages of my workflow: 1) composing polymeter loops, 2) performing live with those loops and recording the transitions, and 3) editing the performance afterwards in order to refine the transitions. Bongo was the loop composing tool, and that's the part I'm working on now. If you're curious you can find screenshots of all three apps on the Gallery page of the project's website.

Before undertaking this port. I evaluated all the well-known DAWs and sequencers, and quite a few obscure ones too, including downloading trial versions. Most weren't even close, but the closest fits were Ableton and Fruity Loops. Ableton supports creating polymeter loops, including controller loops, but with a fatal flaw: when you go to place the loops and build a song, the tracks don't necessarily stay synchronized with the start of the song, and thus the critical phase relationships between tracks are easily lost. This is the problem that "Stencil" has to solve: make it easy to "paint" with your tracks, much the way Fruity Loops does, so that when tracks dub in and out, their original phase relationships are always preserved. The idea is that you should be able to edit your track transitions with 100% confidence that the underlying polymeter phase relationships are always preserved.

The most important feature of Bongo that's not ported yet is the ability to "mask" tracks with other tracks. In other words, a track in one meter, muting and unmuting another track in a different meter. A mask can also apply to multiple tracks, all in different meters. But the kicker is, a mask can also mask another mask! It's crucial to grasp that muting and unmuting only affect notes in the future, and have no effect on notes already in progress. This is a very different effect from a volume controller loop, for example, which continuously affects all notes in progress on that instrument.

Masking is a huge PITA to implement for reasons I won't go into, but it's also the indispensable heart and soul of the invention, so I'm packing for a long hike. I expect it to take months, but it really depends on how much time I have to work on it.

I also hope to enhance the masking capability to include modulation of other track parameters, not only mute/unmute. For example why can't a track transpose another track's note, or change another track's offset or swing? These were things I wanted to do back in the 1990s but couldn't, due to limitations in the software's architecture, and in some cases, insufficient programming skill, it must be admitted.

But the holy grail is chord masking: note tracks in different meters, having their notes continuously "corrected" in real time to fit a set of chord changes that's running in another meter. Much of the code could be transplanted from another one of my projects (ChordEase). This concept might solve one of my longest-running problems, which is that it's excruciatingly difficult to make non-modal polymeter that isn't brutally dissonant. With ChordEase-style note correction, this would no longer be a problem. The tracks would just magically fit the changes as closely as possible, with no fussing around. I think this idea has tremendous potential and would be well worth the development cost, which will be daunting to say the least.

Polymeter 0.0.05 released: controller tracks, velocity editing and much more

Polymeter 0.0.05 is available for download and adds many new features, including velocity editing and controller tracks. Tracks now have a Type drop list, which can be set to note, controller, patch change, aftertouch (key or channel), or pitch bend.
The velocities pane lets you edit step values of any type, not just note velocities. It shows one bar per note, or one bar per step for track types other than note. Just left-click on a bar to change its height, or left-click and drag to edit multiple bars in a row. The bars can be fit to lines, triangle waves, and sine waves, using combinations of the Ctrl, Shift, and Alt keys while dragging the cursor. And you can edit multiple tracks at once of course.
New editing commands were also added: a Reverse command that reverses the order of selected tracks or steps, and a Rotate command that rotates selected tracks or steps left or right.

Major milestone reached: note velocity editing

Polymeter reached a major milestone today: note velocity editing! The velocity pane is an optional addition to the step view. It's resizable via a splitter bar, and stays synchronized with the step view's scrolling and zoom. Multiple tracks can be edited at once, in which case their note velocities are merged. Various refinements are still in progress, e.g. a tool tip showing the current velocity, and fancy editing behaviors such as clamping velocities to a line segment or periodic function. The velocity editing pane can be reused to edit other values besides notes, e.g. controller values, and is thus a big step towards the next milestone, which is controller tracks. However controller tracks will require significant changes to the sequencer core, particularly for linear interpolation.

Initial alpha release of Polymeter available for download

The initial alpha release of Polymeter is now available for download, at both SourceForge and GitHub. It's a fully functional MIDI step sequencer optimized for polymeter. Note however that this version is ALPHA software and many planned features are not yet implemented fully or at all. In particular, while this version is fine for drums or other percussive samples, it's less useful for melodic lines, due to its lack of the crucial "Legato" feature that ties consecutive steps together. Please check the Features list before requesting features, as the thing you want may already be planned.
https://polymeter.sourceforge.io/download.html
https://victimofleisure.github.io/Polymeter/download.html
https://polymeter.sourceforge.io/features.html

Input device and MIDI Thru

I got MIDI Thru working! No MIDI sequencer is complete without it. The input side was easy, I pretty much ripped it straight out of ChordEase.
The output side was tricky however, because you can't write events directly to a MIDI device that's been opened for synchronized streaming (as opposed to immediate output, as in ChordEase). Instead your callback function (which is being called at regular intervals by Windows) has to add the events to its next buffer, which it subsequently queues to the output device. My solution was to add a special input queue for "live" events. At the start of each callback, I check the live input queue, and if there are events in it, I dequeue them and add them to the start of my next MIDI buffer, before any events that may get added from the song's tracks. This is quite safe (provided a thread-safe queue is used) and it works pretty well. It does have some limitations though.

1. MIDI thru is only operational while the sequencer is playing, not while it's stopped or paused. This is potentially annoying, and the only solution is to keep the MIDI output device open all the time, not only during playback. Doing so would have the additional benefit of reducing the lag between pressing "Play" and playback actually starting. Most of that lag is caused by opening the device, though some devices are worse than others. It's a cool idea and it's on the list, but I'm not going to deal with it right away.
   
2. The delay can be longer than one might like, depending on how the sequencer's latency is set. At the default latency of 10ms, it's probably fine for controlling parameters, but Chopin is out of the question. The sequencer's latency can be set lower, all the way down to 1ms, but the lower it is, the greater the risk of timing glitches in song playback, due to the callback taking too long and not being able to keep up. This also depends on other factors such as tempo and the density of the sequence.
   
3. The delay isn't constant. This is because I grab all the live input events that have occurred since the last callback and write them all to the start of the buffer. They probably didn't all happen at the same time, but they're all output at the same time. In other words, the live input can get "bunched up" into discrete packets.
   

Whether "bunching" is a problem depends on how fast and dense the input is. If the input is sparse, there might only be one input event per callback. But if there are multiple events per callback, the callback could try to approximate their original timing. The events have timestamps, which can presumably be used to space out the events in time. Note however that doing so makes the events even LATER than they already are. A nasty trade-off! This is a hard problem and I intend to ignore it for now. Possible bunching aside, the current scheme is straightforward and minimizes delay.
The live input queue also makes it possible to do live patch changes, along with volume and panning. This is already implemented in the Channels bar. The addition of a MIDI input device is a prerequisite to other useful things, such as MIDI mapping of all parameters, which is also on the list.

Time to repeat: longer than the universe has existed

After a day spent studying number theory, my Polymeter sequencer has something I always wanted: a "Time to Repeat" command! The command computes how long it would take for the selected tracks to repeat, in both beats and time (along with the greatest prime factor) and displays the results. I tried it on my test track that uses all prime meters less than 32 (except 23). "Only" 22,428 days. That's longer than I've been alive! That's some serious polymeter. During stress testing I made some combinations that would take trillions of years. Thomas Wilfred would be impressed. https://en.wikipedia.org/wiki/Thomas_Wilfred