Making Music with Network Graphs: Auditory Feedback on Your Research Process via MIDI

InfraNodus can visualize any text as a network of meaning relations. You can then automatically convert your graph nodes into MIDI notes and send them to an external MIDI device to play music using the text graph. It is also possible to receive the incoming MIDI signals (notes with channels) and to build a graph from them). Here is how it works.

Sending the MIDI Signal from InfraNodus to another MIDI Device

You can activate the MIDI option, which enables you to send the actions that you perform on the graph to an external MIDI device via USB from your computer. You can then play the graph as an instrument, using the text as a score. 

The sound will be played when you select the nodes, when you scroll through the topics (if you activate the Timer Watch dynamic graph mode), when you add the new text, nodes, and connections. 

Each concept (a node on the graph) is then ascribed a MIDI note, octave, and channel, so you can connect your midi synth via the USB. In our case it was an iPad with the GliderVerb app synth connected to the computer through iConnectAudio USB card MIDI interface.

It is then possible to receive the signals as you read, manipulate, or write a text using InfraNodus' text network graph. This way working with a research tool also becomes a way of making music, while receiving auditory feedback on the research process, which may lead to new interesting insights or at least a more playful use of an instrument: turning it into heuristic device for finding meaning through the aesthetics of patterns and sounds.  

Receiving the MIDI Signal to InfraNodus — Building a Network Graph from Your Music

InfraNodus can also receive MIDI signal from an external device, so you can build a text network graph from your music. Depending on the signal you send, you can visualize every note (e.g. C1) or every note with an octave (e.g. C12) as a node in the graph, their co-occurrences is a relation. So, in a very simple case, if you have a melody like C12 D21 C22 then you'd have a triangle connecting those nodes together. Of course, the more complex your structure gets, the more interesting the text network graph becomes. You can then use the analytical tools inside InfraNodus to get some interesting insights about your music: which patterns you tend to use more often, what are the "latent" nodes, what are the structural parameters of your music, etc.

Video Tutorial:

Read step-by-step instruction to activate your MIDI interface in InfraNodus below.

Step-by-Step Instructions:

Step 1: Open the settings tab

Step 2: Scroll to MIDI and activate it

Step 3: Specify MIDI settings

You can skip the first two settings where the nodes and edges are mapped on to a channel, as this is a legacy setting.

The only thing you need to adjust are the MIDI device IDs you're going to be using to send and to receive MIDI signals via USB. Normally you want to set your output channel to 1, but in some cases it may also be 0. You can set 1 first and experiment to see whether it works, then re-adjust.

Note, that you can also record the incoming MIDI input signal if you choose the Device 01 or Device 00 from the list "Record the MIDI input graph". In this case, every time you play something on your external device, you will receive a MIDI note into the editor, which will then be saved into the graph.

Step 4: Save Settings

Click Save Settings to save the changes to your configuration

Step 5: Connect devices

Make sure your MIDI synth is connected via a USB. A setup we like to use is a MIDI enabled soundcard like iConnectAUDIO+, which then forwards the MIDI signal it receives on its input channel from the computer to an iPad, where we have GliderVerb synth installed. You could also bypass an iPad / soundcard setup and plug in your USB MIDI synthesizer directly into your computer. For example, Teenage Engineering OP-1 or OPZ work pretty well, but you could also try others. 

Step 6: Create a new graph and start writing text 

For example, you can add a sentence like

"All music starts from the very beginning and then unfolds into multiple threads simultaneously"

This will be visualized as a graph. 

The MIDI signal will be sent to the USB device of your choice (0 or 1) to play every node that is added. 

Step 7: Open an existing graph to play a text

You can also open an existing graph and play your text from there. Simply click the nodes and a note will play accordingly.

Important Note: Language to MIDI Note Mapping Algorithm

The algorithm automatically converts the words to specific MIDI notes, which will always stay the same, no matter which graph you are creating.

The first letter of the word is used to generate a MIDI note (e.g. C, D, E) — effectively mapping 26 letters of the English alphabet to the 7 notes (A to F). So depending on the sound of the word's beginning you will have a different tonality. The word's length is used to determine the octave, which basically determines the pitch of the note. So a longer word will have a higher pitch and a shorter word will have a lower pitch. As the median size of an English word is about 5 letters, those are the most typical C3 sounds. Everything above is higher in tonality, everything below is lower (bass line).

Troubleshooting:

If your MIDI synthesizer is not playing anything, please, open Developer Tools in your browser and go to Javascript Console. If you have MIDI activated in InfraNodus you will see which connected MIDI devices are mapped and recognized by your browser. You can then check if the mapping (especially the MIDI device ID) is correct and make changes accordingly in the settings. You can do that either in Settings or using the MIDI menu on the left. 

What Graph Actions are Sonified:

1) Adding new words (nodes)

2) Clicking on nodes (selecting and deselecting)

3) Selecting topics (via Analytics panel — small color square plays a chord)

4) Using the Dynamic Graph feature where you can "play" the evolution of the graph (see 🎥 How to Watch the Dynamic Evolution of a Graph?)

5) Using the Timer button (left menu) to be able to highlight a certain portion of the graph when you scroll through the statements — great for manually playing or "scratching" — see the video below

Examples

Below you can see some of the examples of how we used InfraNodus in a music / artistic context.

Example Case Study: The 3 Readings of the Bible

The Bible's Genesis was visualized as a text network. Each node is a word, every co-occurrence is the relation. The nodes are then aligned in a 2D space using Mathieu Jacomy's Force Layout 2 algorithm built into Alexis Jacomy's Sigma.Js library. The most connected nodes are dissuaded while the nodes (words) that appear more often in the same context are clustered together. As a result we can see the network structure of the text's narrative. If we "play" it using the Dynamic Graph feature of InfraNodus we can see how the text evolved in time and observe the patterns of meaning activation in that particular structure. 

Each word in the graph also has a musical component: one-to-one mapping to a MIDI note (e.g. F1, C2 etc.) This note plays every time the word is activated through the network of meaning produced when we read the text chronologically (also also when we pray repeatedly or when we look through the genesis of the Bible's narrative and its structure), revealing the frequencies and the patterns, which represent the way the story evolves over space and time through the sound.