Connecting Wekinator & Arduino

In this tutorial I’ll explain how you can connect the machine learning software Wekinator with Arduino, enabling you to combine basic machine learning with any gizmo-like project you desire.

Introduction & Motivation

Developed by Dr. Rebecca Fiebrink and named after the flightless Weka bird, Wekinator is a free machine learning software that enables artists, musicians and beyond to easily integrate basic machine learning capabilities into any project. In combination with Arduino, Wekinator has great potential in opening up a new dimension of interaction, allowing makers to experiment with and understand the capabilities of machine learning on a whole new level. However, getting the two programs to talk to one another can be a little tricky, therefore I felt the need to create a simple guide to make the process a bit more approachable, something I wish I had myself!

But before going any further, I recommend that you first read the overview on Wekinator to understand a little more about how the program works. I also highly recommend that you check out their quick walkthrough, not only to become more familiarised with Wekinator’s UI but also to understand its capabilities and limitations.

Finally, I’ve put together a little troubleshooting guide at the bottom of the page, which should hopefully resolve any common issues you may encounter throughout the tutorial.

1. Getting your Arduino Online

Wekinator receives inputs and sends outputs through a network protocol called OSC (Open Sound Control). You can read up more about OSC here. This is where our main obstacle lies. Since OSC is transmitted via a network, the common, non-internet-enabled Arduino like the Uno cannot support it. So the first thing we must do is get our Arduino online.

So we can do this either via an Ethernet or Wifi shield, or by using an internet enabled Arduino model such as the MKR1000 (which I recommend for its small size). Apparently both the Ethernet and Wifi shields are retired on Arduino’s shop but you can still buy them from other sites such as Adafruit or just Ebay.

For this tutorial, I’ll be explaining how to get the Arduino online via Wifi, however there’s plenty of documentation out there if you prefer to use Ethernet instead, such as on Arduino’s website itself.

Start a new sketch in the Arduino IDE. The first thing we need to do is add the relevant libraries. SPI.h and Wifi.h should both already be available in the program and ready to add to your sketch, however if you’re using a Wifi101 Shield or the MKR1000, you must download and include the Wifi101.h library instead, which you can do so from ‘Sketch -> Include Library -> Manage Libraries…’.

// Libraries //
#include <SPI.h>
#include <WiFi.h> // or #include <WiFi101.h>

Next we need to add our properties, being the name and password of the network you wish to connect to.

// Properties // 
char ssid[] = "yourNetwork"; // your network SSID (name) 
char pass[] = "12345678"; // your network password
int status = WL_IDLE_STATUS; // the Wifi radio's status

The ‘status’ property will be used to check whether your Arduino has successfully connected or not.

Next is to write the setup() function which is as follows:

void setup() {
  // initialise serial:
  Serial.begin(9600);
  // attempt to connect to Wifi network:
  while (status != WL_CONNECTED) {
    Serial.print("Attempting to connect to WPA SSID: ");
    Serial.println(ssid);
    // Connect to WPA/WPA2 network:
    status = WiFi.begin(ssid, pass);
    // wait 10 seconds for connection:
    delay(10000);
  }
  // you're connected now, so print out the data:
  Serial.println("You're connected to the network");
  // print your WiFi shield's IP address:
  IPAddress ip = WiFi.localIP();
  Serial.print("IP Address: ");
  Serial.println(ip);
}
void loop() {
  // do nothing
}

So first we initialise serial, setting it at a baud rate of 9600. Next, we attempt to connect, using the name and password we’ve provided. Once the connection is successful, we print a success message and then print the private IP address of our Arduino. You’ll need this later on. Underneath all of this is our loop() function, which so far we can leave blank.

So, so far your sketch should look like this:

// Libraries //
#include <SPI.h>
#include <WiFi.h> // or #include <WiFi101.h>
// Properties //
char ssid[] = "yourNetwork"; // your network SSID (name) 
char pass[] = "12345678"; // your network password
int status = WL_IDLE_STATUS; // the Wifi radio's status
void setup() {
  // initialise serial:
  Serial.begin(9600);
  // attempt to connect to Wifi network:
  while (status != WL_CONNECTED) {
    Serial.print("Attempting to connect to WPA SSID: ");
    Serial.println(ssid);
    // Connect to WPA/WPA2 network:
    status = WiFi.begin(ssid, pass);
    // wait 10 seconds for connection:
    delay(10000);
  }
  // you're connected now, so print out the data:
  Serial.println("You're connected to the network");
  // print your WiFi shield's IP address:
  IPAddress ip = WiFi.localIP();
  Serial.print("IP Address: ");
  Serial.println(ip);
}
void loop() {
  // do nothing
}

Now it’s time to run our sketch and make sure our Arduino can connect online. Upload the sketch to your board and open up the serial monitor. Once you’ve successfully connected then you’re ready to move on to the next section! Again, if you’re having problems at this stage, please check out the troubleshooting guide at the bottom of the tutorial.

2. Setting up OSC

Now that we’ve got our Arduino online, the next step is to get it sending and receiving OSC packets. The folks at the Center for New Music and Audio Technologies (CNMAT) have created a handy library that allows us to do this with ease. Download the zip file from their Github page here. Once downloaded, unzip the file and remove ‘-master’ from the ‘OSC-master’ folder name, leaving just ‘OSC’ (this is because the Arduino IDE apparently doesn’t allow certain characters in library names), then move this OSC folder into the Arduino’s ‘libraries’ folder. You should now be able to access the OSC library files and examples in the Arduino IDE.

Before we go any further, I think it’s important to explain a bit about how OSC works. When we send information via OSC we transmit little ‘packets’ of data, the contents of which being either an OSC ‘message’ or ‘bundle’. A message consists of an ‘address pattern’, as well as zero or more arguments. A bundle is simply a collection of multiple messages or other bundles.

The address pattern specifies where an OSC packet is being sent. Similar to a real-world address, each part of an address pattern narrows down its destination, for example: /country/city/street/houseNumber. Later you’ll see that /wek/inputs is the address we’ll use for our OSC packets to reach Wekinator’s input.

Finally, OSC packets are transported across a network through UDP (User Datagram Protocol), and so we must also include its functionality in our sketch. Hopefully this will all make a lot more sense in practice!

3. Setting up our Circuit

To demonstrate how we can send and receive information to and from Wekinator, we shall be building a simple circuit consisting of a light sensor, LED and a couple of resistors. The values from the light sensor will be our input and the brightness of the LED will ultimately be our output. With these, our goal is to train Wekinator so that we can control the brightness of the LED using the light sensor.

So first of all, construct the circuit as shown below:

Once you’ve done this, we’ll then add these components to our code under our existing properties:

int sensorPin = A0;
int ledPin = 5;

Note that our ledPin must be a PWM pin in order for our LED to fade. Next we’ll specify which pins are inputs and outputs in the setup() function. Add the following just after Serial.begin(9600);:

// initialise pins:
pinMode(sensorPin, INPUT);
pinMode(ledPin, OUTPUT);

Finally, in the loop() function, we’ll read the incoming value from our light sensor, storing it in a new float calledsensorValue. We’ll then print this value in the serial monitor:

// read value from light sensor:
float sensorValue = analogRead(sensorPin);
Serial.println(sensorValue);

And that’s it for now for our circuit setup! So at this point, your sketch should look like this:

// Libraries //
#include <SPI.h>
#include <WiFi.h> // or #include <WiFi101.h>
// Properties //
char ssid[] = "yourNetwork"; // your network SSID (name) 
char pass[] = "12345678"; // your network password
int status = WL_IDLE_STATUS; // the Wifi radio's status
int sensorPin = A0;
int ledPin = 5;
void setup() {
  // initialise serial:
  Serial.begin(9600);
  // initialise pins:
  pinMode(sensorPin, INPUT);
  pinMode(ledPin, OUTPUT);
  // attempt to connect to Wifi network:
  while (status != WL_CONNECTED) {
    Serial.print("Attempting to connect to WPA SSID: ");
    Serial.println(ssid);
    // Connect to WPA/WPA2 network:
    status = WiFi.begin(ssid, pass);
    // wait 10 seconds for connection:
    delay(10000);
  }
  // you're connected now, so print out the data:
  Serial.println("You're connected to the network");
  // print your WiFi shield's IP address:
  IPAddress ip = WiFi.localIP();
  Serial.print("IP Address: ");
  Serial.println(ip);
}
void loop() {
  // read value from light sensor:
  float sensorValue = analogRead(sensorPin);
  Serial.println(sensorValue);
}

Upload the sketch and check whether you’re successfully getting readings from your light sensor. Next we can moving onto sending these values via OSC.

4. Sending Inputs to Wekinator

So to transmit information via OSC we must first add the necessary libraries. Add the following libraries to your sketch:

#include <WiFiUdp.h>
#include <OSCMessage.h>

Next add the following properties:

WiFiUDP Udp;
IPAddress outIp(192, 168, 1, 17); // your computer's private IP
const unsigned int outPort = 6448;
const unsigned int inPort = 12000;

There’s a lot here to break down. First, we are declaring a new WiFiUDP object, naming it simply Udp. Next we declare an IPAddress, outIp, which will indicate the destination of our outgoing OSC messages. As they need to reach the Wekinator program on your computer, this will be your computer’s private IP address (which is different to your public IP address!).

Our outPort is the port the Arduino will be sending inputs from to Wekinator. It can be almost any number but it must match the input port of the destination application. We’ll use 6448, as this is Wekinator’s default input port. Conversely, our inPort is the port the Arduino will be receiving outputs from Wekinator. We’ll set this to 12000, as this is Wekinator’s default output port.

Next, in the setup() function, add the following underneath all the network connection setup code from earlier:

// initialise UDP:
Udp.begin(inPort);

In this we are initialising UDP, passing in our inPort (not outPort!), so that we can start sending and receiving.

Now we can move onto the loop() function in which we will create an OSC message and send our sensorValue. Add the following to the end of your loop() function:

// create an OSCMessage:
OSCMessage msgOUT("/wek/inputs");
msgOUT.add("sensorValue");
// send the OSC packet via UDP:
Udp.beginPacket(outIp, outPort);
msgOUT.send(Udp);
Udp.endPacket();
msgOUT.empty();
delay(20);

Again, there’s a lot going on here to break down. First, we’re declaring an OSC message, naming it msgOUT and adding the address /wek/inputs as its first argument. We then add the data we want to transmit, this will be the reading from our light sensor, sensorValue. Please note that Wekinator only accepts floats as inputs. Anything else will not be accepted by the program. To send our message we then tell UDP to initialise the packet, stating our outIp and outPort. We then send the packet using the OSC library’s send() function. Finally, we mark the end of the packet before emptying the OSC message to free space. We also add a short time delay at the end of our loop().

So here’s what your sketch should look like thus far:

// Libraries //
#include <SPI.h>
#include <WiFi.h> // or #include <WiFi101.h
#include <WiFiUdp.h>
#include <OSCMessage.h>
// Properties //
char ssid[] = "yourNetwork"; // your network SSID (name) 
char pass[] = "12345678"; // your network password
int status = WL_IDLE_STATUS; // the Wifi radio's status
int sensorPin = A0;
int ledPin = 5;
WiFiUDP Udp;
IPAddress outIp(192, 168, 1, 17); // your computer's private IP
const unsigned int outPort = 6448;
const unsigned int inPort = 12000;
void setup() {
  // initialise serial:
  Serial.begin(9600);
  // initialise pins:
  pinMode(sensorPin, INPUT);
  pinMode(ledPin, OUTPUT);
  // attempt to connect to Wifi network:
  while (status != WL_CONNECTED) {
    Serial.print("Attempting to connect to WPA SSID: ");
    Serial.println(ssid);
    // Connect to WPA/WPA2 network:
    status = WiFi.begin(ssid, pass);
    // wait 10 seconds for connection:
    delay(10000);
  }
  // you're connected now, so print out the data:
  Serial.println("You're connected to the network");
  // print your WiFi shield's IP address:
  IPAddress ip = WiFi.localIP();
  Serial.print("IP Address: ");
  Serial.println(ip);
  // initialise UDP:
  Udp.begin(inPort);
}
void loop() {
  // read value from light sensor:
  float sensorValue = analogRead(sensorPin);
  Serial.println(sensorValue);
  // create an OSCMessage:
  OSCMessage msgOUT("/wek/inputs");
  msgOUT.add(sensorValue);
  // send the OSC packet via UDP:
  Udp.beginPacket(outIp, outPort);
  msgOUT.send(Udp);
  Udp.endPacket();
  msgOUT.empty();
  delay(20);
}

Before we can go ahead and upload our sketch, we must first set up Wekinator to start listening in for our outgoing OSC messages. Download Wekinator if you haven’t already done so, and also download the ‘Continuous control of animation colour’ output program which you can find in their walkthrough page. We’ll use this to help test whether our inputs are reaching Wekinator.

Once you’ve opened Wekinator, you’ll see that the input port, output port and input OSC address are the same as what we declared, so no need to change any of those. The only thing we must do is change the #inputs and #outputs each to 1. So now your Wekinator project setup screen should look like this:

Once you hit ‘Next’ you’ll be greeted by Wekinator’s main project interface. Next, open up the colour output program. Once it loads up you’ll be able to change the colour by sliding the project interface’s ‘outputs-1' slider. Neat!

Now you can upload your sketch to your Arduino. If all has gone well, you should notice the ‘OSC In’ indicator in the project interface change from yellow to green. This means that Wekinator is successfully receiving inputs, huzzah!

So now that Wekinator can successfully receive inputs from our Arduino, we can now start to train our model. First drag the slider under ‘outputs-1’ to a desired position, in this case, I slid it all the way to 1. Next, place your hand over your Arduino’s light sensor. This will be our first state. With your hand covering the light sensor, hit ‘Start Recording’ for about a second to gather enough training examples.

Next move the ‘output-1’ slider back down to 0 and leave the light sensor uncovered. Hit ‘Start Recording’ in this new position, again for about a second.

Now that we’ve gathered samples of two different states, we can hit the ‘Train’ button to train our model. Once that’s completed hit the ‘Run’ button. Now, when we change the value of the light sensor, we should see that the colour changing program reacts accordingly. Congratulations! You have now successfully trained a model in Wekinator through inputs sent from Arduino.

5. Receiving Outputs from Wekinator

So far we’ve sent inputs from a light sensor to Wekinator to change the colour of the colour changing program, however the LED which we added to our circuit is still untouched. So next we’ll be setting up our Arduino to receive outputs from Wekinator, using them to alter the brightness of the LED.

First, add the following to your loop() function below the code that sends our OSC messages:

// receive OSC packet via UDP:
OSCMessage msgIN;
int size = Udp.parsePacket();
if (size > 0) {
  while (size--) {
    msgIN.fill(Udp.read());
  }
  if (!msgIN.hasError()) {
    msgIN.dispatch("/wek/outputs", changeLed);
  } 
}

So firstly we’re declaring another OSC message but this time for the incoming message, calling it msgIN. Next we create a new int called size, assigning it the size of the received packet. If this size is greater than zero, i.e. it’s not empty, we then fill our OSC message with its contents. Finally, if there are no errors, we then ‘dispatch’ the data found under the address /wek/outputs into our function changeLed , which we will now write. Below and outside the entire loop() function, create the following function:

// convert value and change LED brightness: 
void changeLed(OSCMessage &msg) {
  float wekValue = msg.getFloat(0);
  float ledBrightness = wekValue * 255;
  analogWrite(ledPin, ledBrightness);
}

We name the function changeLed, stating that it expects an argument of type OSCMessage. We then retrieve the value stored in the message passed into the function, assigning it to the float wekValue. Next, in order to make this value useable to control our LED, we must multiply it by 255. This is because the Arduino’s analog output ranges from 0–255, whereas the output from Wekinator only ranges between 0–1. We store the new value under the float ledBrightness. Finally, we use analogWrite() to pass the new value to our LED, altering its brightness.

So our final code should now look like this:

// Libraries //
#include <SPI.h>
#include <WiFi.h> // or #include <WiFi101.h
#include <WiFiUdp.h>
#include <OSCMessage.h>
// Properties //
char ssid[] = "yourNetwork"; // your network SSID (name) 
char pass[] = "12345678"; // your network password
int status = WL_IDLE_STATUS; // the Wifi radio's status
int sensorPin = A0;
int ledPin = 5;
WiFiUDP Udp;
IPAddress outIp(192, 168, 1, 17); // your computer's private IP
const unsigned int outPort = 6448;
const unsigned int inPort = 12000;
void setup() {
  // initialise serial:
  Serial.begin(9600);
  // initialise pins:
  pinMode(sensorPin, INPUT);
  pinMode(ledPin, OUTPUT);
  // attempt to connect to Wifi network:
  while (status != WL_CONNECTED) {
    Serial.print("Attempting to connect to WPA SSID: ");
    Serial.println(ssid);
    // Connect to WPA/WPA2 network:
    status = WiFi.begin(ssid, pass);
  // wait 10 seconds for connection:
    delay(10000);
  }
  // you're connected now, so print out the data:
  Serial.println("You're connected to the network");
  // print your WiFi shield's IP address:
  IPAddress ip = WiFi.localIP();
  Serial.print("IP Address: ");
  Serial.println(ip);
  // initialise UDP:
  Udp.begin(inPort);
}
void loop() {
  // read value from light sensor:
  sensorValue = analogRead(sensorPin);
  Serial.println(sensorValue);
  // create an OSCMessage:
  OSCMessage msgOUT("/wek/inputs");
  msgOUT.add(sensorValue);
  // send the OSC packet via UDP:
  Udp.beginPacket(outIp, outPort);
  msgOUT.send(Udp);
  Udp.endPacket();
  msgOUT.empty();
// receive OSC packet via UDP:
  OSCMessage msgIN;
  int size = Udp.parsePacket();
  if (size > 0) {
    while (size--) {
      msgIN.fill(Udp.read());
    }
    if (!msgIN.hasError()) {
      msgIN.dispatch("/wek/outputs", changeLed);
    } 
  }
  delay(20);
}
// convert value and change LED brightness: 
void changeLed(OSCMessage &msg) {
  float wekValue = msg.getFloat(0);
  float ledBrightness = wekValue * 255;
  analogWrite(ledPin, ledBrightness);
}

Again, before we upload this, we must setup a new Wekinator project. Close the previous project and start a new one. Similarly to before, change the #inputs and #outputs to 1 on the project setup screen. This time, under the outputs section, change the ‘Host (IP address or name)’ to the IP address of your Arduino. Wekinator will need this in order to determine where to send its outputs to. As explained before, this IP address is printed in the serial monitor every time your Arduino connects online. So now your project setup screen should now show this:

Hit ‘Next’ to go to the main project interface and then upload your sketch to your Arduino. If all has gone well, when you move the ‘outputs-1’ slider, you should see the brightness of your LED respond. And like before, the ‘OSC In’ indicator should change from yellow to green, meaning that Wekinator is successfully receiving data from our Arduino.

So now that Wekinator can successfully send and receive data to and from our Arduino, we can now train our model similarly to before. First, drag the ‘outputs-1' slider to your desired position, place your hand over your Arduino’s light sensor and hit ‘Start Recording’ for about a second.

Next move the ‘output-1’ slider back down to 0, leave the light sensor uncovered and record another set of samples.

With our samples gathered in these two states, we can hit the ‘Train’ button. Finally, once training has completed, hit the ‘Run’ button and witness the grand result! Now, when we change the value of the light sensor, our LED’s brightness reacts accordingly. Congratulations! You can now send, receive and use data between Wekinator and Arduino!

Beyond the Tutorial

So in this tutorial we’ve learnt how to connect an Arduino together with Wekinator to send and receive values. Although one may argue that the end result of the exercise could easily be produced through Arduino alone, we have only skimmed the potential of what these programs can do. In addition to sending and receiving values, we can also control Wekinator entirely through Arduino by sending commands via OSC, which you can read more about here. The potential of these two programs also expands once you consider applications such as gesture recognition, the use of multiple sensors in your circuit, or connecting to other OSC devices like gaming controllers or musical instruments.

I hope this tutorial has not only equipped you to apply Wekinator to your own projects but that it has also inspired you to explore, experiment and learn more about machine learning on a deeper level.

Troubleshooting Guide

Please also check out Wekinator’s troubleshooting guide for more help.

I cannot connect my Arduino online

• Incorrect network name and password: Sounds obvious but it happens!
• Missing or incorrect library: Make sure whether it’s the Wifi.h or Wifi101.h library which your Arduino requires.
• Dodgy ethernet shields: Many unofficial ethernet shields have a defect which is mendable. You can see the issue and fix for this here.

My inputs are not reaching Wekinator

• Sensor values are not being read properly: Check that you’re getting readings from your sensor and that it isn’t the circuit at fault.
• outIp address is incorrect: Again, make sure this is your computer’s private IP address, not your public IP address! If you just Googled “ip address” that’s your public one!
• OSC address is incorrect: Make sure it is /wek/inputs don’t forget the forward slash at the start!
• Input port is incorrect: The input port should be 6448, whereas the output port is 12000, not the other way round!
• Not sending a float: Wekinator will only accept values of type float. If sensorValue is declared as an int, it will not work!

My Arduino is not receiving outputs from Wekinator

• IP address incorrect in Wekinator: Make sure that you’ve added your Arduino’s private IP address, under ‘Host (IP address or name)’ in Wekinator’s project setup screen.