harry van haaren, openav, lac 2017 - jean monnet university · drum-pad controller spring '16...

31
Harry van Haaren, OpenAV, LAC 2017

Upload: others

Post on 30-Apr-2020

5 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

Harry van Haaren, OpenAV, LAC 2017

Page 2: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

CTLRA

1) Modern Controllers

2) Why a library

3) Users and Mappings

4) Demo!

5) Bonus Stuff...

Page 3: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

Modern Controllers

Page 4: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

Modern Controllers

"Next Gen" Controllers!

● USB HID devices

● "HD" Screens

● Huge functionality!

● Application support…

Page 5: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

Modern Controllers

MIDI Mapping!

● APC40 released in 2014

○ USB MIDI device

○ Can't do full updates

Page 6: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

Modern Controllers

HD Screens?

● MIDI messages

○ SysEx?

○ No thanks :)

Page 7: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

Modern Controllers

Need for a better API

● Software ⇔ Hardware

○ Easier communication

○ Faster to support modern features

● Required for Tight Integration

○ Input Multiplexing

Page 8: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

Why A Library?

Page 9: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID
Page 10: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

A Library for Controller Access

Developer implements library support

● Growing list of devices supported

● New devices added are "free"

● Don't re-implement device support

○ Testing is a time-sink!

Page 11: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

Generic Event API

Abstract away the Device

● Input from Events

○ Button, Slider, Encoder, Grid

● LED feedback function

● API to blit pixels to a screen

Page 12: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

Linux Support

Community Supported Devices

● Once-off implementations of little value

○ (Unless user has that exact controller)

● Centralize support in a library

○ Enabled by "Generic Events"

Page 13: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

Fancy Features

Hotplug

● Essential to save a Musician on stage

● Difficult to implement

○ Time consuming to test

Page 14: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID
Page 15: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID
Page 16: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

Users and Mappings

Page 17: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

Mappings and User-eXperience

Power User

● Can "script" controls as required

● Has huge flexibility and power

● Creates awesome mappings

Simple Workflow

● Can use software provided mapping

○ Just like MIDI mapping

● Not as powerful, but achieves goal

○ Mapping exists? Use it!

Page 18: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

Power User

Uniting Software and Hardware

● Match functionality of controller to software

○ Just as the user requires

○ Depends on Hardware / Software combo

○ Depends on Musicians Workflow

Page 19: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

Timeline (Just for fun :)

Page 20: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

Jan '16

Bought a USB HID drum-pad controller

Spring '16

Integrated drum-pad and Fabla 2.0

miniLAC '16

Demo Fabla 2.0 with a specific USB HID device

Summer '16

Understand requirements and design v1 "ctlr" API

Autumn '16

Implement v1 and discuss with other developers, design v2

Page 21: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

Dec '16

Development of Ctlra codebase

Jan '17

Finish initial implementation Write Ctlra paper for LAC

Feb '17

Mixxx integration and testing of v2 Ctlra API

(Specifically Hotplug)

Mar/Apr '17

Minor updates to API, Build out demo apps

LAC '17

Ctlra library release!

Page 22: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

Demos!

Page 23: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

Demo

1. Simple Events

2. Simple Feedback

3. Tight Integration

4. Scripting in C

5. DJ Hotplug

Page 24: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

Next Steps

1. Discuss Event Loops

2. Integrate with LV2 Atoms

3. Get Applications using it

4. Discuss "sharing" of devices

5. Support for BlueTooth, MIDI, OSC, Arduino, Serial etc..

Page 25: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

? Questions ?

Page 26: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

Bonus Demo AVTK + Ctlra

Page 27: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

Bonus Slides! Woop Woop :)

Page 28: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

Event Loops

How to manage threading / event handling

● Current API has idle() function

○ Must be called periodically

● Expose events via ringbuffer?

○ How to handle hotplug of new device.. New ring?

Page 29: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

LV2 Atom Integration

Ctlra Events map to LV2 Atoms quite nicely...

● Could the host pass these events "through" to the plugin?

○ Abstract the environment from the plugin?

○ Enable "sharing" of a device?

● Requires "Options" extension function to accept new device?

Page 30: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

USB Device Access

LibUSB

● Enables bulk, interrupt, isochronous, control endpoints

● API provides Sync and ASync modes

○ Async required for latency with multiple devices

HID Raw

● Won't support all devices (Screens are often USB "bulk" endpoints)

● Works better for certain devices..?

Page 31: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID

Timestamps

Arrival time of Event?

● Is it required at library level?

○ Time, Threading and ASync events… Fun!

● Expose an application defined callback to "set" the timestamp?