Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

For information about the various ways we communicate with each other, see our Get Involved wiki page.

Getting Started

Make sure to read through the Getting Started on GSoC page for information on joining the Fluid community and preparing to work on a GSoC project.

Projects

Project Title

Project Description:

Prototype Tasks:

Related Issues:

Difficulty: 

Mentor: 

IRC: 

Skills required: 

...

Cross-Platform Audio Engine for Node.js

Node.js is a library for developing server-side and desktop applications using JavaScript. The Fluid Project community uses Node.js for (among other things) developing applications for small, resource-constrained devices such as the C.H.I.P. and the Raspberry Pi. In particular, we are developing sonification and sound processing applications to support inclusive learning and culture. However, there is currently no reliable, cross-platform, low latency solution for generating audio using Node.js. One popular library, node-speaker, is only sporadically maintained and has very high latency and CPU usage, making it unsuitable for real-time audio streams.

This project will entail the development of a new Node.js native module in C++ that uses an existing cross-platform audio API such as Port Audio, libsoundio, or OpenAL. This module should support macOS, Windows, and Linux. It should include the following features:

  1. The ability to enumerate and list audio devices
  2. The ability to specify the sample formats, including integer and floating point numbers at 8, 16, 24, and 32-bit resolution
  3. Configurable buffer sizes/latency, including reliable support for < 10ms latency
  4. A callback-based API that allows JavaScript code to provide blocks of raw audio samples 


Difficulty: Medium

Mentor: Colin Clark

IRC: colinclark

Skills: Knowledge of C++; basic familiarity with Node.js and how native modules are written for Node.js

How to get started/getting to know us: Prospective students who are interested in this project should spend some time familiarizing themselves with writing native Node.js modules in C++ using Native Abstractions for Node.js (NAN). The NAN project provides excellent tutorials and background information, as does the Node.js documentation. To familiarize yourself with the requirements of the project, you may want to consider developing a very simple native module for Node.js. You should also familiarize yourself with various cross-platform audio libraries, and write a basic example client that generates some simple audio output using Port Audio, libsoundio, or a similar library.

JavaScript SoundFont 2 Parser and Synthesizer Engine

SoundFonts provide a means for packaging and distributing audio samples for use in wavetable synthesizers and samplers. They typically provide a variety of instrument sounds sampled at different pitches and octaves, making it easy to create realistic-sounding digital instruments. SoundFonts are particularly useful for data sonification, since they provide a simple and low-cost way to give users the ability to choose from a variety of instrumental sounds when creating their sound designs.

The Floe Project (Flexible Learning for Open Education) is developing new tools for sonification and data presentation using audio. These tools are based on Flocking, a framework for audio signal processing, synthesis, and music composition, which uses the Web Audio APIs now built into most modern web browsers.

However, neither Flocking nor the Web Audio API currently provides any support for parsing or playing back SoundFont-based instruments. While there are JavaScript libraries (such as MIDI.js and the soundfont-player library) that claim to provide SoundFont support, these are all based on an approach that extracts the sound files from a SoundFont with a fixed duration and envelope, significantly limiting the usefulness and expressiveness of the SoundFonts.

This project will entail the development of a robust, pure JavaScript SoundFont 2 parser as well as a Flocking unit generator that is capable of expressively playing a SoundFont. The student may choose to take the sf2-parser library written by Gree and modified by Colin Clark as the basis of the project. The parser should support all the major features of the SoundFont 2 specification, and should be able to parse standard .sf2 files and expose their contents in a data-oriented, JSON-style data structure. The playback unit generator should provide inputs that can control and modulate all of the essential parameters of a SoundFont. All code written should be accompanied by unit tests. Along the way, the student is encouraged to create an awesome musical demo of their work.


Difficulty: Medium

Mentor: Colin Clark

IRC: colinclark

Skills required: In-depth knowledge of JavaScript and web development. Familiarity with digital audio and MIDI.

How to get started/getting to know us: Prospective students who are interested in this project should spend some time familiarizing themselves with the SoundFont 2 specification and the sf2-parser library.

Add Rate Limiting Control for AChecker Web Service API

Project Description: AChecker is an online accessibility validator that provides a web service API for users to validate the given URL or content via http requests. Each user is given a web service ID that can be sent along with http requests to use the API. To avoid the abuse of AChecker web service API, a rate limit limiting needs to be added to control how many the number of requests can be processed made at a certain period by one web service ID. Adding this feature also requires changes to UIs:

...

:

  1. Implement the rate limiting;
  2. Add HTTP endpoints for querying rate limits, revoking and regenerating IDs etc;
  3. Change the user profile UI ; to allow users to revoke/delete the compromised web service IDs and regenerate new IDs.;
  4. Change the AChecker Administrator's user management UI ; to allow administrators to revoke the compromised web service IDs.

...

Skills required: PHP, Javascript, HTML, CSS. The experience of use and the understanding of the API authentication and authorization rate limiting would be a bonus.

Automated Test Suite for AChecker

...