Processing with Kinect&GUI

I. Experiment Preparation

Experimental Tools: Processing 3.5

Experimental Equipment: Kinect v2.0

Dependencies: ControlP5, Kinect v2 for processing, PixelFlow

II. Experiment Steps

1. Use Processing to build an example system, to simulate particle generation and disturbance.

2. Implement a particle interaction mechanism via the mouse.

3. Recognize hand status through Kinect.

4. Design interactions based on the information recognized by Kinect.

III. Relevant Introduction

Processing is a forward-looking, emerging computer language with a revolutionary concept: introducing programming languages in the context of electronic arts and introducing electronic arts concepts to programmers. It is an extension of the Java language and supports many of Java’s existing frameworks. However, it is much simpler in syntax and features many thoughtful and user-friendly designs. Processing can be used on operating systems such as Windows, MAC OS X, MAC OS 9, Linux, etc. The latest version is Processing 3.5.4. Works completed with Processing can function on a personal local machine or be exported to the web in the form of Java Applets.

Although Graphical User Interfaces (GUIs) became mainstream over twenty years ago, the teaching of basic programming languages still primarily uses command-line interfaces to this day. Why should learning a programming language be so dull? The human brain is naturally good at spatial recognition, and GUIs take advantage of this strength. Coupled with their ability to provide various real-time and vivid visual feedback, GUIs can significantly shorten the learning curve and help understand abstract logical principles. For example, a pixel on a computer screen is a visual representation of the value of a variable. Processing simplifies Java syntax and “sensualizes” its computational results, allowing users to quickly enjoy multimedia works with both sound and light.

The source code of Processing is open, much like the popular Linux operating system, Mozilla browser, or the Perl language. Users can freely tailor the most suitable usage pattern according to their needs. The applications of Processing are very diverse, and they all adhere to open-source rules. This design greatly increases the interaction and learning efficiency of the entire community.

Kinect is a body-sensing peripheral for the XBOX360 that Microsoft officially announced at the E3 Expo on June 2, 2009. Kinect completely revolutionizes the single-mode game operation and thoroughly embodies the concept of human-computer interaction.

It is a 3D motion-sensing camera (development codename “Project Natal”), integrating real-time motion capture, image recognition, microphone input, voice recognition, and community interaction functionalities. Players can use this technology to drive in games, interact with other players, and share pictures and information with other Xbox players through the internet.

Microsoft’s Kinect does not require any controller; it relies on a camera to capture the player’s movements in three-dimensional space. Microsoft points out that it will make the system easier to operate to attract the general public.

IV. Experiment Process

Use Processing to build an example system to simulate the generation and disturbance of particles.

Adjustments to the particle parameters in the interface can be made through the UI.

Control the lifecycle of the disappearing particles.

Design Interaction:

Left-click to disturb particles

Right-click to generate particles

Middle-click to create a cluster of particles

The code of interaction design is as follows:

if(mouse_input && mouseButton == LEFT){
  radius = 15;
  vscale = 15;
  px     = mouseX;
  py     = height-mouseY;
  vx     = (mouseX - pmouseX) * +vscale;
  vy     = (mouseY - pmouseY) * -vscale;



  fluid.addDensity (px, py, radius, 0.25f, 0.0f, 0.1f, 1.0f);
  fluid.addVelocity(px, py, radius, vx, vy);
  particles.spawn(fluid, px, py, radius*2, 300);
}    

// add impulse: density + velocity, particles
if(mouse_input && mouseButton == LEFT){
  radius = 15;
  vscale = 15;
  px     = mouseX;
  py     = height-mouseY;
  vx     = (mouseX - pmouseX) * +vscale;
  vy     = (mouseY - pmouseY) * -vscale;
  fluid.addDensity (px, py, radius, 0.25f, 0.0f, 0.1f, 1.0f);
  fluid.addVelocity(px, py, radius, vx, vy);
  particles.spawn(fluid, px, py, radius*2, 300);
}

// add impulse: density + temperature, particles
if(mouse_input && mouseButton == CENTER){
  radius = 15;
  vscale = 15;
  px     = mouseX;
  py     = height-mouseY;
  temperature = 2f;
  fluid.addDensity(px, py, radius, 0.25f, 0.0f, 0.1f, 1.0f);
  fluid.addTemperature(px, py, radius, temperature);
  particles.spawn(fluid, px, py, radius, 100);
}

// particles
if(mouse_input && mouseButton == RIGHT){
  px     = mouseX;
  py     = height - 1 - mouseY; // invert
  radius = 50;
  particles.spawn(fluid, px, py, radius, 300);
}

Test Kinect data.

Implement hand interaction by reading Kinect skeletal data to obtain hand information and status.

Spread hand to disturb particles.

Close hand to generate particles.

The code of interaction design is as follows.

switch(hstate){
  case KinectPV2.HandState_Closed :
    fluid.addDensity(tempX, tempY, radius, 0.2f, 0.3f, 0.5f, 1.0f);
    fluid.addTemperature(tempX, tempY, radius, temperature);
    particles.spawn(fluid, tempX, tempY, radius, 100);
    break;
  case KinectPV2.HandState_Open :
    radius = 15;
    vscale = 15;
    vx     = +3*vscale;
    vy     = -3*vscale;
    px     = tempX;
    py     = tempY;
    fluid.addDensity (px, py, radius, 0.25f, 0.0f, 0.1f, 1.0f);
    fluid.addVelocity(px, py, radius, vx, vy);
    particles.spawn(fluid, px, py, radius*2, 300);
  break;
}

V. Experiment Experience

This experiment utilized Processing to implement Kinect human-computer interaction, designed a refined UI, and used particle effects to detect human hand gestures. The corresponding particle effects can be changed. However, there is room for improvement, as more functions and designs can be developed. I hope to continuously perfect this in my subsequent studies.

VI. Experiment Code

Fluid CustomParticles:

import com.thomasdiewald.pixelflow.java.DwPixelFlow;
import com.thomasdiewald.pixelflow.java.fluid.DwFluid2D;

import controlP5.Accordion;
import controlP5.ControlP5;
import controlP5.Group;
import controlP5.RadioButton;
import controlP5.Toggle;
import processing.core.*;
import processing.opengl.PGraphics2D;
import processing.opengl.PJOGL;

import java.util.ArrayList;
import KinectPV2.KJoint;
import KinectPV2.*;

KinectPV2 kinect;
private class MyFluidData implements DwFluid2D.FluidData{
  
  // update() is called during the fluid-simulation update step.
  @Override
  public void update(DwFluid2D fluid) {
    float tempX,tempY,tempX2,tempY2,px, py, vx, vy, radius, vscale, temperature;
    int hstate;
    int hstate2;
    radius = 15;
    vscale = 10;
    px    = width/2;
    tempX = width/2; 
    tempX2 = width/2; 
    py     = 50;
    tempY  = 50; 
    tempY2  = 50;
    vx     = 1 * +vscale;
    vy     = 1 *  vscale;
    radius = 40;
    temperature = 1f;
    hstate = KinectPV2.HandState_NotTracked;
    hstate2 = KinectPV2.HandState_NotTracked;

    ArrayList<KSkeleton> skeletonArray =  kinect.getSkeletonDepthMap();
    for (int i = 0; i < skeletonArray.size(); i++) 
    {
      KSkeleton skeleton = (KSkeleton) skeletonArray.get(i);
      //if the skeleton is being tracked compute the skleton joints
      if (skeleton.isTracked()) {
        KJoint[] joints = skeleton.getJoints();
        KJoint RightHand = joints[KinectPV2.JointType_HandRight];
        KJoint LeftHand = joints[KinectPV2.JointType_HandLeft];
        tempX = RightHand.getX();
        tempY = height - RightHand.getY();
        hstate = RightHand.getState();

        tempX2 = LeftHand.getX();
        tempY2 = height - LeftHand.getY();
        hstate2 = LeftHand.getState();
      }
    }


    fluid.addDensity(px, py, radius, 0.2f, 0.3f, 0.5f, 1.0f);
    fluid.addTemperature(px, py, radius, temperature);
    particles.spawn(fluid, px, py, radius, 100);
    
    boolean mouse_input = !cp5.isMouseOver() && mousePressed;
    switch(hstate2){
      case KinectPV2.HandState_Closed :
        fluid.addDensity(tempX2, tempY2, radius, 0.2f, 0.3f, 0.5f, 1.0f);
        fluid.addTemperature(tempX2, tempY2, radius, temperature);
        particles.spawn(fluid, tempX2, tempY2, radius, 100);
        break;
      case KinectPV2.HandState_Open :
        radius = 15;
        vscale = 15;
        vx     = +3*vscale;
        vy     = -3*vscale;
        px     = tempX2;
        py     = tempY2;
        fluid.addDensity (px, py, radius, 0.25f, 0.0f, 0.1f, 1.0f);
        fluid.addVelocity(px, py, radius, vx, vy);
        particles.spawn(fluid, px, py, radius*2, 300);
      break;
    }

    switch(hstate){
      case KinectPV2.HandState_Closed :
        fluid.addDensity(tempX, tempY, radius, 0.2f, 0.3f, 0.5f, 1.0f);
        fluid.addTemperature(tempX, tempY, radius, temperature);
        particles.spawn(fluid, tempX, tempY, radius, 100);
        break;
      case KinectPV2.HandState_Open :
        radius = 15;
        vscale = 15;
        vx     = +3*vscale;
        vy     = -3*vscale;
        px     = tempX;
        py     = tempY;
        fluid.addDensity (px, py, radius, 0.25f, 0.0f, 0.1f, 1.0f);
        fluid.addVelocity(px, py, radius, vx, vy);
        particles.spawn(fluid, px, py, radius*2, 300);
      break;
    }

    if(mouse_input && mouseButton == LEFT){
      radius = 15;
      vscale = 15;
      px     = mouseX;
      py     = height-mouseY;
      vx     = (mouseX - pmouseX) * +vscale;
      vy     = (mouseY - pmouseY) * -vscale;



      fluid.addDensity (px, py, radius, 0.25f, 0.0f, 0.1f, 1.0f);
      fluid.addVelocity(px, py, radius, vx, vy);
      particles.spawn(fluid, px, py, radius*2, 300);
    }    
    
    // add impulse: density + velocity, particles
    if(mouse_input && mouseButton == LEFT){
      radius = 15;
      vscale = 15;
      px     = mouseX;
      py     = height-mouseY;
      vx     = (mouseX - pmouseX) * +vscale;
      vy     = (mouseY - pmouseY) * -vscale;
      fluid.addDensity (px, py, radius, 0.25f, 0.0f, 0.1f, 1.0f);
      fluid.addVelocity(px, py, radius, vx, vy);
      particles.spawn(fluid, px, py, radius*2, 300);
    }
    
    // add impulse: density + temperature, particles
    if(mouse_input && mouseButton == CENTER){
      radius = 15;
      vscale = 15;
      px     = mouseX;
      py     = height-mouseY;
      temperature = 2f;
      fluid.addDensity(px, py, radius, 0.25f, 0.0f, 0.1f, 1.0f);
      fluid.addTemperature(px, py, radius, temperature);
      particles.spawn(fluid, px, py, radius, 100);
    }
    
    // particles
    if(mouse_input && mouseButton == RIGHT){
      px     = mouseX;
      py     = height - 1 - mouseY; // invert
      radius = 50;
      particles.spawn(fluid, px, py, radius, 300);
    }
      
  }
}



int viewport_w = 512;
int viewport_h = 424;
int viewport_x = 230;
int viewport_y = 0;

int gui_w = 200;
int gui_x = 20;
int gui_y = 20;
KJoint[] joints;
int fluidgrid_scale = 3;

DwFluid2D fluid;

// render targets
PGraphics2D pg_fluid;
//texture-buffer, for adding obstacles
PGraphics2D pg_obstacles;

// custom particle system
MyParticleSystem particles;

// some state variables for the GUI/display
int     BACKGROUND_COLOR           = 0;
boolean UPDATE_FLUID               = true;
boolean DISPLAY_FLUID_TEXTURES     = false;
boolean DISPLAY_FLUID_VECTORS      = false;
int     DISPLAY_fluid_texture_mode = 0;
boolean DISPLAY_PARTICLES          = true;


public void settings() {
  size(viewport_w, viewport_h, P2D);
  smooth(4);
  PJOGL.profile = 3;
  
}


public void setup() {
  
  kinect = new KinectPV2(this);
  kinect.enableDepthMaskImg(true);
  kinect.enableSkeletonDepthMap(true);
  kinect.init();
 
  surface.setLocation(viewport_x, viewport_y);
  
  // main library context
  DwPixelFlow context = new DwPixelFlow(this);
  context.print();
  context.printGL();

  // fluid simulation
  fluid = new DwFluid2D(context, viewport_w, viewport_h, fluidgrid_scale);

  // set some simulation parameters
  fluid.param.dissipation_density     = 0.999f;
  fluid.param.dissipation_velocity    = 0.99f;
  fluid.param.dissipation_temperature = 0.80f;
  fluid.param.vorticity               = 0.10f;
  
  // interface for adding data to the fluid simulation
  MyFluidData cb_fluid_data = new MyFluidData();
  fluid.addCallback_FluiData(cb_fluid_data);
  
  // pgraphics for fluid
  pg_fluid = (PGraphics2D) createGraphics(viewport_w, viewport_h, P2D);
  pg_fluid.smooth(4);
  pg_fluid.beginDraw();
  pg_fluid.background(BACKGROUND_COLOR);
  pg_fluid.endDraw();
  
      // pgraphics for obstacles
  pg_obstacles = (PGraphics2D) createGraphics(viewport_w, viewport_h, P2D);
  pg_obstacles.smooth(4);
  pg_obstacles.beginDraw();
  pg_obstacles.clear();
  float radius;
  radius = 200;
  pg_obstacles.stroke(64);
  pg_obstacles.strokeWeight(10);
  pg_obstacles.noFill();
  pg_obstacles.rect(1*width/2f,  1*height/4f, radius, radius, 20);
  // border-obstacle
  pg_obstacles.strokeWeight(20);
  pg_obstacles.stroke(64);
  pg_obstacles.noFill();
  pg_obstacles.rect(0, 0, pg_obstacles.width, pg_obstacles.height);
  pg_obstacles.endDraw();
  
  fluid.addObstacles(pg_obstacles);
  
  // custom particle object
  particles = new MyParticleSystem(context, 1000 * 1000);

  createGUI();
  
  background(0);
  frameRate(60);
}


public void draw() {    
  // update simulation
  if(UPDATE_FLUID){
    fluid.addObstacles(pg_obstacles);
    fluid.update();
    particles.update(fluid);
  }
  
  // clear render target
  pg_fluid.beginDraw();
  pg_fluid.background(BACKGROUND_COLOR);
  pg_fluid.endDraw();
  
  image(kinect.getDepthMaskImage(), 0, 0);


  // render fluid stuff
  if(DISPLAY_FLUID_TEXTURES){
    // render: density (0), temperature (1), pressure (2), velocity (3)
    fluid.renderFluidTextures(pg_fluid, DISPLAY_fluid_texture_mode);
  }
  
  if(DISPLAY_FLUID_VECTORS){
    // render: velocity vector field
    fluid.renderFluidVectors(pg_fluid, 10);
  }
  
  if( DISPLAY_PARTICLES){
    // render: particles; 0 ... points, 1 ...sprite texture, 2 ... dynamic points
    particles.render(pg_fluid, BACKGROUND_COLOR);
  }
  

  // display
  image(pg_fluid    , 0, 0);
  image(pg_obstacles, 0, 0);
  
}





public void fluid_resizeUp(){
  fluid.resize(width, height, fluidgrid_scale = max(1, --fluidgrid_scale));
}
public void fluid_resizeDown(){
  fluid.resize(width, height, ++fluidgrid_scale);
}
public void fluid_reset(){
  fluid.reset();
}
public void fluid_togglePause(){
  UPDATE_FLUID = !UPDATE_FLUID;
}
public void fluid_displayMode(int val){
  DISPLAY_fluid_texture_mode = val;
  DISPLAY_FLUID_TEXTURES = DISPLAY_fluid_texture_mode != -1;
}
public void fluid_displayVelocityVectors(int val){
  DISPLAY_FLUID_VECTORS = val != -1;
}

public void fluid_displayParticles(int val){
  DISPLAY_PARTICLES = val != -1;
}

public void keyReleased(){
  if(key == 'p') fluid_togglePause(); // pause / unpause simulation
  if(key == '+') fluid_resizeUp();    // increase fluid-grid resolution
  if(key == '-') fluid_resizeDown();  // decrease fluid-grid resolution
  if(key == 'r') fluid_reset();       // restart simulation
  
  if(key == '1') DISPLAY_fluid_texture_mode = 0; // density
  if(key == '2') DISPLAY_fluid_texture_mode = 1; // temperature
  if(key == '3') DISPLAY_fluid_texture_mode = 2; // pressure
  if(key == '4') DISPLAY_fluid_texture_mode = 3; // velocity
  
  if(key == 'q') DISPLAY_FLUID_TEXTURES = !DISPLAY_FLUID_TEXTURES;
  if(key == 'w') DISPLAY_FLUID_VECTORS  = !DISPLAY_FLUID_VECTORS;
}


//draw the body
void drawBody(KJoint[] joints) {
  drawBone(joints, KinectPV2.JointType_Head, KinectPV2.JointType_Neck);
  drawBone(joints, KinectPV2.JointType_Neck, KinectPV2.JointType_SpineShoulder);
  drawBone(joints, KinectPV2.JointType_SpineShoulder, KinectPV2.JointType_SpineMid);
  drawBone(joints, KinectPV2.JointType_SpineMid, KinectPV2.JointType_SpineBase);
  drawBone(joints, KinectPV2.JointType_SpineShoulder, KinectPV2.JointType_ShoulderRight);
  drawBone(joints, KinectPV2.JointType_SpineShoulder, KinectPV2.JointType_ShoulderLeft);
  drawBone(joints, KinectPV2.JointType_SpineBase, KinectPV2.JointType_HipRight);
  drawBone(joints, KinectPV2.JointType_SpineBase, KinectPV2.JointType_HipLeft);

  // Right Arm
  drawBone(joints, KinectPV2.JointType_ShoulderRight, KinectPV2.JointType_ElbowRight);
  drawBone(joints, KinectPV2.JointType_ElbowRight, KinectPV2.JointType_WristRight);
  drawBone(joints, KinectPV2.JointType_WristRight, KinectPV2.JointType_HandRight);
  drawBone(joints, KinectPV2.JointType_HandRight, KinectPV2.JointType_HandTipRight);
  drawBone(joints, KinectPV2.JointType_WristRight, KinectPV2.JointType_ThumbRight);

  // Left Arm
  drawBone(joints, KinectPV2.JointType_ShoulderLeft, KinectPV2.JointType_ElbowLeft);
  drawBone(joints, KinectPV2.JointType_ElbowLeft, KinectPV2.JointType_WristLeft);
  drawBone(joints, KinectPV2.JointType_WristLeft, KinectPV2.JointType_HandLeft);
  drawBone(joints, KinectPV2.JointType_HandLeft, KinectPV2.JointType_HandTipLeft);
  drawBone(joints, KinectPV2.JointType_WristLeft, KinectPV2.JointType_ThumbLeft);

  // Right Leg
  drawBone(joints, KinectPV2.JointType_HipRight, KinectPV2.JointType_KneeRight);
  drawBone(joints, KinectPV2.JointType_KneeRight, KinectPV2.JointType_AnkleRight);
  drawBone(joints, KinectPV2.JointType_AnkleRight, KinectPV2.JointType_FootRight);

  // Left Leg
  drawBone(joints, KinectPV2.JointType_HipLeft, KinectPV2.JointType_KneeLeft);
  drawBone(joints, KinectPV2.JointType_KneeLeft, KinectPV2.JointType_AnkleLeft);
  drawBone(joints, KinectPV2.JointType_AnkleLeft, KinectPV2.JointType_FootLeft);

  //Single joints
  drawJoint(joints, KinectPV2.JointType_HandTipLeft);
  drawJoint(joints, KinectPV2.JointType_HandTipRight);
  drawJoint(joints, KinectPV2.JointType_FootLeft);
  drawJoint(joints, KinectPV2.JointType_FootRight);

  drawJoint(joints, KinectPV2.JointType_ThumbLeft);
  drawJoint(joints, KinectPV2.JointType_ThumbRight);

  drawJoint(joints, KinectPV2.JointType_Head);
}

//draw a single joint
void drawJoint(KJoint[] joints, int jointType) {
  pushMatrix();
  translate(joints[jointType].getX(), joints[jointType].getY(), joints[jointType].getZ());
  ellipse(0, 0, 25, 25);
  popMatrix();
}

//draw a bone from two joints
void drawBone(KJoint[] joints, int jointType1, int jointType2) {
  pushMatrix();
  translate(joints[jointType1].getX(), joints[jointType1].getY(), joints[jointType1].getZ());
  ellipse(0, 0, 25, 25);
  popMatrix();
  line(joints[jointType1].getX(), joints[jointType1].getY(), joints[jointType1].getZ(), joints[jointType2].getX(), joints[jointType2].getY(), joints[jointType2].getZ());
}

//draw a ellipse depending on the hand state
void drawHandState(KJoint joint) {
  noStroke();
  handState(joint.getState());
  pushMatrix();
  translate(joint.getX(), joint.getY(), joint.getZ());
  println(joint.getX(), joint.getY(), joint.getZ());
  ellipse(0, 0, 70, 70);
  popMatrix();
}

/*
Different hand state
 KinectPV2.HandState_Open
 KinectPV2.HandState_Closed
 KinectPV2.HandState_Lasso
 KinectPV2.HandState_NotTracked
 */

//Depending on the hand state change the color
void handState(int handState) {
  switch(handState) {
  case KinectPV2.HandState_Open:
    fill(0, 255, 0);
    break;
  case KinectPV2.HandState_Closed:
    fill(255, 0, 0);
    break;
  case KinectPV2.HandState_Lasso:
    fill(0, 0, 255);
    break;
  case KinectPV2.HandState_NotTracked:
    fill(100, 100, 100);
    break;
  }
}



ControlP5 cp5;

public void createGUI(){
  cp5 = new ControlP5(this);
  
  int sx, sy, px, py, oy;
  
  sx = 100; sy = 14; oy = (int)(sy*1.5f);
  

  ////////////////////////////////////////////////////////////////////////////
  // GUI - FLUID
  ////////////////////////////////////////////////////////////////////////////
  Group group_fluid = cp5.addGroup("fluid");
  {
    group_fluid.setHeight(20).setSize(gui_w, 300)
    .setBackgroundColor(color(16, 180)).setColorBackground(color(16, 180));
    group_fluid.getCaptionLabel().align(CENTER, CENTER);
    
    px = 10; py = 15;
    
    cp5.addButton("reset").setGroup(group_fluid).plugTo(this, "fluid_reset"     ).setSize(80, 18).setPosition(px    , py);
    cp5.addButton("+"    ).setGroup(group_fluid).plugTo(this, "fluid_resizeUp"  ).setSize(39, 18).setPosition(px+=82, py);
    cp5.addButton("-"    ).setGroup(group_fluid).plugTo(this, "fluid_resizeDown").setSize(39, 18).setPosition(px+=41, py);
    
    px = 10;
    
    cp5.addSlider("velocity").setGroup(group_fluid).setSize(sx, sy).setPosition(px, py+=(int)(oy*1.5f))
        .setRange(0, 1).setValue(fluid.param.dissipation_velocity).plugTo(fluid.param, "dissipation_velocity");
    
    cp5.addSlider("density").setGroup(group_fluid).setSize(sx, sy).setPosition(px, py+=oy)
        .setRange(0, 1).setValue(fluid.param.dissipation_density).plugTo(fluid.param, "dissipation_density");
    
    cp5.addSlider("temperature").setGroup(group_fluid).setSize(sx, sy).setPosition(px, py+=oy)
        .setRange(0, 1).setValue(fluid.param.dissipation_temperature).plugTo(fluid.param, "dissipation_temperature");
    
    cp5.addSlider("vorticity").setGroup(group_fluid).setSize(sx, sy).setPosition(px, py+=oy)
        .setRange(0, 1).setValue(fluid.param.vorticity).plugTo(fluid.param, "vorticity");
        
    cp5.addSlider("iterations").setGroup(group_fluid).setSize(sx, sy).setPosition(px, py+=oy)
        .setRange(0, 80).setValue(fluid.param.num_jacobi_projection).plugTo(fluid.param, "num_jacobi_projection");
          
    cp5.addSlider("timestep").setGroup(group_fluid).setSize(sx, sy).setPosition(px, py+=oy)
        .setRange(0, 1).setValue(fluid.param.timestep).plugTo(fluid.param, "timestep");
        
    cp5.addSlider("gridscale").setGroup(group_fluid).setSize(sx, sy).setPosition(px, py+=oy)
        .setRange(0, 50).setValue(fluid.param.gridscale).plugTo(fluid.param, "gridscale");
    
    RadioButton rb_setFluid_DisplayMode = cp5.addRadio("fluid_displayMode").setGroup(group_fluid).setSize(80,18).setPosition(px, py+=(int)(oy*1.5f))
        .setSpacingColumn(2).setSpacingRow(2).setItemsPerRow(2)
        .addItem("Density"    ,0)
        .addItem("Temperature",1)
        .addItem("Pressure"   ,2)
        .addItem("Velocity"   ,3)
        .activate(DISPLAY_fluid_texture_mode);
    for(Toggle toggle : rb_setFluid_DisplayMode.getItems()) toggle.getCaptionLabel().alignX(CENTER);
    
    cp5.addRadio("fluid_displayVelocityVectors").setGroup(group_fluid).setSize(18,18).setPosition(px, py+=(int)(oy*2.5f))
        .setSpacingColumn(2).setSpacingRow(2).setItemsPerRow(1)
        .addItem("Velocity Vectors", 0)
        .activate(DISPLAY_FLUID_VECTORS ? 0 : 2);
  }
  
  
  ////////////////////////////////////////////////////////////////////////////
  // GUI - DISPLAY
  ////////////////////////////////////////////////////////////////////////////
  Group group_display = cp5.addGroup("display");
  {
    group_display.setHeight(20).setSize(gui_w, 50)
    .setBackgroundColor(color(16, 180)).setColorBackground(color(16, 180));
    group_display.getCaptionLabel().align(CENTER, CENTER);
    
    px = 10; py = 15;
    
    cp5.addSlider("BACKGROUND").setGroup(group_display).setSize(sx,sy).setPosition(px, py)
        .setRange(0, 255).setValue(BACKGROUND_COLOR).plugTo(this, "BACKGROUND_COLOR");
    
    cp5.addRadio("fluid_displayParticles").setGroup(group_display).setSize(18,18).setPosition(px, py+=(int)(oy*1.5f))
        .setSpacingColumn(2).setSpacingRow(2).setItemsPerRow(1)
        .addItem("display particles", 0)
        .activate(DISPLAY_PARTICLES ? 0 : 2);
  }
  
  
  ////////////////////////////////////////////////////////////////////////////
  // GUI - ACCORDION
  ////////////////////////////////////////////////////////////////////////////
  cp5.addAccordion("acc").setPosition(gui_x, gui_y).setWidth(gui_w).setSize(gui_w, height)
    .setCollapseMode(Accordion.MULTI)
    .addItem(group_fluid)
    .addItem(group_display)
    .open(4);
}

MyParticleSystem:

/**
 * 
 * PixelFlow | Copyright (C) 2016 Thomas Diewald - http://thomasdiewald.com
 * 
 * A Processing/Java library for high performance GPU-Computing (GLSL).
 * MIT License: https://opensource.org/licenses/MIT
 * 
 */



import com.jogamp.opengl.GL2ES2;
import com.thomasdiewald.pixelflow.java.DwPixelFlow;
import com.thomasdiewald.pixelflow.java.dwgl.DwGLSLProgram;
import com.thomasdiewald.pixelflow.java.dwgl.DwGLTexture;
import com.thomasdiewald.pixelflow.java.fluid.DwFluid2D;

import processing.core.PConstants;
import processing.opengl.PGraphics2D;


static public class MyParticleSystem{
  
  public DwGLSLProgram shader_particleSpawn;
  public DwGLSLProgram shader_particleUpdate;
  public DwGLSLProgram shader_particleRender;
  
  public DwGLTexture.TexturePingPong tex_particles = new DwGLTexture.TexturePingPong();
  
  DwPixelFlow context;
  
  public int particles_x;
  public int particles_y;
  
  public int MAX_PARTICLES;
  public int ALIVE_LO = 0;
  public int ALIVE_HI = 0;
  public int ALIVE_PARTICLES = 0;
  
  // a global factor, to comfortably reduce/increase the number of particles to spawn
  public float spwan_scale = 1.0f;
  public float point_size  = 1.5f;
  
  public Param param = new Param();
  
  static public class Param{
    public float dissipation = 0.90f;
    public float inertia     = 0.20f;
  }
  
  public MyParticleSystem(){
  }
  
  public MyParticleSystem(DwPixelFlow context, int MAX_PARTICLES){
    context.papplet.registerMethod("dispose", this);
    this.resize(context, MAX_PARTICLES);
  }
  
  public void dispose(){
    release();
  }
  
  // call this, when the object is not used any longer!
  // OpenGL resources must be released to void memory leaks
  public void release(){
    tex_particles.release();
  }
  
  public void resize(DwPixelFlow context, int MAX_PARTICLES_WANTED){
    particles_x = (int) Math.ceil(Math.sqrt(MAX_PARTICLES_WANTED));
    particles_y = particles_x;
    resize(context, particles_x, particles_y);
  }
  
  public void resize(DwPixelFlow context, int num_particels_x, int num_particels_y){
    this.context = context;
    
    context.begin();
    
    release(); // just in case its not the first resize call
    
    MAX_PARTICLES = particles_x * particles_y;
    System.out.println("ParticelSystem: texture size = "+particles_x+"/"+particles_y +" ("+MAX_PARTICLES+" particles)");
    
    // create shader
    String dir = "data/";
    shader_particleSpawn  = context.createShader(dir + "particleSpawn.frag");
    shader_particleUpdate = context.createShader(dir + "particleUpdate.frag");
    shader_particleRender = context.createShader(dir + "particleRender.glsl", dir + "particleRender.glsl");
    shader_particleRender.vert.setDefine("SHADER_VERT", 1);
    shader_particleRender.frag.setDefine("SHADER_FRAG", 1);

    // allocate texture
    tex_particles.resize(context, GL2ES2.GL_RGBA32F, particles_x, particles_y, GL2ES2.GL_RGBA, GL2ES2.GL_FLOAT, GL2ES2.GL_NEAREST, 4, 4);

    context.end("ParticleSystem.resize");
 
    reset();  // initialize particles
  }
  
  
  public void reset(){

    ALIVE_LO = ALIVE_HI = ALIVE_PARTICLES = 0;

    // clear to 0 first, just in case
    tex_particles.src.clear(0);
    tex_particles.dst.clear(0);
    
    // additionally:
    // spawning ALL particles at (-1,-1)
    // this "kind of" clears the texture
    // also, in the render-pass, the vertex gets clipped and no fragment is generated
    // --> speeds up everything
    spawn(null, -1, -1, 0, particles_x *particles_y);
    
    ALIVE_LO = ALIVE_HI = ALIVE_PARTICLES = 0;
  }
  

  /**
   * 
   * @param px_norm normalized x spawn-position [0, 1]
   * @param py_norm normalized y spawn-position [0, 1]
   * @param count
   */
  public void spawn(DwFluid2D fluid, float px, float py, float radius, int count){
    
    count = Math.round(count * spwan_scale);
    
    if(ALIVE_HI == MAX_PARTICLES){
      System.out.println("all particles spawned, respawning from 0");
      ALIVE_HI = 0;
    }

    int spawn_lo = ALIVE_HI; 
    int spawn_hi = Math.min(spawn_lo + count, MAX_PARTICLES); 
    float noise = (float)(Math.random() * Math.PI);

    context.begin();
    context.beginDraw(tex_particles.dst);
    shader_particleSpawn.begin();
    if(fluid != null){
      shader_particleSpawn.uniform2f("wh_viewport", fluid.viewp_w, fluid.viewp_h);
    }
    shader_particleSpawn.uniform1i("spawn_lo", spawn_lo);
    shader_particleSpawn.uniform1i("spawn_hi", spawn_hi);
    shader_particleSpawn.uniform2f("spawn_origin", px, py);
    shader_particleSpawn.uniform1f("spawn_radius", radius);
    shader_particleSpawn.uniform1f("noise", noise);
    shader_particleSpawn.uniform2f("wh_particles" , particles_x, particles_y);
    shader_particleSpawn.uniformTexture("tex_particles" , tex_particles.src);
    shader_particleSpawn.drawFullScreenQuad();
    shader_particleSpawn.end();
    context.endDraw();
    context.end("ParticleSystem.spawn");
    tex_particles.swap();
    
    ALIVE_HI = spawn_hi;
    ALIVE_PARTICLES = Math.max(ALIVE_PARTICLES, ALIVE_HI - ALIVE_LO);
  }
  
  public void update(DwFluid2D fluid){
    context.begin();
    context.beginDraw(tex_particles.dst);
    shader_particleUpdate.begin();
    shader_particleUpdate.uniform2f     ("wh_fluid"     , fluid.fluid_w, fluid.fluid_h);
    shader_particleUpdate.uniform2f     ("wh_particles" , particles_x, particles_y);
    shader_particleUpdate.uniform1f     ("timestep"     , fluid.param.timestep);
    shader_particleUpdate.uniform1f     ("rdx"          , 1.0f / fluid.param.gridscale);
    shader_particleUpdate.uniform1f     ("dissipation"  , param.dissipation);
    shader_particleUpdate.uniform1f     ("inertia"      , param.inertia);
    shader_particleUpdate.uniformTexture("tex_particles", tex_particles.src);
    shader_particleUpdate.uniformTexture("tex_velocity" , fluid.tex_velocity.src);
    shader_particleUpdate.uniformTexture("tex_obstacles", fluid.tex_obstacleC.src);
    shader_particleUpdate.drawFullScreenQuad();
    shader_particleUpdate.end();
    context.endDraw();
    context.end("ParticleSystem.update");
    tex_particles.swap();
  }
  

  public void render(PGraphics2D dst, int background){
    // no need to run the vertex shader for particles that haven't spawned yet
    int num_points_to_render = ALIVE_PARTICLES;
    int w = dst.width;
    int h = dst.height;
    
    dst.beginDraw();
    dst.blendMode(PConstants.BLEND);
    if(background == 0) dst.blendMode(PConstants.ADD); // works nicely on black background
    
    context.begin();
    shader_particleRender.begin();
    shader_particleRender.uniform2f     ("wh_viewport", w, h);
    shader_particleRender.uniform2i     ("num_particles", particles_x, particles_y);
    shader_particleRender.uniform1f     ("point_size"   , point_size);
    shader_particleRender.uniformTexture("tex_particles", tex_particles.src);
    shader_particleRender.drawFullScreenPoints(num_points_to_render);
    shader_particleRender.end();
    context.end("ParticleSystem.render");

    dst.endDraw();
  }

  
}