1.介绍

此项目用kinect V2捕捉人体动作，在window10电脑上用Processing3.5.4 处理Kinect的骨骼数据（此处用到Kinect PV2的库）并显示在电脑屏幕上.后续有两种方案。 案例一通过Arduino Uno 获取Processing 解析出来的坐标数据，从而传递控制信号给CNC Shield v3 和A4988，进行控制35步进电机进行皮影动作运行。此案例方案适用于简单的动作，复杂的运动由于Arduino Uno性能局限无法达到精度要求，故此案例只作为调试用途。 案例二通RISC-vduino 获取Processing 解析出来的坐标数据，从而传递控制信号给CNC Shield v3 和A4988，进行控制35步进电机进行皮影动作运行。 RISC-vduino 性能足够满足此项目，能实现复杂的运动跟踪.

故最终版我们使用的软硬件有以下

2.准备工作

2.1硬件

  • A Kinect for Windows v2 Device (K4W2)：此硬件用于捕捉人体动作,Kinect分为Kinect for xbox和Kinect for windows，我们项目中使用Kinect for xbox，kinect for windows同样适用；
  • Arduino UNO：测试时用于检测初步代码和功能；
  • RISC-vduino：最终版本的电控核心，将从Processing3.5.4 获得的数据转化处理，从而将电机运转信号给CNC Shield；
  • CNC Shield v3 & A4988；CNC Sheild是电机控制的载体，A4988为驱动电机控制板，是CNCshield的核心，将最终的放大信号传至35步进电机；
  • 35步进电机；把获得的运行型号执行为旋转运动，从而推动丝杠上的滑块运行；
  • 12V2A电源；

  • 配件：联轴器，滑块，3d打印结构件，杜邦线，面包板；

    2.2 软件

    • Kinect for Windows SDK 2.0：此处用于搭建kinect 在PC端的运行环境和kinect的准备工作；
    • Processing3.5.4 ：通过和下面的Kinect PV2的库，将Kinect的信号转化成电脑的图像信号，并捕捉人体运动的关键坐标数据；
    • Kinect PV2：Processing和Kinect连接后的获得人体骨骼数据的库文件
    • Arduino IDE：测试阶段（案例一）的控制程序的软件，此软件和Arduino Uno连接，从而控制电机运动；
    • MounRiver Studio V1.3.0：最终阶段（案例二）的控制程序的软件，此软件和RISC-vduino连接，从而控制电机运动；

2.3 kinect 运行环境

2.3.1 Windows10电脑

  • Windows10 64位电脑；
  • 拥有USB3.0;
  • 确定升级最新的显卡驱动，此处根据实际电脑配置进行
  • 确定是否安装 DirectX 11;

2.3.2 Kinect SDK

我们的硬件使用的的是Kinect v2 for xbox， 所以SDK使用 Kinect SDK1.8,如果使用者使用Kinect v2 for Windows，推荐使用 Kinect SDK v2

  • 安装SDK 前需要拔掉Kinect；
  • 安装Kinect SDK1.8；
  • 安装 Kinect for Windows Developer Toolkit v1.8,这是sdk的一个工具箱，里面有很多丰富的例子。可以不安装，推荐安装；
  • 检测驱动:插上kinect，跑Developer Toolkit里面的任意例子，正常说明kinect设备能驱动了，反之没有。

2.3.3 Processing

  • 在官网选择适合你电脑的版本，我们目前使用 Processing 3.5.4，下载并且解压压缩包。Windows用户找到processing.exe便可以运行。

  • Processing中安装Kinect v2 library for Processing,在Sketch-Import library-Add Library中搜索并安装 另外可以在github上下载Kinect v2 library for Processing并解压缩放到安装目录的library内“C:\Users\Documents\Processing\libraries”， 安装完成后在文档目录和软件中分别显示为以下
  • 运行“Skeleton color”测试软件环境是否成功 如果程序运行类似上图能显示人体骨骼，说明Kinect和processing的安装环境成功

3.测试

我们会用两步去测试，如果大家熟悉RISC-Vduino，可以直接到 “3.2案例二（Kinect+Processing+RISC-Vduino test）”

3.1 案例一（Kinect+Processing+Arduino test）

Arduino UNO下载安装Arduino IDE,测试Arduino IDE 案例看是否成功。 接下来正式的程序，我们目标是以下几点

  • 将用Processing 程序去和Kinect连接，读取人体手掌相对于脖颈的坐标位置；
  • 坐标位置写成string通过串口传至Arduino Uno；
  • Arduino Uno将string值解析成电机转动数值；
  • Arduino Uno通过CNC Shield v3+A4988驱动35步进电机的运行，从而做出对应动作

以下为processing程序


import KinectPV2.KJoint;
import KinectPV2.*;
import processing.serial.*;
Serial port;
KinectPV2 kinect;

//int value_X1;
//int value_Y1;
//int value_Z1;
//int value_X2;
//int value_Y2;
//int value_Z2;
//int value_running=0;
String [] strs;
String data;


void setup() {
  size(1920, 1080, P3D);
   port=new Serial(this,"COM4",9600);

  kinect = new KinectPV2(this);

  kinect.enableSkeletonColorMap(true);
  kinect.enableColorImg(true);

  kinect.init();


}

void draw() {
  background(0);

  image(kinect.getColorImage(), 0, 0, width, height);

  ArrayList<KSkeleton> skeletonArray =  kinect.getSkeletonColorMap();

  //individual JOINTS
  for (int i = 0; i < skeletonArray.size(); i++) {
    KSkeleton skeleton = (KSkeleton) skeletonArray.get(i);
    if (skeleton.isTracked()) {
      KJoint[] joints = skeleton.getJoints();

      color col  = skeleton.getIndexColor();
      fill(col);
      stroke(col);
      drawBody(joints);



     //test
     float X0= joints[ KinectPV2.JointType_Neck].getX()- joints[KinectPV2.JointType_SpineBase].getX();
     float Y0= joints[ KinectPV2.JointType_Neck].getY()- joints[KinectPV2.JointType_SpineBase].getY();
     float Z0= joints[ KinectPV2.JointType_Neck].getZ()- joints[KinectPV2.JointType_SpineBase].getZ();
     float A1= joints[ KinectPV2.JointType_Neck].getX()- joints[KinectPV2.JointType_HandRight].getX();
     float B1= joints[ KinectPV2.JointType_Neck].getY()- joints[KinectPV2.JointType_HandRight].getY();
      float C1= joints[ KinectPV2.JointType_Neck].getZ()- joints[KinectPV2.JointType_HandRight].getZ();
      int value_X1 =int((abs((A1)*100/(X0)))); 
      int value_Y1 =int(abs((B1)*100/(Y0)));
      int value_Z1 =int(abs((C1)*100/(Z0)));
      float A2= joints[ KinectPV2.JointType_Neck].getX()- joints[KinectPV2.JointType_HandLeft].getX();
      float B2= joints[ KinectPV2.JointType_Neck].getY()- joints[KinectPV2.JointType_HandLeft].getY();
      float C2= joints[ KinectPV2.JointType_Neck].getZ()- joints[KinectPV2.JointType_HandLeft].getZ();
      int value_X2 =int((abs((A2)*100/(X0)))); 
      int value_Y2 =int(abs((B2)*100/(Y0)));
      int value_Z2 =int(abs((C2)*100/(Z0)));
      text("X1=",20, 150);text(value_X1, 50, 150);text("Y1=",120, 150); text(value_Y1, 150, 150);text("Z1=",220, 150);text(value_Z1, 250, 150);
      text("X2=",20, 250);text(value_X2, 50, 250);text("Y2=",120, 250); text(value_Y2, 150, 250);text("Z2=",220, 250);text(value_Z2, 250, 250);
      strs=new String[6];
      strs[0]=str(int((abs((A1)*100/(X0)))));
      strs[1]=str(int((abs((B1)*100/(X0)))));
      strs[2]=str(int((abs((C1)*100/(X0)))));
      strs[3]=str(int((abs((A2)*100/(X0)))));
      strs[4]=str(int((abs((B2)*100/(X0)))));
      strs[5]=str(int((abs((C2)*100/(X0)))));
      data =join(strs,"a");



     //test

      //draw different color for each hand state
      drawHandState(joints[KinectPV2.JointType_HandRight]);
      drawHandState(joints[KinectPV2.JointType_HandLeft]);
      drawHandState(joints[KinectPV2.JointType_Neck]);
    }
  }

  fill(0, 255, 0);
  text(frameRate, 50, 50);

}

//DRAW BODY
void drawBody(KJoint[] joints) {
  drawBone(joints, KinectPV2.JointType_Head, KinectPV2.JointType_Neck);
  drawBone(joints, KinectPV2.JointType_Neck, KinectPV2.JointType_SpineShoulder);
  drawBone(joints, KinectPV2.JointType_SpineShoulder, KinectPV2.JointType_SpineMid);
  drawBone(joints, KinectPV2.JointType_SpineMid, KinectPV2.JointType_SpineBase);
  drawBone(joints, KinectPV2.JointType_SpineShoulder, KinectPV2.JointType_ShoulderRight);
  drawBone(joints, KinectPV2.JointType_SpineShoulder, KinectPV2.JointType_ShoulderLeft);
  drawBone(joints, KinectPV2.JointType_SpineBase, KinectPV2.JointType_HipRight);
  drawBone(joints, KinectPV2.JointType_SpineBase, KinectPV2.JointType_HipLeft);

  // Right Arm
  drawBone(joints, KinectPV2.JointType_ShoulderRight, KinectPV2.JointType_ElbowRight);
  drawBone(joints, KinectPV2.JointType_ElbowRight, KinectPV2.JointType_WristRight);
  drawBone(joints, KinectPV2.JointType_WristRight, KinectPV2.JointType_HandRight);
  drawBone(joints, KinectPV2.JointType_HandRight, KinectPV2.JointType_HandTipRight);
  drawBone(joints, KinectPV2.JointType_WristRight, KinectPV2.JointType_ThumbRight);

  // Left Arm
  drawBone(joints, KinectPV2.JointType_ShoulderLeft, KinectPV2.JointType_ElbowLeft);
  drawBone(joints, KinectPV2.JointType_ElbowLeft, KinectPV2.JointType_WristLeft);
  drawBone(joints, KinectPV2.JointType_WristLeft, KinectPV2.JointType_HandLeft);
  drawBone(joints, KinectPV2.JointType_HandLeft, KinectPV2.JointType_HandTipLeft);
  drawBone(joints, KinectPV2.JointType_WristLeft, KinectPV2.JointType_ThumbLeft);

  // Right Leg
  drawBone(joints, KinectPV2.JointType_HipRight, KinectPV2.JointType_KneeRight);
  drawBone(joints, KinectPV2.JointType_KneeRight, KinectPV2.JointType_AnkleRight);
  drawBone(joints, KinectPV2.JointType_AnkleRight, KinectPV2.JointType_FootRight);

  // Left Leg
  drawBone(joints, KinectPV2.JointType_HipLeft, KinectPV2.JointType_KneeLeft);
  drawBone(joints, KinectPV2.JointType_KneeLeft, KinectPV2.JointType_AnkleLeft);
  drawBone(joints, KinectPV2.JointType_AnkleLeft, KinectPV2.JointType_FootLeft);


  //data we need
  fill(0, 255, 2255);
  strokeWeight(10);
  stroke(0,0,0);
  drawBone(joints, KinectPV2.JointType_Neck, KinectPV2.JointType_HandRight);
  drawBone(joints, KinectPV2.JointType_Neck, KinectPV2.JointType_HandLeft);



  drawJoint(joints, KinectPV2.JointType_HandLeft);
  drawJoint(joints, KinectPV2.JointType_HandRight);
  drawJoint(joints, KinectPV2.JointType_Neck);
  //drawJoint(joints, KinectPV2.JointType_FootLeft);
  //drawJoint(joints, KinectPV2.JointType_FootRight);

  //drawJoint(joints, KinectPV2.JointType_ThumbLeft);
 // drawJoint(joints, KinectPV2.JointType_ThumbRight);

  //drawJoint(joints, KinectPV2.JointType_Head);
}

//draw joint
void drawJoint(KJoint[] joints, int jointType) {
  pushMatrix();
  translate(joints[jointType].getX(), joints[jointType].getY(), joints[jointType].getZ());
  ellipse(0, 0, 25, 25);
  popMatrix();
}

//draw bone
void drawBone(KJoint[] joints, int jointType1, int jointType2) {
   stroke(255,255,0);
  pushMatrix();
  translate(joints[jointType1].getX(), joints[jointType1].getY(), joints[jointType1].getZ());
  //text(joints[jointType1].getX(),50,50);
  ellipse(0, 0, 25, 25);
  popMatrix();
  line(joints[jointType1].getX(), joints[jointType1].getY(), joints[jointType1].getZ(), joints[jointType2].getX(), joints[jointType2].getY(), joints[jointType2].getZ());



  pushMatrix();
  translate(joints[jointType1].getX(), joints[jointType1].getY(), joints[jointType1].getZ());
  //text(joints[jointType1].getX(),50,50);
  ellipse(0, 0, 25, 25);
  popMatrix();
  stroke(0);
  fill(0);

  line(joints[ KinectPV2.JointType_Neck].getX(), joints[ KinectPV2.JointType_Neck].getY(), joints[ KinectPV2.JointType_Neck].getZ(), joints[KinectPV2.JointType_SpineBase].getX(), joints[KinectPV2.JointType_SpineBase].getY(), joints[KinectPV2.JointType_SpineBase].getZ());
  stroke(255,0,255);
  line(joints[ KinectPV2.JointType_Neck].getX(), joints[ KinectPV2.JointType_Neck].getY(), joints[ KinectPV2.JointType_Neck].getZ(), joints[KinectPV2.JointType_HandRight].getX(), joints[KinectPV2.JointType_HandRight].getY(), joints[KinectPV2.JointType_HandRight].getZ());

  line(joints[ KinectPV2.JointType_Neck].getX(), joints[ KinectPV2.JointType_Neck].getY(), joints[ KinectPV2.JointType_Neck].getZ(), joints[KinectPV2.JointType_HandLeft].getX(), joints[KinectPV2.JointType_HandLeft].getY(), joints[KinectPV2.JointType_HandLeft].getZ());




}




//draw hand state
void drawHandState(KJoint joint) {
  noStroke();
  handState(joint.getState());
  pushMatrix();
  translate(joint.getX(), joint.getY(), joint.getZ());//text(joint.getX(), 50,0);
  ellipse(0, 0, 70, 70);
  popMatrix();


}

/*
Different hand state
 KinectPV2.HandState_Open
 KinectPV2.HandState_Closed
 KinectPV2.HandState_Lasso
 KinectPV2.HandState_NotTracked
 */
void handState(int handState) {
  switch(handState) {
  case KinectPV2.HandState_Open:
    fill(0, 255, 0);
    println("open");

    break;
  case KinectPV2.HandState_Closed:
    fill(255, 0, 0);

    println(data);
    port.write(data);

    delay(1000);

    //value_running=0;
    //get cordination

    break;
  case KinectPV2.HandState_Lasso:
    fill(0, 0, 255);
    break;
  case KinectPV2.HandState_NotTracked:
    fill(255, 255, 255);
    break;
  }
}

3.2 案例二（Kinect+Processing+RISC-Vduino test）

RISC-vduino：最终版本的电控核心，将从Processing3.5.4 获得的数据转化处理，从而将电机运转信号给CNC Shield；

4. 参考

results matching ""

    No results matching ""