STEM Project Showcase 2016

You are cordially invited to attend the ElRo STEM Project Showcase 2016. Students from the Advanced STEM Research class learn to develop and integrate their knowledge in Math, Science and Technology, and use Engineering method to solve complex real-world problems. They will present their year-long projects.

STEM Project Showcase 2016
Date: June 8, 2016 (Wednesday)
Time: 3:00 - 5:00 pm
Location: RM 201

Perceptron Challenge: A Practice Project

Once you finish the perceptron training, you have a new challenge to tackle - modifying the

perceptron code to classify a new data set. The information of the data set is listed in the following:

1. No.of classes: 2 (class 0 and 1)
2. No. of training data: 2000
3. No. of testing data: 400
4. No. of features: 3 (feature vector is in 3-dimensional space)
5. Files: training.txt, and testing.txt

Since the original perceptron is only for 2-dimensional data, you need to identify and modify all the dimension-related code to make it work. After you finishing the classification, visualize your training and testing data, and the decision boundary using Grapher in Mac laptop (see example). You can post your results once you finish the project, you can post your results onto the blogger. 

Please feel free to drop by Mr. Lin's office or lab for any questions.

ML Workshop Update

During the ML Workshop last Saturday, four members (Tai Wei, Alan, Jennifer and Crystal) of our research group have been trained to understand perceptron at code level (in C language). They are ready to help other members who are interested in ML go through the learning curve of this basic Artificial Neural Network (ANN) skill. This basic training will form the foundation for the next-level neural network- Multi-layer Perceptron (MLP), which is much more complicated. 

Currently, we are actively exploring the MLP in two Advanced STEM Research projects. One is applying MLP to do regression for the drone position detection; another one is to use MPL to do classification for hand gesture recognition based on EMG signals. Once we understand more about MLP at code level, we will be hosting our next ML Workshop to train our group members. 

In the mean time, any one who missed the last workshop and is interested in learning the perceptron can contact Tai Wei, Alan, Jennifer or Crystal directly to arrange an one-to-one training session (about 1~2 hours). You can also contact me for any training arrangement. We are expecting to use MLP and more advanced algorithms to solve real-world problems.

 

Workshop Resource: Perceptron Code

Please click the following perceptron code including training and classification. There are two project folders:
link to find the

1. PerceptronTrainer: Perform training based on training data in training.txt and save the trained weight at weights.txt.
2. PerceptronClassifier: Perform classification based on the weights.txt and output the result to classification.txt. 

ML Saturday Workshop

Hi All,

We are going to have our ML Workshop from 9:00 am to 4:00 pm this Saturday along with STEM Hackathon in RM 201. The focus will be the hands-on experience of your chosen projects. Each team will be working on your own projects and try to make visible progress in one day. Tutorials will be provided to help you kick start your project. For people who are interested in join one the project but haven't signed up, you can just come in and join a team (see previous post).

Since there will be ACT Test administrated in the school the same day, some of the participants may come after the test. So, when you arrive on Saturday, please go directly to RM 201 (don't go to other test-administrating floors). If you will come late, please email me in advance for lunch arrangement.

You are also encouraged to invite your friend to join the research group. I am looking forward to seeing you this Saturday!

ML Meeting Notes & Research Opportunities

 

Introduction & Progress Update

 

We finally had a long-due gathering of machine learning late afternoon yesterday. Even though some of our members had college class, piano class, etc.. We still had eight members attended the meeting including new friends! Mr. Lin went through the basic introduction of Artificial Neural Networks (ANN), Perceptrons, and Multi-Layer Perceptrons (MLP). He also showed the current progress of Machine Learning (ML) research group. It can be summarized as the following:

• Perceptron has been studied, understood, implemented in C code, and proved to be functioning. It can be used to separate two linearly separable classes.
• Perceptron algorithm has been used to distinguish hand gestures between thumb and pinky by the Electromyography (EMG) signals acquired from two muscles in the forearm. This "Hand-to-Hand Communication" project is a current project of Advanced STEM Research class.
• Multi-Layer Perceptron has been studied, partially understood, and implemented in C code. It has been trained (20,000 samples) and tested (2,500 samples) using artificially generated data set including five classes in the 3D space. So far, it can separate the testing data properly, and the error rate is 2.48 %. 
• The current focus is to apply the MLP to the "Hand-to-Hand Communication" project such that we can recognize hand gestures form all five fingers.

You can view the detailed presentation by clicking the link. 

Research Opportunities

At this moment, four research opportunities are available for students to ramp up the ML learning curve and develop skill sets in specific fields. 

1. Subway Signs Recognition: We will start from recognizing train numbers using MLP. Some image pre-processing and feature extraction will be provided to create the feature vectors as the input of the MLP algorithm. Our goal is to recognize all the subway signs by classification.
2. Drone Tracking: We will use MLP to find the location (3D coordinates) of a flying drone through the positions (2D coordinates) in the image planes of multi-angle cameras by regression. 
3. Bioinformatics: Bioinformatics is the application of computer technology to the management of biological information. Computers are used to gather, store, analyze and integrate biological and genetic information which can then be applied to gene-based drug discovery and development. We are in the process of forming a bioinformatics project with the Evolutionary Bioinformatics Lab at Hunter College.
4. Watson Analytics: IBM is actively developing their Watson-based business. Harnessing the power of Watson to various fields in our lives depends on software applications. We may be able enter the Watson world by learning and developing apps on this commercially available platform.   

So far, a few students have decided to sign up specific activities:

1. Subway Signs Recognition: Qin Ying, Tai Wei and Alan.
2. Drone Tracking: (open).
3. Bioinformatics: Belinda.
4. Watson Analytics: Michael.

For people who haven't signed up specific projects yet or people who have missed the meeting, you can email Mr. Lin your choice. You are more than welcome to review the presentation and discuss the details of the research opportunities with Mr. Lin. Mr. Lin will soon post further information for each team to start working on their project!

Invitation: Machine Learning Group Gathering

Hi All,

It has been a while since our last meeting. Even though it seems to be relatively quiet on the blog, some substantial progresses have been made in the past few weeks. The Perceptron has been understood, coded, and applied to investigate a finger gesture recognition problem encountered in the Advanced STEM Research class. The first version of Multi-Layer Perception (MLP) has been implemented. The results of initial training and testing based on artificially generated data sets have been very promising. It will soon be applied to several projects including recognizing finger gestures, locating drone positions, and recognizing subway signs!

I believe that it's a good timing for us to gather together, share what we have learned, and plan for the next step such that every member can be benefited from the information and knowledge accumulated in this research, build skills through hands-on exercises, and contribute to the Machine Learning Group in a substantial way in the coming months.

You are invited to attend the Machine Learning Group Gathering at 4:00 pm on Monday (03/28) in Room 417. You may spread the words to anyone of your friends who is also interested. I am looking forward to seeing you then!

Machine Learning in Healthcare

I attended a technical symposium yesterday hosted by the CIE-USA (Chinese Institute of Engineers-USA). One of the most popular plenary presentation was delivered by Dr. Jianying Hu (a Distinguished Research Staff Member and Senior Manger of Health Informatics Research at IBM T. J. Watson Research Center, NY) about "Data Driven Healthcare Analytics". Her team leveraged the data existing in the healthcare ecosystem, and developed various machine learning-based tools to help players such as doctors and medical directors in this ecosystem to make better decisions in care delivery and care management. For example, the Care Pathway Explorer is a system that mines and visualizes a set of frequent event sequences from patient EMR (Electronic Medical Records) data. The data of more than 200,000 patients over three years time frame has been analyzed in her research. 

Overview of the Care Pathway Explorer system: It contains four major modules: Event Database, Data Preprocessor, Frequent Pattern Analytics, and a Visual Interface.

"The goal is to utilize historical EMR data to extract common sequences of medical events such as diagnoses and treatments, and investigate how these sequences correlate with patient outcome." Patient similarity has been extracted through machine learning techniques to help decision making process. 

Care Pathway Explorer: A bubble chart and displays events of the most frequent patterns mined (left), and a flow visualization to show the most frequent patterns (right).

Insights of Care Pathway Explorer: Examples of insights reached interactively when mining patterns from patients with a hypertension pre-condition before the initial hyperlipidemi a diagnosis (2,800 patients, featuring 14,979 hyperlipidemia-related diagnoses and 24,898 medication events).

Machine learning has been integrated into the healthcare process and is ready for commercial deployment. I was so impressed to see how this new technology has started impacting the healthcare system. Though the speaker insisted that the system will only be a powerful tool for doctors to improve their efficiency, there were lots of feedback and comments about its potential to dramatically change (or even replace) doctors' roles in the healthcare system in the years to come.

I found a YouTube video and a paper in technical journal include the similar information shared by Dr. Jianying Hu yesterday. If you are interested in this field, you should take a look. Besides healthcare, machine learning has potential to change many other aspects of our lives. It encourages me to spend more time to catch up this fast-growing technology.

Here is an article I found in my Time Magazine about AI
December 24, 2015-January 4, 2016
Pg. 44

Progress Presentation

Here is my progress presentation.

Perceptron Algorithm Notes

Introduction

• Perceptron models a neuron
• n inputs are received by it that correspond to features
• It takes the inputs and sums them, then produces an output after checking the result
• Its main use is to classify groups that are linearly separable
• It is often used for binary classification

Components

• It is composed of a summation processor, and activation function (does the thresholding) and weights.
• It takes a weighted sum of the inputs and outputs 
• e.g. 1 if the sum is greater than the threshold and 0 if it isn't
• Usually the input and output weights are real numbers

• Perceptron can also have an input called the bias
• Bias allows the transfer function curve to be shifted horizontally along the input axis while the curve is left unaltered.
• Weight determines the slope 
• e.g. in the eq. y = mx + b, m is the weight and b is the bias

• Activation Function
• Also know as transfer function
• It translates input signals into output signals 
• The output is produced using a threshold
• The 4 types of activation functions most commonly used are: Unit step, sigmoid, piecewise linear, and Gaussian
• The unit step is a simple threshold where the output is 1 of 2 levels and it depends on whether the input is greater than or less than the threshold value
• The sigmoid contains 2 functions: tangential and logistic. Values of tangential functions are from -1 to 1 and the values of logistic functions are from 0 to 1.
• In the piecewise linear function the output arrived at from being proportional to the total weighted output.
• The Gaussian functions are continuous bell shaped curved functions. The node output which is high or low is interpreted from class membership which is 1 or 0, and depends on how close the net input is to a chosen value or average. 
• Learning Rate
• Used when updating the weights and bias to get a smaller error
• Helps control how much the weights and bias are changed

• Learning In Perceptrons 
• when trying to find a line to separate two classes or groups it responds to the input vectors with a target value of 0 or 1
• If the classes can be separated the solution will be found in a finite amount of time.