无标题

Most Popular Deep Learning Projects

Top Deep Learning Projects A list of popular github projects related to deep learning (ranked by stars). Last Update: 2016.08.09 Project Name Stars Description TensorFlow 29622 Computation using data flow graphs for scalable machine learning. Caffe 11799 Caffe: a fast open framework for deep learning. Neural Style 10148 Torch implementation of neural style algorithm. Deep Dream 9042 Deep Dream.…

Good articles to learn Convolution Neural Networks

Understanding Convolutions from: https://colah.github.io/posts/2014-07-Understanding-Convolutions/ neural networks, convolutional neural networks, convolution, math, probability In a previous post, we built up an understanding of convolutional neural networks, without referring to any significant mathematics. To go further, however, we need to understand convolutions. If we just wanted to…

Machine learning in 10 pictures

Machine learning in 10 pictures  from: http://www.denizyuret.com/2014/02/machine-learning-in-5-pictures.html   I find myself coming back to the same few pictures when explaining basic machine learning concepts. Below is a list I find most illuminating. 1. Test and training error: Why lower training error is not always a good thing: ESL Figure 2.11. Test and training error as a function of model…

Good resources to learn how to use websocket push api in python

How to connect to poloniex.com websocket api using a python library The problem: I am trying to connect to wss://api.poloniex.com and subscribe to ticker. I can’t find any working example in python. I have tried to use autobahn/twisted and websocket-client 0.32.0.The purpose of this is to get real time ticker data and store it in a mysql database. The solution: What you are trying to accomplish…

Best examples to learn machine learning

  Here are some good examples to learn machine learning and data science using python pandas. The following resources are from https://github.com/savarin/pyconuk-introtutorial The tutorial will start with data manipulation using pandas – loading data, and cleaning data. We’ll then use scikit-learn to make predictions. By the end of the session, we would have worked on the Kaggle Titanic…

feature engineering in PySpark

We often need to do feature transformation to build a training data set before training a model.  Here are some good examples to show how to transform your data, especially if you need to derive new features from other columns using spark data frame.  Encode and assemble multiple features in PySpark Encode and assemble multiple features in PySpark from pyspark.sql import Row from…

Good resources to learn auto trade backtest

Research Backtesting Environments in Python with pandas By Michael Halls-Moore on January 16th, 2014 from: https://www.quantstart.com/articles/Research-Backtesting-Environments-in-Python-with-pandas Backtesting is the research process of applying a trading strategy idea to historical data in order to ascertain past performance. In particular, a backtester makes no guarantee about the future…

visualize iris dataset using python

This notebook demos Python data visualizations on the Iris dataset from: https://www.kaggle.com/benhamner/d/uciml/iris/python-data-visualizations This Python 3 environment comes with many helpful analytics libraries installed. It is defined by the kaggle/python docker image We’ll use three libraries for this tutorial: pandas, matplotlib, and seaborn. Press “Fork” at the top-right of this screen…

Adding Multiple Columns to Spark DataFrames

Adding Multiple Columns to Spark DataFrames from: https://p058.github.io/spark/2017/01/08/spark-dataframes.html I have been using spark’s dataframe API for quite sometime and often I would want to add many columns to a dataframe(for ex : Creating more features from existing features for a machine learning model) and find it hard to write many withColumn statements. So I monkey patched spark…

如何在 Kaggle 首战中进入前 10%

 如何在 Kaggle 首战中进入前 10% from: https://dnc1994.com/2016/04/rank-10-percent-in-first-kaggle-competition/ Introduction Kaggle 是目前最大的 Data Scientist 聚集地。很多公司会拿出自家的数据并提供奖金，在 Kaggle 上组织数据竞赛。我最近完成了第一次比赛，在 2125 个参赛队伍中排名第 98 位（~ 5%）。因为是第一次参赛，所以对这个成绩我已经很满意了。在 Kaggle 上一次比赛的结果除了排名以外，还会显示的就是 Prize Winner，10% 或是 25% 这三档。所以刚刚接触 Kaggle 的人很多都会以 25% 或是 10% 为目标。在本文中，我试图根据自己第一次比赛的经验和从其他 Kaggler 那里学到的知识，为刚刚听说 Kaggle…

Using Azure ML to Build Clickthrough Prediction Models

Using Azure ML to Build Clickthrough Prediction Models   from: https://blogs.technet.microsoft.com/machinelearning/2015/11/03/using-azure-ml-to-build-clickthrough-prediction-models/   This blog post is by Girish Nathan, a Senior Data Scientist at Microsoft. Ad click prediction is a multi-billion dollar industry, and one that is still growing rapidly. In this post, we build ML models on the…

Data Transformation methods: one hot encoding, learning with counts

Data Transformation One hot encoding transforms categorical features to a format that works better with classification and regression algorithms. Let’s take the following example. I have seven sample inputs of categorical data belonging to four categories. Now, I could encode these to nominal values as I have done here, but that wouldn’t make sense from a machine learning perspective. We can’t…

from: http://www.jianshu.com/p/32def2294ae6 最近挤出时间，用python在kaggle上试了几个project，有点体会，记录下。 Step1: Exploratory Data Analysis EDA，也就是对数据进行探索性的分析，一般就用到pandas和matplotlib就够了。EDA一般包括： 每个feature的意义，feature的类型,比较有用的代码如下 df.describe() df['Category'].unique() 看是否存在missing value df.loc[df.Dates.isnull(),'Dates'] 每个特征下的数据分布，可以用boxplot或者hist来看 %matplotlib inline import matplotlib.pyplot as plt…

An Introduction to Stock Market Data Analysis with Python

Here  are some best article for stock data analysis using python. An Introduction to Stock Market Data Analysis with Python (Part 1) from: https://ntguardian.wordpress.com/2016/09/19/introduction-stock-market-data-python-1/ This post is the first in a two-part series on stock data analysis using Python, based on a lecture I gave on the subject for MATH 3900 (Data Science) at the University of…

use spark to calculate moving average for time series data

Spark Window Functions for DataFrames and SQL from: http://xinhstechblog.blogspot.de/2016/04/spark-window-functions-for-dataframes.html Introduced in Spark 1.4, Spark window functions improved the expressiveness of Spark DataFrames and Spark SQL. With window functions, you can easily calculate a moving average or cumulative sum, or reference a value in a previous row of a table. Window functions…

How to send email on linux terminal

You may want to send email using linux terminal.  The easiest way is to use mail tool. for example, you can use the following command to send an email easily.  echo “content the email” | mail -s “subject of this email” [email protected] You can also use -a option to attach a file: echo “content the email” | mail -s “subject of this email” -a ~/filename  [email protected]  However, you may want to specify…

Good Articles to learn how to implement a neural network 1

  How to implement a neural network Part 1  From: http://peterroelants.github.io/posts/neural_network_implementation_part01/ This page is part of a 5 (+2) parts tutorial on how to implement a simple neural network model. You can find the links to the rest of the tutorial here: Part 1: Linear regression Intermezzo 1: Logistic classification function Part 2: Logistic regression (classification)…