Machine Learning: Supervised Learning

Posted on by
Reply

The goal of this post is to present the most popular supervised learning algorithms.

There are two types supervised learning algorithms: regression and classification. Both of them have the goal to make a prediction based on input data provided during the training. In that input data we have information about the independent variables and dependent variables. The dependent variable is what the algorithm will predict later using new data.

Regression: The dependent variable is a number.

  • Linear Regression
  • Polynomial Regression
  • Support Vector Regression (SVR)
  • K-nearest neighbors (KNN)
  • Decision Trees & Random Forest

Classification: The dependent variable is a category.

  • Logistic Regression (Binary classification)
  • Naive Bayes
  • Support Vector Machine (SVM)
  • K-nearest neighbors (KNN)
  • Decision Trees & Random Forest

Supervised Learning

Big resolution image.

What I did the last 3 years

Posted on by
Reply

I had no time to write something for a while, the goal of this post is to summarize what did last years beside my job.

1. Master’s degree: Data Analytics and Business Intelligence

I decided to come back to the university to study Data Science I completed the following studies:

Master’s degree Business Intelligence 2018 – 2019 (Universitat Oberta de Catalunya – Spain)
– Gather and analyze information relevant to a company’s environment.
– Business oriented data analytics. (Customer and operations analytics)
– Use of procedures, skills, applications, tools and practices to support decision-making.

Post degree Data Analytics and Big Data 2016 – 2018 (Universitat Oberta de Catalunya – Spain)
– Data mining, data analytics and visual analytics.
– Data management, data governance and Big Data.
– Machine learning and artificial intelligence.

During that time I was involved in a couple of personal projects beside my job in OMICRON Electroncis.

2. Data Analyst hobbyist

I supported the eSport club x6tence doing data analysis for Clash Royale. To do this a webpage and a database SQLServer in Azure was used to do the data acquisition and PowerBI to do the Data Analysis.

This project was excelent to apply a lot of things that I was learning during my studies, special mention to PowerBI.

3. Software Development: Doctor Decks

Doctor Decks was a project in which I work together with a work colleague. The goal of the software was to find good combinations of cards for the game Clash Royale. The product was available in a web and also had phone apps: Windows universal, iOS and Android. A cloud app in Microsoft Azure was collecting the data from differents apis.

This was an excelent project to learn a lot about data analysis and improve my knowledge about Azure Cloud, web development with Angular and apps development with Xamarin.

The web had thousands of visits everyday and the app more than 1M downloads! After more than two years of success we decided to finish the project due to the maintenance effort.

 

ASP.NET Core 1 – Authorization using Policies

Posted on by
2

The goal of this post is to show how can we protect controller actions in ASP.NET Core 1 using Policies.

The whole code is available on GitHub: ASP.NET Core 1, Security using Policies.

With policies we don’t need to hard code anymore Roles or Names in our Authorize attribute. A policy is an authorization logic that contains one of more requirements.

How to use a policy?

The concept is very simple, once we have a defined policy we can add it to our Authorize attributes…

[HttpGet]
[Authorize(CookieMonsterSecurity.OnlyGoodMonstersPolicy)]
public IActionResult Info()
{
//... something that only good monsters can do
}
view raw UsingPoliciesAuthorize.cs hosted with ❤ by GitHub

How to create a policy?

We have to define our policies in our Startup class, in ConfigureServices. We need a policy name, a list of valid authentication schemes and a list of requirements.

// Configure authorization
services.AddAuthorization(options => options.AddPolicy(CookieMonsterSecurity.OnlyGoodMonstersPolicy, policy =>
{
policy.AuthenticationSchemes.Add(CookieMonsterSecurity.CookieMonsterAuthenticationSchema);
// Our own requirement logic...
policy.AddRequirements(new IsGoodMonsterRequirement());
}));
view raw Policies1-Startup.cs hosted with ❤ by GitHub

We can add more than one requirement to our policy, there are some pre-build requirements:

  • policy.RequireAuthenticatedUser()
  • policy.RequireClaim(…)
  • policy.RequireRole(…)

But the more flexible way is to add a custom requirement, doing this we can write our own logic:

  • policy.AddRequirements(new IsGoodMonsterRequirement());

To write our requirement we use the base class AuthorizationHandler and implement the interface IAuthorizationRequirement.

This requirement checks that the user is authenticated and has the claim “MonsterTypeClaim” = “Good”

public class IsGoodMonsterRequirement : AuthorizationHandler<IsGoodMonsterRequirement>, IAuthorizationRequirement
{
protected override void Handle(AuthorizationContext context, IsGoodMonsterRequirement requirement)
{
Console.WriteLine("Is a good monster?");
if (!context.User.Identity.IsAuthenticated)
{
Console.WriteLine("... is authenticated...");
}
if (context.User.HasClaim(CookieMonsterSecurity.MonsterTypeClaim, CookieMonsterSecurity.MonsterTypes.Good))
{
Console.WriteLine("... and has the MonsterTypeClaim = MonsterTypes.Good!");
context.Succeed(requirement);
}
}
}
view raw IsGoodMonsterPolicyRequirement.cs hosted with ❤ by GitHub

Getting Started with ASP NET Core 1 and Angular 2 in Visual Studio 2015

Posted on by
5

The goal of this post is to set up a Visual Studio 2015 project with ASP NET Core 1 and Angular 2 Typescript that can be used as a template for all my projects.

ASP NET Core1 Logo         angular2Logo

The whole code is available on GitHub: https://github.com/softwarejc/angular2-aspmvc-core1-getting-started

We will use npm (Node Package Manager) to add our dependencies and we will configure the typescript transpiler integrated in Visual Studio. We will also configure a gulp task to bundle our javascript an css dependencies.

Content

  • Introduction
  • Install Pre-requisites
  • Create an empty ASP Core 1 project
  • Auto transpile Typescript after save
  • Add Angular 2 with npm
  • Set up gulp task to bundle dependencies

Introduction

The former ASP.NET 5 was renamed to .NET Core 1. It is a modular Microsoft runtime library implementation that includes a subset of the .NET Framework.

.NET Core 1 was written from scratch. It has a lot of advantages, for example it is lighter, open-source and multi-platform: Mac, Linux and Windows.

Although the version 1 is in Release Candidate there are some ASP.NET 4.6 features missing, like SignalR.

It is up to you to choose a mature Framework like ASP.NET 4.6 or a modern one like ASP.NET Core 1… you can read more about it in this post by Scott Hanselman. I copied the following diagram from that post :-).

ASPNET Core 1

Install Pre-requisites

There are some problems in Windows using npm version 2. It nest dependencies and ends up having too long paths for Windows. Let’s start updating npm to version 3 and configuring Visual Studio to use it.

The first thing that we should do is to install Python version 2.7.X and NodeJS version 4.2.X. Remember the path where you install NodeJS, we will need it later.

Now open a command line and update npm:

npm install npm -g

Check your npm version:

npm -v

The version should be higher than 3.X.X.

Open visual studio and add the path where NodeJS is installed as the first path to look for external tools (at the top). The next time that Visual Studio has to restore a node package it will start looking for nodejs.exe in that path. The version we just installed will be found and used.

Configure VS to use npm 3

Create an empty ASP NET Core 1 project

Let’s create our project and configure a very simple server-side code.

In Visual Studio 2015 Update 1 ASP NET Core 1 is still called ASP NET 5. Create a new project and select the WebAPI template. Our project will have a SPA that will call a WebAPI controller to get a message.

Create ASPNET 5 Project

 

Delete the Project_Readme.html and update the values controller with this simple WebAPI controller that returns a hello message and the server UTC Time:

using System;
using Microsoft.AspNet.Mvc;
namespace GettingStarted_ASPNetCore1_Angular2.Controllers
{
[Route("api/[controller]")]
public class HelloController : Controller
{
[HttpGet]
public IActionResult Get(string name)
{
var time = DateTime.UtcNow.ToString("hh:mm:ss");
var response = new
{
message = $"{time} - Hello {name}, server-side speaking!"
};
return new ObjectResult(response);
}
}
}
view raw HelloController.cs hosted with ❤ by GitHub

We will call this method later from angular to check that the communication between our SPA and the server works. For now we can call it from the browser to check it. Run the application and write something like this in your browser using your configured port:

http://localhost:10137/api/hello

You should see something like:

apiTest

You can also pass your name to check that the server receive your parameters:

http://localhost:10137/api/hello?name=JuanCarlos

The file Startup.cs contains the configuration of the middleware. We will see in a future post how it works and how to configure it to use features like authentication. The default configuration when you select a WebAPI project template is ok for us.

Auto transpile Typescript after save

We can tell Visual Studio to auto transpile always that we edit and save a typescript file enabling:

“Automatically compile TypeScript files which are not part of a project”
ts Compile on Save

Our files will be part of the project… but this is an easy way to make Visual Studio auto transpile. Running typescript compiler in watch mode is an alternative… but I like this solution because I have everything integrated in Visual Studio.

If you know a better way please leave me a comment. 🙂

We will have a tsconfig.json file on the project root of our project to configure typescript, Visual Studio will use that file and not the settings that we configure in Options:

{
"compilerOptions": {
"target": "es5",
"module": "system",
"moduleResolution": "node",
"sourceMap": true,
"emitDecoratorMetadata": true,
"experimentalDecorators": true,
"removeComments": false,
"noImplicitAny": true,
"suppressImplicitAnyIndexErrors": true
},
"exclude": [
"node_modules"
]
}
view raw tsconfig.json hosted with ❤ by GitHub

Add Angular 2 with npm

The application that we will use as an example will be a simple application that call the server to get a message and will display using Angular 2.

Let’s add Angular 2 and other dependencies to npm, we need a file called “package.json” on the root of our project:

{
"name": "aspnetcore1-angular2-gettingstarted",
"version": "1.0.0",
"scripts": {
"gulp": "gulp"
},
"license": "ISC",
"dependencies": {
"jquery": "2.2.0",
"bootstrap": "3.3.6",
"systemjs": "0.19.17",
"rxjs": "5.0.0-beta.0",
"angular2": "2.0.0-beta.0",
"es6-shim": "^0.33.3"
},
"devDependencies": {
"del": "2.2.0",
"gulp": "3.9.0",
"gulp-concat": "2.6.0",
"gulp-rename": "1.2.2",
"gulp-uglify": "1.5.1",
"gulp-concat-css": "2.2.0"
}
}
view raw package.json hosted with ❤ by GitHub

Save the file and Visual Studio will create node_modules folder and download all dependencies for you.

It is out of the scope of this post to explain how Angular 2 works you can read more about it here. But basically we define a component that we can use in our index.

<hello></hello>

This component is a html view that has bindings to its code behind or “view-model”.

hello.view.html

<div style="background: ghostwhite">
<h3>
<label>Name:</label>
<!--binding to component field-->
<input type="text" [(ngModel)]="name" placeholder="Enter a name here">
<!--binding click event to component method-->
<button class="btn btn-primary col-md-2" (click)="sayHello()">
Hello!
</button>
</h3>
<!-- conditionally display `message` -->
<h4 [hidden]="!message">{{message}}</h4>
</div>
view raw hello.view.html hosted with ❤ by GitHub

hello.component.ts

import {Component} from 'angular2/core';
import {OnInit} from 'angular2/core';
import {HelloService} from './hello.service';
@Component({
// Declare the tag name in index.html to where the component attaches
selector: 'hello',
// Location of the template for this component
templateUrl: 'views/hello.view.html',
// Dependencies
providers: [HelloService]
})
// Component controller
export class Hello implements OnInit {
public message: string;
public name: string;
constructor(private _helloService: HelloService) {
}
ngOnInit() {
this.name = "";
this.message = "";
}
sayHello() {
// Use hello service to call the API and bind result
this._helloService.sayHello(this.name).subscribe(response => {
this.message = response;
});
}
}
view raw hello.component.ts hosted with ❤ by GitHub

The hello.component uses the service hello.service that makes REST calls to the server and returns Observable<T>.

hello.service.ts

import {Http} from 'angular2/http';
import {Injectable} from 'angular2/core';
import {Observable} from "rxjs";
@Injectable()
export class HelloService {
private _sayHelloServiceUrl: String = '/api/hello/';
constructor(private http: Http) { }
public sayHello(name: string): Observable<string> {
var url = `${this._sayHelloServiceUrl}?name=${name}`;
return this.http.get(url).map(res => res.json().message);
}
}
view raw hello.service.ts hosted with ❤ by GitHub

We need to “start” / “bootstrap” our application. We will use System.JS to do that. This is this index.html where we “start” our Angular 2 app:

index.html

<!DOCTYPE html>
<html>
<head>
<title>Getting started with ASPNET Core 1 and Angular 2</title>
<!--Styles bundle-->
<link href="css/styles.css" rel="stylesheet" />
</head>
<body>
<header>
<div class="container">
<h1>Getting started with ASPNET Core 1 and Angular 2</h1>
<h3>wwww.softwarejuancarlos.com</h3>
</div>
</header>
<script src="https://gist.github.com/softwarejc/f5cc7ed938f12378f902.js"></script>
<!--Use angular "hello" component-->
<div class="container">
<hello>Loading...</hello>
</div>
<!--Dependencies bundle-->
<script src="scripts/deps.min.js"></script>
<script>
// Load all app js files
System.config({
"transpiler": false,
"packages": {
"app": {defaultExtension: 'js'}
}
});
// Run app
System.import('./app/bootstrap').then(null, console.error.bind(console));
</script>
</body>
</html>
view raw index.html hosted with ❤ by GitHub

This is the piece of code of index.html that bootstrap our application:

System.import(‘./app/bootstrap’).then(null, console.error.bind(console));

And this is our bootstrap.ts:

import {bootstrap} from 'angular2/platform/browser'
import {HTTP_PROVIDERS} from 'angular2/http';
import {Hello} from './hello/hello.component';
import 'rxjs/Rx'
// Bootstrap Angular2
bootstrap(Hello, [HTTP_PROVIDERS]).then(
success => console.log('app bootstrapped...'),
error => console.log(error)
);
view raw bootstrap.ts hosted with ❤ by GitHub

Set up gulp task to bundle dependencies

Our index.html only references a style.css and a deps.js file. Those files are a bundle with all our dependencies. To create that bundle we will use gulp. We need this gulpfile.js on the root of our project:

/// <binding Clean='clean, bundle' />
var gulp = require('gulp');
var del = require('del');
var concat = require("gulp-concat");
var rename = require("gulp-rename");
var uglify = require("gulp-uglify");
var concatCss = require("gulp-concat-css");
var config = {
scriptsPath: "./wwwroot/scripts/",
cssPath: "./wwwroot/css/",
depsFiles: [
"./node_modules/systemjs/dist/system.src.js",
"./node_modules/es6-shim/es6-shim.js",
"./node_modules/rxjs/bundles/Rx.js",
"./node_modules/angular2/bundles/angular2.dev.js",
"./node_modules/angular2/bundles/http.js",
"./node_modules/angular2/bundles/angular2-polyfills.js"
],
cssFiles: [
"./node_modules/bootstrap/dist/css/bootstrap.min.css"
]
};
// Bundle js and css
gulp.task('bundle', ['bundle:js', 'bundle:css']);
// Bundle js files
gulp.task('bundle:js', function () {
return gulp.src(config.depsFiles)
.pipe(concat("deps.js"))
.pipe(gulp.dest(config.scriptsPath))
.pipe(rename("deps.min.js"))
.pipe(uglify())
.pipe(gulp.dest(config.scriptsPath));
});
// Bundle css files
gulp.task('bundle:css', function () {
return gulp.src(config.cssFiles)
.pipe(concatCss("styles.css"))
.pipe(gulp.dest(config.cssPath));
});
// Clean up copied scripts and generated js files
gulp.task("clean", function () {
return del([
config.scriptsPath,
config.cssPath
]);
});
// The default task (called when you run `gulp` from cli)
gulp.task("default", ["bundle"]);
view raw gulpfile.js hosted with ❤ by GitHub

We want to create our bundles automatically when we rebuild or clean the project (not in every build).

Open Task Runner Explorer and bind the clean and bundle tasks to the “Clean” Visual Studio event:

GulpBinding

This will trigger our tasks always that we clean or rebuild our solution. You can also add the bundle task to the “Before Build” event… but it can takes 8-10 seconds and normally the dependencies are not changing in every build.

We have a pre-configured project that hopefully will save us some time in the future!

If you know how to make it better please leave me a comment!

Have fun with ASP.NET Core and Angular 2!