Scala实现逻辑回归分类，Titanic

本文主要是介绍Scala实现逻辑回归分类，Titanic，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！

https://yixuan.cos.name/cn/2015/04/spark-beginner-1/

1.读取文件

import scala.io.Source
object myfirst {//titanic,LR
  def main(args: Array[String]) {val data= Source.fromFile("D:\\IDEA\\_01\\train.csv")data.foreach(print)}
}

import org.apache.spark.{SparkContext, SparkConf}
object myfirst{//titanic,LR
  def main(args: Array[String]) {val conf=new SparkConf().setAppName("regression").setMaster("local[1]")val sc=new SparkContext(conf)val lines = {sc.textFile("D:\\IDEA\\_01\\train.csv")}//lines.foreach(print)
  }
}

2.数据预处理

2.1数据结构

spark MLlib之数据类型http://uohzoaix.github.io/studies/2014/10/14/sparkMLlib(1)/

dense和sparse矩阵

LabeldPoint

import org.apache.spark.mllib.linalg.Vectorsimport org.apache.spark.mllib.regression.LabeledPoint

// Create a labeled point with a positive label and a dense feature vector.

val pos = LabeledPoint(1.0, Vectors.dense(1.0, 0.0, 3.0))

// Create a labeled point with a negative label and a sparse feature vector.

val neg = LabeledPoint(0.0, Vectors.sparse(3, Array(0, 2), Array(1.0, 3.0)))

2.2特征提取

tfidf

word2vec

countvectorizer

2.3特征转换

https://my.oschina.net/hblt147/blog/1519465

3.建模

• SVMWithSGD
• LogisticRegressionWithLBFGS
• LogisticRegressionWithSGD
• LinearRegressionWithSGD
• RidgeRegressionWithSGD
• LassoWithSGD

模型选择

4.模型训练

5.模型优化

6.模型优化

这篇关于Scala实现逻辑回归分类，Titanic的文章就介绍到这儿，希望我们推荐的文章对编程师们有所帮助！

---