3.4 Spark RDD Action操作1-first、count、lookup、collect

云计算 waitig 854℃ 百度已收录 0评论

1 first
def first(): T
first返回RDD中的第一个元素,不排序。
例子:
scala> var rdd1 = sc.makeRDD(Array((“A”,”1”),(“B”,”2”),(“C”,”3”)),2)
rdd1: org.apache.spark.rdd.RDD[(String, String)] = ParallelCollectionRDD[33] at makeRDD at :21

scala> rdd1.first
res14: (String, String) = (A,1)

scala> var rdd1 = sc.makeRDD(Seq(10, 4, 2, 12, 3))
rdd1: org.apache.spark.rdd.RDD[Int] = ParallelCollectionRDD[0] at makeRDD at :21

scala> rdd1.first
res8: Int = 10

2 count
def count(): Long
count返回RDD中的元素数量。
例子:
scala> var rdd1 = sc.makeRDD(Array((“A”,”1”),(“B”,”2”),(“C”,”3”)),2)
rdd1: org.apache.spark.rdd.RDD[(String, String)] = ParallelCollectionRDD[34] at makeRDD at :21

scala> rdd1.count
res15: Long = 3

3 lookup
def lookup(key: K): Seq[V]
lookup用于(K,V)类型的RDD,指定K值,返回RDD中该K对应的所有V值。
例子:
scala> var rdd1 = sc.makeRDD(Array((“A”,0),(“A”,2),(“B”,1),(“B”,2),(“C”,1)))
rdd1: org.apache.spark.rdd.RDD[(String, Int)] = ParallelCollectionRDD[0] at makeRDD at :21

scala> rdd1.lookup(“A”)
res0: Seq[Int] = WrappedArray(0, 2)

scala> rdd1.lookup(“B”)
res1: Seq[Int] = WrappedArray(1, 2)

4 collect
def collect(): Array[T]
collect用于将一个RDD转换成数组。
例子:
scala> var rdd1 = sc.makeRDD(1 to 10,2)
rdd1: org.apache.spark.rdd.RDD[Int] = ParallelCollectionRDD[36] at makeRDD at :21

scala> rdd1.collect
res23: Array[Int] = Array(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)


本文由【waitig】发表在等英博客
本文固定链接:3.4 Spark RDD Action操作1-first、count、lookup、collect
欢迎关注本站官方公众号,每日都有干货分享!
等英博客官方公众号
点赞 (0)分享 (0)