Apache Spark create vertices from String -


given string

val s = "my-spark-app" 

how can vertices created in following way spark?

"my-", "y-s", "-sp", "spa", "par", "ark", "rk-", "k-a", "-ap", "app"

can problem parallelized?

it matter of simple sliding on string:

val n: int = 3  val vertices: seq[(vertexid, string)] =  s.sliding(n)   .zipwithindex   .map{case (s, i) => (i.tolong, s)}   .toseq  sc.parallelize(vertices) 

can problem parallelized?

yes can, if single string doesn't make sense. still, if want:

import org.apache.spark.rdd.rdd  val vertices: rdd[(vertexid, string)] = sc.parallelize(s)   .sliding(n)   .zipwithindex   .map{case (cs, i) => (i, cs.mkstring)} 

Comments

Popular posts from this blog

javascript - Slick Slider width recalculation -

jsf - PrimeFaces Datatable - What is f:facet actually doing? -

angular2 services - Angular 2 RC 4 Http post not firing -