Apache Spark create vertices from String -
given string
val s = "my-spark-app"
how can vertices created in following way spark?
"my-", "y-s", "-sp", "spa", "par", "ark", "rk-", "k-a", "-ap", "app"
can problem parallelized?
it matter of simple sliding on string:
val n: int = 3 val vertices: seq[(vertexid, string)] = s.sliding(n) .zipwithindex .map{case (s, i) => (i.tolong, s)} .toseq sc.parallelize(vertices)
can problem parallelized?
yes can, if single string doesn't make sense. still, if want:
import org.apache.spark.rdd.rdd val vertices: rdd[(vertexid, string)] = sc.parallelize(s) .sliding(n) .zipwithindex .map{case (cs, i) => (i, cs.mkstring)}
Comments
Post a Comment