pyspark - Joining multiple key-value pair files in spark -
i have multiple features_xx , id_xx files, , each of them maintain order in correspondence.
example - features_00 file has 50 rows each 5 numbers. id_00 file contains id each of these 50. order important in both files. have 1-1 mapping. , there 100 of these pair files.
i want have mapping in 1 dataframe. how can stitch/zip them in common spark dataframe id 1 of columns alongside features.
Comments
Post a Comment