Java 类名：com.alibaba.alink.operator.batch.dataproc.HugeMultiIndexerStringPredictBatchOp
Python 类名：HugeMultiIndexerStringPredictBatchOp

功能介绍

提供ID转换为字符串的功能，与 HugeMultiStringIndexerPredictBatchOp 功能相反。
由 MultiStringIndexerTrainBatchOp 生成词典模型，将输入数据的ID转化成原文。

参数说明

名称 中文名称 描述 类型 是否必须？ 取值范围 默认值
selectedCols 选择的列名 计算列对应的列名列表 String[] ✓ 所选列类型为 [LONG]
handleInvalid 未知token处理策略 未知token处理策略。”keep”表示用最大id加1代替, “skip”表示补null， “error”表示抛异常 String “KEEP”, “ERROR”, “SKIP” “KEEP”
outputCols 输出结果列列名数组 输出结果列列名数组，可选，默认null String[] null
reservedCols 算法保留列名 算法保留列 String[] null

代码示例

Python 代码

  1. from pyalink.alink import *
  3. import pandas as pd
  5. useLocalEnv(1)
  7. df = pd.DataFrame([
  8.     [1, "football", "apple"],
  9.     [2, "football", "apple"],
  10.     [3, "football", "apple"],
  11.     [4, "basketball", "apple"],
  12.     [5, "basketball", "apple"],
  13.     [6, "tennis", "pair"],
  14.     [7, "tennis", "pair"],
  15.     [8, "pingpang", "banana"],
  16.     [9, "pingpang", "banana"],
  17.     [0, "baseball", "banana"]
  18. ])
  20. data = BatchOperator.fromDataframe(df, schemaStr='id long, f0 string, f1 string')
  22. stringindexer = MultiStringIndexerTrainBatchOp()\
  23.     .setSelectedCols(["f0", "f1"])\
  24.     .setStringOrderType("frequency_asc")
  26. model = stringindexer.linkFrom(data)
  28. predictor = HugeMultiStringIndexerPredictBatchOp()\
  29.     .setSelectedCols(["f0", "f1"])
  30. result = predictor.linkFrom(model, data)
  32. stringPredictor = HugeMultiIndexerStringPredictBatchOp()\
  33.     .setSelectedCols(["f0", "f1"])\
  34.     .setOutputCols(["f0_source", "f1_source"])
  35. stringPredictor.linkFrom(model, result).print();

Java 代码

  1. import org.apache.flink.types.Row;
  3. import com.alibaba.alink.operator.batch.BatchOperator;
  4. import com.alibaba.alink.operator.batch.source.MemSourceBatchOp;
  5. import org.junit.Test;
  7. import java.util.Arrays;
  8. import java.util.List;
  10. public class HugeMultiIndexerStringPredictBatchOpTest {
  11.     @Test
  12.     public void testHugeMultiStringIndexerPredict() throws Exception {
  13.         List <Row> df = Arrays.asList(
  14.             Row.of(1L, "football", "apple"),
  15.             Row.of(2L, "football", "apple"),
  16.             Row.of(3L, "football", "apple"),
  17.             Row.of(4L, "basketball", "apple"),
  18.             Row.of(5L, "basketball", "apple"),
  19.             Row.of(6L, "tennis", "pair"),
  20.             Row.of(7L, "tennis", "pair"),
  21.             Row.of(8L, "pingpang", "banana"),
  22.             Row.of(9L, "pingpang", "banana"),
  23.             Row.of(0L, "baseball", "banana")
  24.         );
  25.         // baseball 1
  26.         // basketball,pair,tennis,pingpang 2
  27.         // footbal,banana 3
  28.         // apple 5
  29.         BatchOperator <?> data = new MemSourceBatchOp(df, "id long,f0 string,f1 string");
  30.         BatchOperator <?> stringindexer = new MultiStringIndexerTrainBatchOp()
  31.             .setSelectedCols("f0", "f1")
  32.             .setStringOrderType("frequency_asc");
  33.         BatchOperator model = stringindexer.linkFrom(data);
  34.         model.lazyPrint(10);
  36.         BatchOperator <?> predictor = new HugeMultiStringIndexerPredictBatchOp().setSelectedCols("f0", "f1");
  37.         BatchOperator result = predictor.linkFrom(model, data);
  38.         result.lazyPrint(10);
  40.         BatchOperator <?> stringPredictor = new HugeMultiIndexerStringPredictBatchOp().setSelectedCols("f0", "f1")
  41.             .setOutputCols("f0_source", "f1_source");
  42.         stringPredictor.linkFrom(model, result).print();
  43.     }
  44. }

运行结果

| column_index | token | token_index | | —- | —- | —- |

| 1 | apple | 2 |

| 1 | pair | 0 |

| 1 | banana | 1 |

| -1 | {“selectedCols”:”[“f0”,”f1”]”,”selectedColTypes”:”[“VARCHAR”,”VARCHAR”]”} | null |

| 0 | football | 4 |

| 0 | baseball | 0 |

| 0 | basketball | 1 |

| 0 | tennis | 2 |

| 0 | pingpang | 3 |

| id | f0 | f1 | | —- | —- | —- |

| 1 | 4 | 2 |

| 2 | 4 | 2 |

| 6 | 2 | 0 |

| 7 | 2 | 0 |

| 5 | 1 | 2 |

| 3 | 4 | 2 |

| 9 | 3 | 1 |

| 0 | 0 | 1 |

| 4 | 1 | 2 |

| 8 | 3 | 1 |

| id | f0 | f1 | f0_source | f1_source | | —- | —- | —- | —- | —- |

| 5 | 1 | 2 | basketball | apple |

| 2 | 4 | 2 | football | apple |

| 6 | 2 | 0 | tennis | pair |

| 4 | 1 | 2 | basketball | apple |

| 8 | 3 | 1 | pingpang | banana |

| 3 | 4 | 2 | football | apple |

| 7 | 2 | 0 | tennis | pair |

| 9 | 3 | 1 | pingpang | banana |

| 0 | 0 | 1 | baseball | banana |

| 1 | 4 | 2 | football | apple |