Java 类名:com.alibaba.alink.operator.batch.dataproc.vector.VectorImputerTrainBatchOp
Python 类名:VectorImputerTrainBatchOp
功能介绍
训练Vecotor 缺失值填充模型的组件,输出模型。
填充策略包含最大值,最小值,均值和指定数值4种。
参数说明
| 名称 | 中文名称 | 描述 | 类型 | 是否必须? | 取值范围 | 默认值 |
|---|---|---|---|---|---|---|
| selectedCol | 选中的列名 | 计算列对应的列名 | String | ✓ | 所选列类型为 [DENSE_VECTOR, SPARSE_VECTOR, STRING, VECTOR] | |
| fillValue | 填充缺失值 | 自定义的填充值。当strategy为value时,读取fillValue的值 | Double | null | ||
| strategy | 缺失值填充规则 | 缺失值填充的规则,支持mean,max,min或者value。选择value时,需要读取fillValue的值 | String | “MEAN”, “MIN”, “MAX”, “VALUE” | “MEAN” |
代码示例
Python 代码
from pyalink.alink import *import pandas as pduseLocalEnv(1)df = pd.DataFrame([["1:3,2:4,4:7", 1],["1:3,2:NaN", 3],["2:4,4:5", 4]])data = BatchOperator.fromDataframe(df, schemaStr="vec string, id bigint")vecFill = VectorImputerTrainBatchOp().setSelectedCol("vec")model = data.link(vecFill)VectorImputerPredictBatchOp().setOutputCol("vec1").linkFrom(model, data).print()
Java 代码
import org.apache.flink.types.Row;import com.alibaba.alink.operator.batch.BatchOperator;import com.alibaba.alink.operator.batch.dataproc.vector.VectorImputerPredictBatchOp;import com.alibaba.alink.operator.batch.dataproc.vector.VectorImputerTrainBatchOp;import com.alibaba.alink.operator.batch.source.MemSourceBatchOp;import org.junit.Test;import java.util.Arrays;import java.util.List;public class VectorImputerTrainBatchOpTest {@Testpublic void testVectorImputerTrainBatchOp() throws Exception {List <Row> df = Arrays.asList(Row.of("1:3,2:4,4:7", 1),Row.of("1:3,2:NaN", 3),Row.of("2:4,4:5", 4));BatchOperator <?> data = new MemSourceBatchOp(df, "vec string, id int");BatchOperator <?> vecFill = new VectorImputerTrainBatchOp().setSelectedCol("vec");BatchOperator <?> model = data.link(vecFill);new VectorImputerPredictBatchOp().setOutputCol("vec1").linkFrom(model, data).print();}}
运行结果
| vec | id | vec1 | | —- | —- | —- |
| 1:3,2:4,4:7 | 1 | 1:3.0 2:4.0 4:7.0 |
| 1:3,2:NaN | 3 | 1:3.0 2:4.0 |
| 2:4,4:5 | 4 | 2:4.0 4:5.0 |
