Java 类名：com.alibaba.alink.operator.batch.feature.ChiSqSelectorBatchOp
Python 类名：ChiSqSelectorBatchOp

功能介绍

针对table数据，进行特征筛选

参数说明

名称 中文名称 描述 类型 是否必须？ 取值范围 默认值
labelCol 标签列名 输入表中的标签列名 String ✓
selectedCols 选择的列名 计算列对应的列名列表 String[] ✓
fdr 发现阈值 发现阈值, 默认值0.05 Double 0.05
fpr p value的阈值 p value的阈值，默认值0.05 Double 0.05
fwe 错误率阈值 错误率阈值, 默认值0.05 Double 0.05
numTopFeatures 最大的p-value列个数 最大的p-value列个数, 默认值50 Integer 50
percentile 筛选的百分比 筛选的百分比，默认值0.1 Double 0.1
selectorType 筛选类型 筛选类型，包含”NumTopFeatures”,”percentile”, “fpr”, “fdr”, “fwe”五种。 String “NumTopFeatures”, “PERCENTILE”, “FPR”, “FDR”, “FWE” “NumTopFeatures”

代码示例

Python 代码

  1. from pyalink.alink import *
  3. import pandas as pd
  5. useLocalEnv(1)
  7. df = pd.DataFrame([
  8.     ["a", 1, 1,2.0, True],
  9.     ["c", 1, 2, -3.0, True],
  10.     ["a", 2, 2,2.0, False],
  11.     ["c", 0, 0, 0.0, False]
  12. ])
  14. source = BatchOperator.fromDataframe(df, schemaStr='f_string string, f_long long, f_int int, f_double double, f_boolean boolean')
  16. selector = ChiSqSelectorBatchOp()\
  17.             .setSelectedCols(["f_string", "f_long", "f_int", "f_double"])\
  18.             .setLabelCol("f_boolean")\
  19.             .setNumTopFeatures(2)
  21. selector.linkFrom(source)
  23. modelInfo: ChisqSelectorModelInfo = selector.collectModelInfo()
  25. print(modelInfo.getColNames())

Java 代码

  1. import org.apache.flink.types.Row;
  3. import com.alibaba.alink.operator.batch.BatchOperator;
  4. import com.alibaba.alink.operator.batch.feature.ChiSqSelectorBatchOp;
  5. import com.alibaba.alink.operator.batch.source.MemSourceBatchOp;
  6. import com.alibaba.alink.operator.common.feature.ChisqSelectorModelInfo;
  7. import org.junit.Test;
  9. import java.util.Arrays;
  10. import java.util.List;
  12. public class ChiSqSelectorBatchOpTest {
  13.     @Test
  14.     public void testChiSqSelectorBatchOp() throws Exception {
  15.         List <Row> df = Arrays.asList(
  16.             Row.of("a", 1L, 1, 2.0, true),
  17.             Row.of("c", 1L, 2, -3.0, true),
  18.             Row.of("a", 2L, 2, 2.0, false),
  19.             Row.of("c", 0L, 0, 0.0, false)
  20.         );
  21.         BatchOperator <?> source = new MemSourceBatchOp(df,
  22.             "f_string string, f_long long, f_int int, f_double double, f_boolean boolean");
  23.         ChiSqSelectorBatchOp selector = new ChiSqSelectorBatchOp()
  24.             .setSelectedCols("f_string", "f_long", "f_int", "f_double")
  25.             .setLabelCol("f_boolean")
  26.             .setNumTopFeatures(2);
  27.         selector.linkFrom(source);
  28.         ChisqSelectorModelInfo modelInfo = selector.collectModelInfo();
  29.         System.out.println(modelInfo.toString());
  30.     }
  31. }

运行结果

  1. ------------------------- ChisqSelectorModelInfo -------------------------
  2. Number of Selector Features: 2
  3. Number of Features: 4
  4. Type of Selector: NumTopFeatures
  5. Number of Top Features: 2
  6. Selector Indices: 
  7.     | ColName|ChiSquare|PValue| DF|Selected|
  8.     |--------|---------|------|---|--------|
  9.     |  f_long|        4|0.1353|  2|    true|
  10.     |   f_int|        2|0.3679|  2|    true|
  11.     |f_double|        2|0.3679|  2|   false|
  12.     |f_string|        0|     1|  1|   false|