注意使用java函数一定要导入相应的类！要不然会报错！对于自定义工具类怎么导入还没有研究

kettle内无论你取出的字段值是为空，还是为null值，若经过JAVA脚本的处理后都会变成NULL 字符串了，java代码中的integer类型无法直接判断，必须用,就算在128字节以内的也判断不了（不会报错） //如果指定的数与参数相等返回0。 //如果指定的数小于参数返回-1。 //如果指定的数大于参数返回1。 Integer1.compareTo(Integer2)

本次测试内容

需要将原来的数据清洗到新的表中！新表中有新的字段需要随机生成数据


流程图

image.png

解析

表输入（由于旧数据中没有字段所以 首先现 在sql中虚拟字段）


image.png

Java 生成随机数据


image.png
image.png

  1.             import java.util.*;
  2.             import java.text.*;
  3.             import java.util.regex.Pattern;
  4.             public boolean processRow(StepMetaInterface smi, StepDataInterface sdi) throws KettleException {
  7.               Object[] r = getRow();
  9.               if (r == null) {
  10.                 setOutputDone();
  11.                 return false;
  12.               }
  16.                 //如果为空着进行随机添加
  17.                 String book_insert_date = get(Fields.In, "book_insert_date").getString(r);
  18.                  if(book_insert_date==null || book_insert_date==""){
  19.                     String str = "";
  20.                     Date date = new Date();
  21.                     try {
  22.                         str = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(date)+".0";
  23.                     } catch (Exception e) {
  24.                         str="";
  25.                         e.printStackTrace();
  26.                     }
  27.                     get(Fields.Out, "book_insert_date").setValue(r, str);
  28.                 }
  30.                  //设置uuid
  31.                 get(Fields.Out, "uuid").setValue(r, UUID.randomUUID().toString().replaceAll("-", ""));
  32.                 //设置推荐指数 随机0-5、
  33.                 get(Fields.Out, "level").setValue(r, new Random().nextInt(6));
  34.                 //设置副本数 随机1000
  35.                 get(Fields.Out, "copy").setValue(r, new Random().nextInt(1000)+1);
  37.                 //获取中图号  注意sql 判空 这里判断也行
  38.                 String book_ztype = get(Fields.In, "book_ztype").getString(r);
  39.                 String chineseType = book_ztype.substring(0,1);
  40.                 String pattern = "^[a-zA-Z]"; //包含 a-Z
  41.                 if(!Pattern.matches(pattern, chineseType)){ //如果包含a-Z 就强行置为Z
  42.                       chineseType = "Z";
  43.                 }
  44.                 get(Fields.Out, "chineseType").setValue(r,chineseType.toUpperCase());
  47.                   putRow(data.outputRowMeta, r);
  52.               return true;
  53.             }

表输出

image.png

点击mian函数进去java执行主体

image.png
image.png

获取输入值对其进行判断等操作

image.png

对上面的输入值操作之后输出理想的值

image.png

有也可以自定义输出值

image.png

代码参考

  1. <?xml version="1.0" encoding="UTF-8"?>
  2. <transformation-steps>
  3.     <steps>
  4.         <step>
  5.             <name>Java 代码</name>
  6.             <type>UserDefinedJavaClass</type>
  7.             <description/>
  8.             <distribute>Y</distribute>
  9.             <custom_distribution/>
  10.             <copies>1</copies>
  11.             <partitioning>
  12.                 <method>none</method>
  13.                 <schema_name/>
  14.             </partitioning>
  16.             <definitions>
  17.                 <definition>
  18.                     <class_type>TRANSFORM_CLASS</class_type>
  20.                     <class_name>Processor</class_name>
  22.                     <class_source><![CDATA[import java.util.*;
  24. public boolean processRow(StepMetaInterface smi, StepDataInterface sdi) throws KettleException {
  25.   if (first) {
  26.     first = false;
  27.   }
  29.   Object[] r = getRow();
  31.   if (r == null) {
  32.     setOutputDone();
  33.     return false;
  34.   }
  36.   // It is always safest to call createOutputRow() to ensure that your output row's Object[] is large
  37.   // enough to handle any new fields you are creating in this step.
  38.   r = createOutputRow(r, data.outputRowMeta.size());
  40.    //设置uuid
  41.    get(Fields.Out, "uuid").setValue(r, UUID.randomUUID().toString().replaceAll("-", ""));
  44.   putRow(data.outputRowMeta, r);
  46.   return true;
  47. }
  48. ]]></class_source>
  49.                 </definition>
  50.             </definitions>
  51.             <fields>
  52.                 <field>
  53.                     <field_name>uuid</field_name>
  55.                     <field_type>String</field_type>
  57.                     <field_length>-1</field_length>
  59.                     <field_precision>-1</field_precision>
  61.                 </field>
  62.             </fields><clear_result_fields>N</clear_result_fields>
  63.             <info_steps></info_steps><target_steps></target_steps><usage_parameters></usage_parameters>    <cluster_schema/>
  64.             <remotesteps>
  65.                 <input>
  66.                 </input>
  67.                 <output>
  68.                 </output>
  69.             </remotesteps>
  70.             <GUI>
  71.                 <xloc>400</xloc>
  72.                 <yloc>176</yloc>
  73.                 <draw>Y</draw>
  74.             </GUI>
  75.         </step>
  77.         <step>
  78.             <name>表输入</name>
  79.             <type>TableInput</type>
  80.             <description/>
  81.             <distribute>Y</distribute>
  82.             <custom_distribution/>
  83.             <copies>1</copies>
  84.             <partitioning>
  85.                 <method>none</method>
  86.                 <schema_name/>
  87.             </partitioning>
  88.             <connection>mysql5.6_hadoop</connection>
  89.             <sql>SELECT
  90.                 log_id
  91.                 , three_name
  92.                 , book_isbn
  93.                 , create_by
  94.                 , create_time
  95.                 , door_way
  96.                 , field_info
  97.                 , jp_record_source
  98.                 , jp_source_flag
  99.                 , log_book_type
  100.                 , log_bookid
  101.                 , log_details
  102.                 , log_from
  103.                 , log_ip
  104.                 , log_tableid
  105.                 , log_table_name
  106.                 , log_time
  107.                 , log_type
  108.                 , log_userid
  109.                 , room_num
  110.                 , time_length
  111.                 , update_by
  112.                 , update_time
  113.                 , vp_record_source
  114.                 , vp_source_flag
  115.                 FROM re_logs where log_from=&#34;门禁管理系统&#34;
  116.             </sql>
  117.             <limit>0</limit>
  118.             <lookup/>
  119.             <execute_each_row>N</execute_each_row>
  120.             <variables_active>N</variables_active>
  121.             <lazy_conversion_active>N</lazy_conversion_active>
  122.             <cluster_schema/>
  123.             <remotesteps>
  124.                 <input>
  125.                 </input>
  126.                 <output>
  127.                 </output>
  128.             </remotesteps>
  129.             <GUI>
  130.                 <xloc>144</xloc>
  131.                 <yloc>192</yloc>
  132.                 <draw>Y</draw>
  133.             </GUI>
  134.         </step>
  136.         <step>
  137.             <name>表输出</name>
  138.             <type>TableOutput</type>
  139.             <description/>
  140.             <distribute>Y</distribute>
  141.             <custom_distribution/>
  142.             <copies>1</copies>
  143.             <partitioning>
  144.                 <method>none</method>
  145.                 <schema_name/>
  146.             </partitioning>
  147.             <connection>mysql5.6_hadsource</connection>
  148.             <schema/>
  149.             <table>sys_doorlog</table>
  150.             <commit>1000</commit>
  151.             <truncate>N</truncate>
  152.             <ignore_errors>N</ignore_errors>
  153.             <use_batch>Y</use_batch>
  154.             <specify_fields>Y</specify_fields>
  155.             <partitioning_enabled>N</partitioning_enabled>
  156.             <partitioning_field/>
  157.             <partitioning_daily>N</partitioning_daily>
  158.             <partitioning_monthly>Y</partitioning_monthly>
  159.             <tablename_in_field>N</tablename_in_field>
  160.             <tablename_field/>
  161.             <tablename_in_table>Y</tablename_in_table>
  162.             <return_keys>N</return_keys>
  163.             <return_field/>
  164.             <fields>
  165.                 <field>
  166.                     <column_name>create_by</column_name>
  167.                     <stream_name>create_by</stream_name>
  168.                 </field>
  169.                 <field>
  170.                     <column_name>create_time</column_name>
  171.                     <stream_name>create_time</stream_name>
  172.                 </field>
  173.                 <field>
  174.                     <column_name>door_way</column_name>
  175.                     <stream_name>door_way</stream_name>
  176.                 </field>
  177.                 <field>
  178.                     <column_name>log_details</column_name>
  179.                     <stream_name>log_details</stream_name>
  180.                 </field>
  181.                 <field>
  182.                     <column_name>log_from</column_name>
  183.                     <stream_name>log_from</stream_name>
  184.                 </field>
  185.                 <field>
  186.                     <column_name>log_tableid</column_name>
  187.                     <stream_name>log_tableid</stream_name>
  188.                 </field>
  189.                 <field>
  190.                     <column_name>log_table_name</column_name>
  191.                     <stream_name>log_table_name</stream_name>
  192.                 </field>
  193.                 <field>
  194.                     <column_name>log_time</column_name>
  195.                     <stream_name>log_time</stream_name>
  196.                 </field>
  197.                 <field>
  198.                     <column_name>log_type</column_name>
  199.                     <stream_name>log_type</stream_name>
  200.                 </field>
  201.                 <field>
  202.                     <column_name>log_userid</column_name>
  203.                     <stream_name>log_userid</stream_name>
  204.                 </field>
  205.                 <field>
  206.                     <column_name>update_by</column_name>
  207.                     <stream_name>update_by</stream_name>
  208.                 </field>
  209.                 <field>
  210.                     <column_name>update_time</column_name>
  211.                     <stream_name>update_time</stream_name>
  212.                 </field>
  213.                 <field>
  214.                     <column_name>uuid</column_name>
  215.                     <stream_name>uuid</stream_name>
  216.                 </field>
  217.                 <field>
  218.                     <column_name>id</column_name>
  219.                     <stream_name>log_id</stream_name>
  220.                 </field>
  221.                 <field>
  222.                     <column_name>door_aisle_remark</column_name>
  223.                     <stream_name>log_ip</stream_name>
  224.                 </field>
  225.             </fields>
  226.             <cluster_schema/>
  227.             <remotesteps>
  228.                 <input>
  229.                 </input>
  230.                 <output>
  231.                 </output>
  232.             </remotesteps>
  233.             <GUI>
  234.                 <xloc>640</xloc>
  235.                 <yloc>192</yloc>
  236.                 <draw>Y</draw>
  237.             </GUI>
  238.         </step>
  240.     </steps>
  241.     <order>
  242.         <hop>
  243.             <from>Java 代码</from>
  244.             <to>表输出</to>
  245.             <enabled>Y</enabled>
  246.         </hop>
  248.         <hop>
  249.             <from>表输入</from>
  250.             <to>Java 代码</to>
  251.             <enabled>Y</enabled>
  252.         </hop>
  254.     </order>
  255.     <notepads>
  256.     </notepads>
  257.     <step_error_handling>
  258.     </step_error_handling>
  259. </transformation-steps>

🍔注意

数据结构有变化是需要清除下缓存要不然可能找不到 新的结构字段

image.png