Importing multi-valued field into Solr from mySQL using Solr Data Import Handler

http://stackoverflow.com/questions/20233837/importing-multi-valued-field-into-solr-from-mysql-using-solr-data-import-handler

 

Q:

We have the following two tables in our mySQL:

mysql> describe comment;
+--------------+--------------+------+-----+---------+-------+
| Field        | Type         | Null | Key | Default | Extra |
+--------------+--------------+------+-----+---------+-------+
| id           | int(11)      | YES  |     | NULL    |       |
| blogpost_id  | int(11)      | YES  |     | NULL    |       |
| comment_text | varchar(256) | YES  |     | NULL    |       |
+--------------+--------------+------+-----+---------+-------+

mysql> describe comment_tags;
+------------+-------------+------+-----+---------+-------+
| Field      | Type        | Null | Key | Default | Extra |
+------------+-------------+------+-----+---------+-------+
| comment_id | int(11)     | YES  |     | NULL    |       |
| tag        | varchar(80) | YES  |     | NULL    |       |
+------------+-------------+------+-----+---------+-------+

Where each comment can have multiple tags. We can import the entire comment into Solr using the Data Import Handler. However I am not sure how to import the tags for each comment into a multivalued field defined the schema.xml for each comment document.

 

A:

You can also use GROUP_CONCAT with a Seperator(e.g " , ") and then try something like this :

<dataConfig>
<!-- dataSource is just an example. Included just for completeness. -->
 <dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver" url="jdbc:mysql://localhost/db" user="root" password="root"/>
   <document>
     <entity name="comment" pk="id" query="SELECT *, group_concat(tags) as comment_tags FROM comment" transformer="RegexTransformer">
      <field column="blogpost_id" name="blogpost_id"/>
      <field column="comment_text" name="comment_text" />
      <field column="tag" name="comment_tags" splitBy = "," />       
    </entity>
  </document>    
</dataConfig> 

It'll increase the Performance and also will remove the Dependency of another query.

 

------------------

solr在DIH中常见的场景：

表A一条记录对应表B多条记录（明细）。