Natural Language Explanations for Query Results论文学习

研究内容

• providing NL explanations to query answers（为查询答案提供自然语言解释）
• The answers that we present are importantly based on the provenance of tuples in the query result, detailing not only the results but also their explanations.（我们给出的答案主要基于查询结果中元组的出处，不仅详细说明了结果，还详细说明了它们的解释。）

技术方案

• We propose a novel approach of presenting provenance information for answers of NL queries.The method can transform provenance information to NL, by leveraging the original NL query structure. （我们提出了一种为 NL 查询的答案呈现出处信息的新方法。这个方法利用原始 NL 查询结构将出处信息转换为自然语言）
  1. modify NaLIR so that we store exactly which parts of the NL query translate to which parts of the formal query.（修改NaLIR，以便准确地存储NL查询的哪些部分转换为正式查询的哪些部分）
  2. modify the provenance aware engine so that it stores which parts of the formal query “contribute” to which parts of the provenance.（修改起源感知引擎，以便它存储正式查询的哪些部分“贡献”到出处的哪些部分）
  3. By composing these two “mappings” (text-to-query-parts and query-parts-to-provenance) we infer which parts of the NL query text are related to which provenance parts.（通过组合这两个“映射”(文本到查询部分和查询部分到出处)，我们可以推断出NL查询文本的哪些部分与哪些出处部分相关。）
  4. Finally, we use the latter information in an “inverse” manner, to translate the provenance to NL text.（最后，我们以一种“反向”的方式使用后一种信息，将出处转换为NL文本。）
• since provenance information is typically large and complex, we present two solutions for its effective presentation as NL text:（由于出处信息通常很大且很复杂，因此我们提出了两种解决方案，以将其有效地呈现为 NL 文本：一种基于出处分解，具有与 NL 案例相关的新需求，另一种基于摘要。）
  • one that is based on provenance factorization, with novel desiderata relevant to the NL case.
  • and one that is based on summarization.

 

接下来的两部分是在论文NLProv: Natural Language Provenance中叙述的

系统架构

We have implemented our solution in a system prototype called NLProv, forming an end-to-end NL interface to database querying where the NL queries, answers and provenance information are all expressed in NL.（我们已经在一个名为 NLProv 的系统原型中实现了我们的解决方案，形成了一个端到端的 NL 接口到数据库查询，其中 NL 查询、答案和出处信息都用 NL 表示。）

NLProv is implemented in JAVA 8 with JAVAFX GUI using SceneBuilder, and runs on Windows 8. It uses MySQL server as its underlying database management system.（NLProv 在 JAVA 中使用 SceneBuilder 通过 JAVAFX GUI 实现，并在 Windows 8 上运行。它使用 MySQL 服务器作为其底层数据库管理系统。）

• First, the user enters a query in Natural Language. This NL sentence is fed to the augmented NaLIR system which interprets it and generates a formal query.（首先，用户用自然语言输入查询。这个 NL 句子被馈送到增强的 NaLIR 系统，该系统对其进行解释并生成正式查询。）
  • a parser generates the dependency tree for the NL query. （解析器为 NL 查询生成依赖树）
  • the nodes of the tree are mapped to attributes in the tables of the database and to functions, to form a formal query.（树的节点被映射到数据库表中的属性和函数，以形成正式查询。）
  • NLProv stores the mapping from the nodes of the dependency tree to the query variables.（NLProv 存储从依赖树的节点到查询变量的映射）
• Once a query has been produced, NLProv uses the SelP system to evaluate it while storing the provenance, keeping track of the mapping of dependency tree nodes to parts of the provenance.（生成查询后，NLProv 使用 SelP 系统对其进行评估，同时存储出处，跟踪依赖树节点到出处部分的映射。）
• The provenance information is then factorized and the factorization is compiled to an NL answer containing explanations.（对出处信息进行因式分解，并将因式分解编译为包含解释的 NL 答案。）
• Finally, the factorized answer is shown to the user. If the answer contains excessive details and is too difficult to understand, the user may choose to view summarizations of different nesting levels.（最后，将分解后的答案显示给用户。如果答案包含过多的细节并且太难理解，用户可以选择查看不同嵌套级别的摘要。）

如何做实验

• The operation of NLProv will be demonstrated with respect to the Microsoft database of publications.（NLProv 的操作将通过 Microsoft 出版物数据库进行演示）
• For the first part of the demonstration we will use a set of pre-defined questions of varying complexity levels.（在演示的第一部分，我们将使用一组不同复杂程度的预定义问题。）
• We will then allow participants to pose, in natural language, queries of their liking with respect to the publications dataset. （然后，我们将允许参与者以自然语言对出版物数据集提出他们喜欢的查询。）

 

接下来的实验研究部分是在论文Provenance for Natural Language Queries中叙述的

实验研究

• We have examined the usefulness of the system through a user study, involving 15 non-expert users.（我们通过一项涉及 15 名非专家用户的用户研究来检验该系统的有效性。）
  • For each of the NL queries, users were shown the NL provenance computed by NLProv for cases of single derivations, factorized and summarized answers for multiple derivations (where applicable).（对于每个 NL 查询，向用户显示 NLProv 为单推导情况计算的 NL 出处、多重推导的分解和汇总答案（如果适用）。）
  • We have asked users three questions about each case, asking them to rank the results on a 1–5 scale. The summarized explanation received the highest scores on all accounts.（我们针对每个案例向用户提出了三个问题，要求他们按照 1-5 的等级对结果进行排名。总结性解释获得最高分。）
• Another facet of our experimental study includes runtime experiments to examine the scalability of our algorithms.（我们实验研究的另一个方面包括实验的运行时间，以检查我们算法的可扩展性。）

 

接下来的新增部分是在论文Explaining Natural Language query results中叙述的

This paper is an extended version of our PVLDB 2017 paper and includes a new section on the translation of provenance to NL for UCQs, a new section on a generalized solution that is not specific to NaLIR and a discussion of the use of other provenance models, new and comprehensive experiments and an extended in-depth review of related work.（这篇论文是我们 PVLDB 2017 论文的扩展版本，包括一个关于将 UCQs 的出处翻译成 NL 的新部分，一个关于非 NaLIR 的通用解决方案的新部分，以及对其他出处模型使用的讨论， 新的和全面的实验以及对相关工作的扩展深入审查。）

• So far, the solution has been limited to Conjunctive Queries, and the paper extend it to account for unions thereof (UCQs). Recall that in thefirst step, the system takes a Natural Language query and translates it to a dependency tree, while maintaining the dependency-to-query-mapping mapping. The difference here is that a tree node can now be mapped to several variables.（到目前为止，该解决方案仅限于连接查询，本文将其扩展到解释其联合 (UCQ)。 回想一下，在第一步中，系统采用自然语言查询并将其转换为依赖树，同时维护依赖到查询映射的映射。 这里的区别在于现在可以将树节点映射到多个变量。）
• The solution in the paper “marries”, for the first time to our knowledge, two fields: (1) Natural Language Interfaces to Databases and (2) Data Provenance. For each of these two, the paper has made choices in the implementation: NaLIR for the NLIDB, as well as a semiring-like value-level provenance model. Next revisit these choices and discuss alternatives in detail.（据我们所知，论文中的解决方案第一次“结合”了两个领域：(1) 数据库的自然语言接口和 (2) 数据来源。 对于这两者中的每一个，本文都在实现中做出了选择：用于 NLIDB 的 NaLIR，以及类似半环的价值级别来源模型。 接下来重新审视这些选择并详细讨论替代方案。）
  • NLProv can use any existing system that transform Natural Language question to formal query and also return partial mapping from the dependency tree nodes to the query parts. However, not all of the techniques for Natural Language Interface design are designed in a way that allows generation of this mapping.（NLProv 可以使用任何现有系统将自然语言问题转换为正式查询，并返回从依赖树节点到查询部分的部分映射。 然而，并非所有自然语言界面设计技术的设计方式都允许生成此映射。）
  • To this end, the paper proposes an alternative architecture. This architecture does not rely on the query builder to also generate the partial mapping τ from the dependency tree nodes to the query parts. Instead, the paper has added an additional block, Mapper that receives as input the dependency tree along with the generated query and outputs the mapping τ . Note that generating the dependency tree may be done using existing tools such as the Stanford Parser, independently of whether the NLIDB generates it (as NaLIR does) or not (as is the case with semantic parsers).（为此，论文提出了一种替代架构。 这种架构不依赖于查询构建器来生成从依赖树节点到查询部分的部分映射 τ。 相反，该论文添加了一个额外的块 Mapper，它接收依赖树以及生成的查询作为输入并输出映射 τ 。 请注意，可以使用诸如斯坦福解析器之类的现有工具生成依赖树，而与 NLIDB 是否生成它（如 NaLIR 那样）（如语义解析器的情况）无关。）
• The paper has performed an experimental study to assess NLProv through two prisms: (1) the quality of answers produced by the system and (2) the efficiency of the algorithms in terms of execution time.（该论文进行了一项实验研究，通过两个棱镜来评估 NLProv：(1) 系统生成的答案的质量和 (2) 算法在执行时间方面的效率。）
  • The paper has examined the usefulness of the system through a user study, involving 22 non-expert users. The user study was conducted in two phases: First, we asked 15 users to evaluate the solution for SPJ queries, where in the second phase 7 different users were requested to evaluate the solution for union queries.（该论文通过一项涉及 22 名非专家用户的用户研究来检验该系统的实用性。 用户研究分两个阶段进行：首先，我们要求 15 个用户评估 SPJ 查询的解决方案，在第二阶段，要求 7 个不同的用户评估联合查询的解决方案。）
  • We have asked users three questions about each case, asking them to rank the results on a 1–5 scale. The summarized explanation received the highest scores on all accounts.（我们针对每个案例向用户提出了三个问题，要求他们按照 1-5 的等级对结果进行排名。总结性解释获得最高分。）
  • Another facet of our experimental study includes runtime experiments to examine the scalability of our algorithms.（我们实验研究的另一个方面包括实验的运行时间，以检查我们算法的可扩展性。）