【NAACL-HLT-2018】Slot-Gated Modeling for Joint Slot Filling and Intent Prediction

一、Contributions

However, the prior work did not “explicitly” model the relationships between the intent and slots; instead, it applied a joint loss function to “implicitly” consider both cues. Because the slots often highly depend on the intent, this work focuses on how to model the explicit relationships between slots and intent vectors by introducing a slot-gated mechanism.

之前的joint mode只是描述了意图和槽任务之前晦涩的关系,但是本篇论文描述了二者之间明确的关系(因为槽高度依赖意图)。

二、Models

1. Attention-Based RNN Model

1.1 Slot Filling without gate

其实就是针对每个隐藏状态(前后向拼接)进行self attention,但是在简单的数据集比如AITS上,SF增加attention效果没有太大的提升。

1.2 Intent Prediction

这里的attention不用于上面的,这里更简单,不用每两个隐藏状态进行交互,这里只得到一个最终的attention加权和。

2. Slot-Gated Mechanism

其实就是将ID的attention和SF的attention进行交互(SF不加attention的话就用隐藏状态,见Figure2b)得到g,然后让g作为SF的attention的权重,最终受益的是SF。

3. Joint Optimization

极大似然,梯度上升。

三、Refs

paper/reading1