Dynamic Prompting: A Unified Framework for Prompt Tuning

the efficacy of employing fixed soft prompts with a predetermined position for concatenation with inputs for all instances, irrespective of their inherent disparities, remains uncertain. Variables such as the position, length, and representations of prompts across diverse instances and tasks can substantially influence the performance of prompt tuning. In this context, we provide a theoretical analysis, which reveals that optimizing the position of the prompt to encompass the input can capture additional semantic information that traditional prefix or postfix prompt tuning methods fail to capture. Building upon our analysis, we present a unified dynamic prompt (DP) tuning strategy that dynamically determines different factors of prompts based on specific tasks and instances.

Our endeavor encompasses a comprehensive theoretical analysis that unravels the potential benefits of optimizing the position for concatenating prompts with inputs, thereby capturing additional semantics that conventional prefix or postfix prompt tuning methods fail to encapsulate. Motivated by our analysis, we propose a unified strategy for dynamic prompt (DP) tuning, wherein different factors are dynamically determined based on the specific tasks or instances at hand. An overview of our novel approach is summarized in Figure 2. Specifically, we employ a one-layer feedforward network in conjunction with the Gumbel-Softmax technique [22] to learn the categorical distribution of position or length, facilitating optimization at both the task and instance levels. To thoroughly evaluate the efficacy of dynamic concatenation position, length, and prompt pools, we conduct independent as well as simultaneous experiments on the SuperGlue benchmark [52]. The empirical results of our study showcase remarkable performance improvements attained through dynamic prompting, spanning models of varying sizes. This advancement effectively narrows the gap between prompt tuning and traditional fine-tuning, as illustrated in Figure 1.

Prompt-based Graph Model For Joint Liberal Event Extraction And Event Schema Induction

https://arxiv.org/abs/2403.12526

Manually predefined event schemas have limited coverage and are hard to migrate across domains. Therefore, the researchers propose Liberal Event Extraction (LEE), which aims to extract events and discover event schemas simultaneously. However, existing LEE models rely heavily on external language knowledge bases and require the manual development of numerous rules for noise removal and knowledge alignment, which is complex and laborious. To this end, we propose a Prompt-based Graph Model for Liberal Event Extraction (PGLEE). Specifically, we use a prompt-based model to obtain candidate triggers and arguments, and then build heterogeneous event graphs to encode the structures within and between events

Huang et al.[1] first propose Liberal Event Extraction (LEE) to address the above problems, which automatically discovers event schemas and extracts events simultaneously. Nguyen et al.[2] and Sha et al.[3] use entity linking and entity disambiguation for event schema induction, respectively. However, these efforts have the following shortcomings: a. They rely heavily on semantic analysis tools and external knowledge bases and require manual rules to remove noise and construct alignment mappings between multiple language resources. b. They only consider the effect of arguments on trigger classification, ignoring the reverse impact and the inter structure of events. c. Modules in the model are connected in the form of pipelines which can lead error propagation.

This paper proposes an end-to-end approach called PGLEE to implement joint event extraction and event schema induction. We generate candidate event triggers and arguments directly by a prompt-based learning approach without external knowledge bases. Heterogeneous event graphs are constructed to encode semantic interactions within and between the events. We use clustering and label naming to detect events schemas automatically. Experimental results validate the excellent extraction ability of PGLEE on predefined event schemas and the ability to discover new schemas.