Patent
Method and apparatus for query expansion based on multimodal cross-vocabulary mapping
| العنوان: | Method and apparatus for query expansion based on multimodal cross-vocabulary mapping |
|---|---|
| Patent Number: | 7,788,099 |
| تاريخ النشر: | August 31, 2010 |
| Appl. No: | 11/733033 |
| Application Filed: | April 09, 2007 |
| مستخلص: | A computer implemented method, apparatus, and computer usable program code for multimodal cross-vocabulary mapping. A corpus of multimodal content is annotated simultaneously using annotations from a plurality of vocabularies to form a set of common annotations. Relationships between a first vocabulary associated with a first modality and a second vocabulary associated with a second modality are identified using the set of common annotations to form a multimodal vocabulary mapping. Items in the first vocabulary associated with the first modality are mapped to items in the second vocabulary associated with the second modality using the multimodal vocabulary mapping. |
| Inventors: | Haubold, Alexander (New York, NY, US); Naphade, Milind R. (Fishkill, NY, US); Natsev, Apostol Ivanov (Harrison, NY, US) |
| Assignees: | International Business Machines Corporation (Armonk, NY, US) |
| Claim: | 1. A computer implemented method for multimodal cross-vocabulary mapping, the computer implemented method comprising: annotating a corpus of multimodal content simultaneously using annotations from a plurality of vocabularies to form a set of common annotations; identifying relationships between a first vocabulary associated with a first modality and a second vocabulary associated with a second modality using the set of common annotations to form a multimodal vocabulary mapping; and mapping with a computing device items in the first vocabulary associated with the first modality to items in the second vocabulary associated with the second modality using the multimodal vocabulary mapping. |
| Claim: | 2. The computer implemented method of claim 1 further comprising: responsive to receiving a query to search for content that includes query items associated with the first vocabulary, identifying a set of items in the second vocabulary that are mapped to the query items associated with the first vocabulary to form a set of mapped items in a second modality; and adding the set of mapped items in the second modality to the query to form an expanded query, wherein the expanded query can be used to search for content in the first modality and the second modality simultaneously. |
| Claim: | 3. The computer implemented method of claim 1 wherein each vocabulary in the plurality of vocabularies is associated with a different modality in the corpus of multimodal content, and wherein two or more different modalities are associated with the corpus of multimodal content. |
| Claim: | 4. The computer implemented method of claim 3 wherein a modality is selected from a group consisting of a text modality, a graphical modality, and an audio modality. |
| Claim: | 5. The computer implemented method of claim 1 wherein the relationships between the first vocabulary associated with the first modality and the second vocabulary associated with the second modality are relationships selected from a group consisting of equivalence relationships, ontological relationships, lexical relationships, statistical relationships, and semantic association relationships. |
| Claim: | 6. The computer implemented method of claim 1 wherein a part of the set of common annotations is extracted automatically without human intervention. |
| Claim: | 7. The computer implemented method of claim 1 wherein the multimodal vocabulary mapping is generated using a test selected from a group consisting of a statistical independence test, a significance test, and a co-occurrence measurement test between items in the first vocabulary and the second vocabulary. |
| Claim: | 8. The computer implemented method of claim 1 further comprising: performing a search for multimodal content in the corpus of multimodal content using the cross-modal vocabulary mapping. |
| Claim: | 9. The computer implemented method of claim 1 further comprising: performing a content filtering of multimodal content in the corpus of multimodal content using the cross-modal vocabulary mapping. |
| Claim: | 10. The computer implemented method of claim 1 further comprising: routing multimodal content in the corpus of multimodal content using the cross-modal vocabulary mapping. |
| Claim: | 11. The computer implemented method of claim 1 further comprising: performing multimodal concept detection associated with the corpus of multimodal content using the cross-modal vocabulary mapping. |
| Claim: | 12. The computer implemented method of claim 1 further comprising: summarizing multimodal content using the cross-modal vocabulary mapping. |
| Claim: | 13. The computer implemented method of claim 1 further comprising: performing multimodal content translation using the cross-modal vocabulary mapping. |
| Claim: | 14. A computer program product comprising: a non-transitory computer-readable medium including computer usable program code for multimodal cross-vocabulary mapping, said computer program product comprising: computer usable program code for annotating a corpus of multimodal content simultaneously using annotations from a plurality of vocabularies to form a set of common annotations; computer usable program code for identifying relationships between a first vocabulary associated with a first modality and a second vocabulary associated with a second modality using the set of common annotations to form a multimodal vocabulary mapping; and computer usable program code for mapping items in the first vocabulary associated with the first modality to items in the second vocabulary associated with the second modality using the multimodal vocabulary mapping. |
| Claim: | 15. The computer program product of claim 14 further comprising: computer usable program code for identifying a set of items in the second vocabulary that are mapped to a query items associated with the first vocabulary to form a set of mapped items in a second modality in response to receiving the query to search for content that includes query items associated with the first vocabulary; and computer usable program code for adding the set of mapped items in the second modality to the query to form an expanded query, wherein the expanded query can be used to search for content in the first modality and the second modality simultaneously. |
| Claim: | 16. An apparatus comprising: a bus system; a communications system connected to the bus system; a memory connected to the bus system, wherein the memory includes computer usable program code; and a processing unit connected to the bus system, wherein the processing unit executes the computer usable program code to annotate a corpus of multimodal content simultaneously using annotations from a plurality of vocabularies to form a set of common annotations; identify relationships between a first vocabulary associated with a first modality and a second vocabulary associated with a second modality using the set of common annotations to form a multimodal vocabulary mapping; and map items in the first vocabulary associated with the first modality to items in the second vocabulary associated with the second modality using the multimodal vocabulary mapping. |
| Claim: | 17. The apparatus of claim 16 wherein the processor unit further executes the computer usable program code to identify a set of items in the second vocabulary that are mapped to query items associated with the first vocabulary to form a set of mapped items in a second modality in response to receiving a query to search for content that includes the query items associated with the first vocabulary and add the set of mapped items in the second modality to the query to form an expanded query, wherein the expanded query can be used to search for content in the first modality and the second modality simultaneously. |
| Claim: | 18. An annotation system for multimodal cross-vocabulary mapping, the system comprising: a set of annotators, wherein the set of annotators annotates a corpus of multimodal content simultaneously using annotations from a plurality of vocabularies to form a set of common annotations; and a vocabulary mapping device, wherein the vocabulary mapping device identifies relationships between a first vocabulary associated with a first modality and a second vocabulary associated with a second modality using the set of common annotations to form a multimodal vocabulary mapping and maps items in the first vocabulary associated with the first modality to items in the second vocabulary associated with the second modality using the multimodal vocabulary mapping. |
| Claim: | 19. The annotation system of claim 18 further comprising: a query expansion device, wherein the query expansion device identifies a set of items in the second vocabulary that are mapped to query items associated with the first vocabulary to form a set of mapped items in a second modality, in response to receiving a query to search for content that includes the query items associated with the first vocabulary, and wherein the query expansion device adds the set of mapped items in the second modality to the query to form an expanded query, wherein the expanded query can be used to search for content in the first modality and the second modality simultaneously. |
| Claim: | 20. The annotation system of claim 18 further comprising: a multimodal content database, wherein the multimodal content database stores a corpus of multimodal content. |
| Current U.S. Class: | 704/270 |
| Patent References Cited: | 6922699 July 2005 Schuetze et al. 2003/0018632 January 2003 Bays et al. 2003/0177112 September 2003 Gardner 2004/0167892 August 2004 Kirshenbaum et al. 2004/0267774 December 2004 Lin et al. 2005/0210008 September 2005 Tran et al. 2006/0218481 September 2006 Adams Jr. et al. |
| Primary Examiner: | Vo, Huyen X. |
| Attorney, Agent or Firm: | Yee & Associates, P.C. Yamonaco, Lisa M. |
| رقم الانضمام: | edspgr.07788099 |
| قاعدة البيانات: | USPTO Patent Grants |
| الوصف غير متاح. |