So far, there has been no language in the world that can be used to query multi-modal data. For some modal data such as graphs, there is even no recognized and standardized query language. Structured relational data possess a core status in data mining. However, with the rise of the Internet and the application of knowledge graphs in various products, the value of unstructured data is also growing rapidly. How to optimize the query of multi-modal data and the value of fully integrating and utilizing such data are future development trends.

Based on cross-modal entity linkage techniques, we have developed a lightweight cross-modal query system. By extending traditional SQL to construct a unified query language, we can perform the unified query of relational data and graphs to balance multi-modal data’s expressive power and complexity. Our goal is to enable RDBMS with relational and graph query capabilities, rather than building an independent and complete graph computation system through SQL. Therefore, our system will maintain the accessibility and composability of SQL in relational and graph queries, and will not require users to learn a new query language, thus realizing lightweight and easy-to-use cross-modal queries.