Unifying the Search Engine and NoSQL DBMS with a Universal Index

A session at Strange Loop 2010

Unifying the Search Engine and NoSQL DBMS with a Universal Index In contrast to single-function architectures, MarkLogic Server takes an unusual approach to collapsing the usual hierarchies of types of servers that make up a complete application, combining Search, a NoSQL DBMS, and an application server in a single kernel. The computational foundation for this hybrid is the Universal Index.

In this talk, we'll begin with the familiar text indexing data structures and algorithms that underlie search engine technologies. We'll extend that index to cover document structure and semantics, add scalar range indexing in one and two dimensions (including geospatial application), and then incorporate "reverse" indexing of queries. We will demonstrate a novel type of "matchmaking" query whose evaluation is based on a composition of forward and reverse index evaluation, in a true "strange loop" path through abstraction levels. Finally, we'll explore the means by which all of this indexing may efficiently run concurrently with querying, using MultiVersion Concurrency Control and Log-Structured Merge Trees, providing ACID transactions together with lock-free query evaluation, built-in sharding, terabyte-per-server scale-out, replication, and query distribution.

We will conclude with examples of production applications built on this architecture for geospatial service discovery at warriorgateway.org, social networking at bx.businessweek.com, and knowledge management on a US Air Force application.

About the speaker

This person is speaking at this event.
Chris Biow

Coverage of this session

Sign in to add slides, notes or videos to this session

Tell your friends!

Short URL

lanyrd.com/sccbp

Official event site

strangeloop2010.com

View the schedule

Share

Topics

See something wrong?

Report an issue with this session