The Chandler Query System

Overview

The Chandler Repository provides a mechanism for performing queries over the contents of the repository. These queries are declarative: you specify a set of conditions that you want Items to satisfy, and the query processor takes care of retrieving the relevant items. Queries are specified using strings, and the syntax of these strings is inspired by XQuery.

An unusual feature of the query system is the ability to have a query subscribe to changes in the repository. When items are changed, the repository is able to notify the query of that change. The query processor determines whether that change would cause the item to become part of the result of an existing query, or whether the change would cause the item to no longer be part of a query result set. This allows queries to keep their result set up to date automatically.

If you haven't already read the document, "The Busy Developer's Guide to the Chandler repository", you should do so before continuing. Throughout this document we'll use the following terms:

query: A general term denoting a set of boolean conditions which in turn specify a set of Items. Also a specific Python class that provides a programmer with an API for manipulating a query.
query string: A Python string that contains a textual representation of the query
query result set: A set of Chandler Items that satisfy the boolean conditions for a query

Python API

The first thing that you'll need to know when working with queries is how to create them. If you are working within the Chandler Parcel system, then you will be working with queries via parcel.xml when you deal with ItemCollections. When working with queries via parcel.xml, you will only need to know about the string syntax for queries, which is described in the next section. Here's an example of parcel.xml usage, the ItemCollection for Chandler's task list:

  <contentModel:ItemCollection itsName="taskItemCollection">
    <displayName>TaskList</displayName>
    <_rule value="for i in '//parcels/osaf/contentmodel/tasks/TaskMixin' where True"/>
  </contentModel:ItemCollection>

You just supply a query string as the value of the _rule attribute.

To use a query from Python, your code will look something like this

import repository.query.Query as Query

p = rep.findPath('//Queries')
k = rep.findPath('//Schema/Core/Query')
q = Query.Query("for i in '//Schema/Core/Kind' where True", p, k)

for i in q:
    print i

Queries are Chandler Items, which means that they are stored in the repository themselves. The main query class is in repository.query.Query. To instantiate this class, you supply a query string, a path to the query's parent in the repository, and the Kind for a Query (these last two arguments are for the Item constructor). The query in the example specifies all items of type //Schema/Core/Kind.

Once you have an instance of Query, you can ask for the result set of the query. You have two choices for how to do this. You can ask for the 'resultSet' attribute of the Query item, which will return a reference collection containing the items in the result set. You can also call the __iter__ method on the Query Item by iterating over the Item (as shown in the example). This gives you a Python generator for the query results, which can be more efficient thatn asking for the resultSet attribute.

You can update the value of the query string by changing the value of the attribute queryString:

q.queryString = "for i in '//parcels/osaf/contentmodel/contacts/Contact' where True"

The query string may be the empty string, "". In this case, the result set of the query is empty.

Parameters allow you to use values from your Python code in a query. Parameter names are strings that begin with "$". To use parameters, you should set the query's args attribute to a dict containing the arguments. This code shows how to pass reference collection as a parameter to a query. The parameter name is $name. The value of $name is a tuple containing the UUID of the Chandler item that has a reference collection attribute and the name of the reference collection attribute (a string).

q.args = {}
q.args["$name"] = (item.itsUUID, attributeName)

This code shows how to pass an ordinary value -- this allows you to make comparisons to data in your Python code (indicated by the variable name in this case). The name of the parameter is $myname and it's value is a list containing the value of the variable name from your Python code.

q.args["$myname1"] = [ name ]

You can view the entire API for the Query class online.

Query String Syntax

There are several kinds of queries. Simple queries are queries over sets of items. You can write a simple query using the for statement. Compound queries are queries composed from other queries. Keywords and required tokens in a query are shown in bold. The portions of the query that you supply are written in italics

for queries

The most basic query is a for query. The syntax of a for query is:

for var in|inevery set where boolean-condition

The result set of a for query is the set of items in set which satisfy the boolean-condition Here's what you need to supply for the various portions of a for query:

var is the iteration variable for the query. For now, you must use 'i'. This constraint will be removed in the future.

If your for query is iterating over Kinds, and you want to include items of a Kind's subkind, you should use the keyword inevery instead of in.

set specifies a set of Items to apply the boolean-condition to. This allows you to issue a query over a subset of the repository. At the moment you can supply one of three possibilities for set:

repository paths for Kind names

The result of the query below is the set of all Contact items whose contactName child item's firstName attribute contains the letter 'a'. It shows the use of a Kind name as the source. In the where clause, we see the use of the iteration variable i, as well as the names of attributes (contactName, firstName). This example also shows the use of the contains function.

q.queryString = """for i in '//parcels/osaf/contentmodel/contacts/Contact' 
    where contains(i.contactName.firstName,'a')"""

parameters

This query illustrates the use of a parameter, $1 as the source set.

q.queryString = """for i in $1 where contains(i.itsName,"arc")"""

ftcontains(lucene-query , attr-name1 , ... , attr-namen)

lucene-query

attr-name1

attr-namen

This query demonstrates the use of a full text query for the source set. The lucene query is "femme AND homme" which returns all items containing the text "femme" and the text "homme". Since we've provided an attribute argument, synopsis, to ftcontains, the source set will be only those items whose synopsis attribute contains the text "femme" and the text "homme". The query's where clause shows the use of the len function to further limit the items that will be in the query result.

q.queryString = """for i in ftcontains('femme AND homme','synopsis')
    where len(i.title) < 10"""

The boolean-condition is an expression which can refer to the iteration variable and parameter values.

Here are the elements that you can use in the boolean condition (A BNF grammar for query strings appears at the end of this document).

The iteration variable for the query

At the moment, just the variable i. In the future more variable names will be allowed.

The names of Chandler item attributes

You may use the attributes of an Item. For example, most Items have an itsName attribute, so

i.itsName

will give the value of itsName for the current value of i. You can also call methods on Items (since method names are attributes)

Literal values

numeric literals: You can use any integer
string literals: String literals must be enclosed in either single (') or double (") quotes.
boolean literals: The Python boolean literals True and False

Parameters

This query shows that you can use parameters (like $0) in the where clause as well, allowing you to use run time values from your Python program inside a query.

q.queryString="""for i in "//Schema/Core/Kind" 
    where contains(i.itsName,$0)"""

Unary (prefix) operators

+ expr: Make numeric expression expr positive
- expr: Make numeric expression expr negative
not expr: Negate boolean expression expr

Boolean operators

expr1 and expr2: Perform the logical "AND" of expr1 and expr2; Perform the logical "OR" of expr1 and expr2; Negate expr

Relational operators

expr1 >= expr2: Return true if the numeric/date expression expr1 is greater than or equal to the numeric expression/date expr2
expr1 <= expr2: Return true if the numeric/date expression expr1 is less than or equal to the numeric/date expression expr2
expr1 > expr2: Return true if the numeric/date expression expr1 is greater than the numeric/date expression expr2
expr1 < expr2: Return true if the numeric/date expression expr1 is less than the numeric/date expression expr2
expr1 == expr2: Return true if expr1 and expr2 are equal according to the equality rules for their Kinds
expr1 != expr2: Return true if expr1 and expr2 are not equal according to the equality rules for their Kinds

Arithmetic operators

expr1 + expr2: Add the numeric expressions expr1 and expr2
expr1 - expr2: Subract the numeric expression expr2 from the numeric expression expr1
expr1 * expr2: Multiply the numeric expression expr1 by the numeric expression expr2
expr1 / expr2: Divide the numeric expression expr1 by the numeric expression expr2
expr1 div expr2: Produce the result of integer dividing the numeric expression expr1 by the numeric expression expr2
expr1 mod expr2: Produce the remainder of dividing the numeric expression expr1 by the numeric expression expr2

Dates

You can supply dates and times in eGenix mxDateTime ISO format like this:

date(ISO-date-string)

Construct a date instance that represents ISO-date-string

This example shows how to use dates in a query. Note the use of the date constructor to create a date literal which is then compared to the startTime attribute of i (i will be a CalendarEvent)

q.queryString="""for i in '//parcels/osaf/contentmodel/calendar/CalendarEvent' 
    where i.startTime > date('2004-09-31 12:34:56')"""

Functions

At the moment there are only three functions that you can call from queries. We will be expanding this set of functions as the system develops.

len(expr): Return the length of expr. expr can be a string or a list Kind
contains(string, substring): Return true if string contains substring
hasAttribute(string): A method on Chandler Items that returns True if the Item has an attribute named string

Union Queries

union(query1,query2, ... , queryn): Compute the union of query1..queryn and return that as the result. Any item that appears in the result set of any of the queries will appear in the result set of the union.

This query is composed of three for queries that show the same pattern. They all use a Kind path as the source set, and use True as the where clause, to indicate all items of a particular kind. The union operator simply creates the union of the three for queries.

q.queryString="""union(for i in "//parcels/osaf/contentmodel/calendar/CalendarEvent" where True,
      for i in "//parcels/osaf/contentmodel/Note" where True, 
      for i in "//parcels/osaf/contentmodel/contacts/Contact" where True)"""

Intersection Queries

intersect(query1,query2): Compute the intersection of query1 and query2 and return that as the result. Items in the result set of the intersection must appear in the result set of both query1 and query2.

This query computes the intersection of those Kind items whose name contains 'o' and those Kind items whose name contains 't'

q.queryString="""intersect(for i in '//Schema/Core/Kind' where contains(i.itsName,'o'),
          for i in '//Schema/Core/Kind' where contains(i.itsName,'t'))"""

Difference Queries

difference(query1,query2): Compute the difference of query1 and query2 and return that as the result. Items in the result set of the difference consist of any Item that is in the result set of query1 which is not in the result set of query2. You can think of this as starting with the result set of query1 and removing any Item which appears in the result set of query2.