Usability evaluation of Camelis 2

For more information and download Camelis 2, go here!

Tutorial

The tutorial is in french.

Files

The file BenjaminFranklin.ctx is a context file about the genealogy of Benjamin Franklin and used for the tutorial.
The file GeorgeWashington.ctx is a context file about the genealogy of George Washington and used for the usability test.
This dataset can be completed by applying the inference rules from the file rules.ctx (to be imported after the genealogy file).

To do the test, download Camelis 2. Import the file GeorgeWashington.ctx and the rules.ctx. Then go here to begin the test!

Questions

Table 1 shows the questions and the minimum number of navigation steps necessary to answer them.

Tableau 1: Questions for the test and answer, and click's number to answer.
Category   Question Query LISQL Answer #navig. links
Visualisation 1 How many persons are there? a thing 79 0
Visualisation 2 How many men are there? a thing 42 0
Visualisation 3 How many persons have a birth's place in the base? a thing 23 0
Selection 4 How many women are named Mary? a woman and firstname : Mary 4 4
Selection 5 Who was born at Stone Edge? a thing and birth : place : 'Stone Edge' Mary Towneley [I19] and Margaret Hartley [I43] 4
Selection 6 Which man was born in 1659? a man and birth : year : 1659 Lawrence Washington [I45] 5
Selection 7 Who is married with Edward Dymoke [I33]? a thing and married with 'Edward /Dymoke/ [I33]' Anne Talboys [I35] 3
Path 9 Which man has his father married with Alice Cooke [I30]? a man and father : married with 'Alice /Cooke/ [I30]' Robert Reade [I25] 5
Path 11 Which man is married with a woman born in 1708? a man and married with (a woman and birth : year : 1708) Augustine Washington [I12] 7
Disjunction 8 Which women have for mother Jane Butler [I67] or Mary Ball [I23]? a woman and mother : ('Jane /Butler/ [I67]' or 'Mary /Ball/ [I23]') Jane Washington [I03], Betty Washington [I04] and Mildred Washington [I08] 6
Disjunction 12 Which men are married with a woman whose birth's place is Cuckfields or Stone Edge? a man and married with (a woman and birth : place : (Cuckfields or 'Stone Edge')) Augustine Warner [I18], Lawrence Towneley [I42] and Lawrence Washington [I54] 9
Negation 10 How many men were born in the 1600 or 1700 years, and not in Norfolk? a man and birth : (year : (16-- or 17--) and place : not in Norfolk) 9 12
Negation 13 How many women have a mother whose death's place is not Warner Hall? a woman and mother : death : place : not 'Warner Hall' 16 (not death:place:WH) or 7 (death:not place:WH) or 4 (death:place:not WH) 7
Inverse 14 Who was born in the same place as Robert Washington [I64]? a thing and birth : place : place of birth of 'Robert /Washington/ [I64]' Lawrence Washington [I50] and Lawrence Washington [I54] 6
Inverse 15 Who died during the year when Augustine Warner [I18] was born? a thing and death : year : year of birth of 'Augustine /Warner/ [I18]' John Twigden [I57] 6
Cycle 16 Which persons died in the same area where they were born ? a person and death : place : in ?X and birth : place : in ?X 11 9
Cycle 17 How many persons have the same firstname as one of their parent? a person and firstname : ?X and parent : firstname : ?X 12 8
Cycle 18 Which persons were born the same year as their spouse? a person and birth : year : ?X and married with birth : year : ?X 6 10



Results

For the results, questions were grouped by category. Table 2 shows the gross results. Participants are represented by letters and questions by shapes. For each couple participant-question, we measured if the query and the answer were correct. The $ \bullet$ represents a correct query and answer and the $ \circ$ represents a correct query.


Tableau 2: Number of correct answers and queries according to subject and questions. The $ \bullet$ represent a correct answer and query and the $ \circ$ represent a correct query.
  Category A B C D E F G H I J K L M N O P Q R S T Total ($ \bullet$ +$ \circ$ )
1 Visualisation $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ 20
2 Visualisation $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ 20
3 Visualisation $ \bullet$   $ \bullet$ $ \bullet$ $ \circ$   $ \bullet$ $ \bullet$ $ \bullet$   $ \bullet$ $ \bullet$     $ \circ$ $ \bullet$     $ \bullet$ $ \circ$ 13
4 Selection $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \circ$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ 20
5 Selection $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ 20
6 Selection $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ 20
7 Selection $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ 20
9 Path $ \bullet$ $ \circ$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$       $ \bullet$ $ \bullet$     $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ 15
11 Path $ \bullet$ $ \circ$   $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ 19
8 Disjunction $ \bullet$ $ \circ$   $ \bullet$ $ \bullet$   $ \bullet$ $ \bullet$     $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ 16
12 Disjunction $ \bullet$ $ \circ$ $ \circ$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \circ$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ 20
10 Negation     $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$   $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$   $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$   $ \bullet$ 15
13 Negation $ \bullet$ $ \circ$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$       $ \bullet$   $ \bullet$ $ \bullet$   $ \bullet$   $ \bullet$ $ \bullet$ 14
14 Inverse $ \bullet$     $ \bullet$       $ \bullet$   $ \bullet$ $ \bullet$   $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ 13
15 Inverse     $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$     $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ 16
16 Cycle $ \bullet$ $ \bullet$   $ \circ$   $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$     $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$   $ \bullet$ $ \bullet$   14
17 Cycle   $ \bullet$     $ \bullet$   $ \circ$ $ \bullet$ $ \bullet$   $ \bullet$   $ \bullet$ $ \bullet$     $ \bullet$ $ \bullet$ $ \bullet$ $ \bullet$ 12
18 Cycle       $ \bullet$ $ \bullet$ $ \circ$ $ \bullet$ $ \bullet$         $ \bullet$ $ \bullet$ $ \bullet$     $ \bullet$ $ \bullet$ $ \bullet$ 11
  Total ($ \bullet$ +$ \circ$ ) 14 13 12 17 16 14 17 17 11 11 15 14 15 15 17 15 15 16 17 17 298/360