The painters of the Byzantine and post Byzantine artworks use specific rules and iconographic patterns for the creation of sacred figures. Based on these rules, the sacred figure depicted in the artwork is recognizable. In this work, we propose an automatic knowledge-based image analysis system used for Byzantine icons classification on the basis of the sacred figure recognition.

Figure 1. The Byzantine icon classification system’s architecture. The analysis module extracts information about the icon, and the knowledge representation and reasoning module uses this information to infer implicit knowledge that categorizes the icon.

Firstly, the system detects and analyzes the most important facial characteristics providing rich, still imprecise information about the Byzantine icon. Then, the information extracted is expressed in terms of an expressive terminology formalized using Description Logics (DLs) which form the basis of Semantic Web ontology languages.

Figure 2. Image analysis. First, the algorithm detects the sacred figure’s face region, eyes, and nose. Then, it extracts the hair, forehead, cheek, mustache, and beard parts together with the face’s base color layers. Further analysis of the extracted parts provides information about characteristic features. Finally, the algorithm produces a semantic interpretation for each of these features, together with formal assertions.

In order to effectively handle the imprecision involved, fuzzy extensions of DLs are used for the assertional part of the ontology. In this way, the extracted by image analysis information comprises the assertional component while the expressive terminology, formalizing the rules and the iconographic patterns, permits categorization of Byzantine artworks.

Figure 3. A reasoning example. The extracted information from image analysis constitutes the assertional component (ABox) of the knowledge base; the terminological component (TBox) is defined on the basis of Fourna’s specification. These components form the input to the fuzzy reasoning engine, which infers information about the icon.

We evaluated our system on a database, provided by the Mount Sinai Foundation in Greece, containing 2,000 digitized Byzantine icons dating back to the 13th century. The icons depict 50 different characters; according to Dionysios, each character has specific facial features that makes him or her distinguishable. Evaluation of the Byzantine-icon-analysis subsystem produced promising results. The subsystem’s mean response time was approximately 15 seconds on a typical PC. In the semantic-segmentation module, the face detection submodule reached 80 percent accuracy.

In most cases, the failure occurred in icons with a destroyed face area. If the submodule detected the face, it almost always detected the eyes and nose. The base-color-analysis submodule defined the color model used by the icon’s artist, which the feature extraction module later used. Using the nose’s height, which can be easily determined, and taking into account the manual’s rules, the face-component-detection submodule estimated the positions of the background and foreground pixels for every part of the face. These positions constituted the input to the graph-cut algorithm. This submodule achieved 96 percent accuracy.

The feature extraction module extracted features for every icon segment. Then, using the fuzzy partitions, the semantic-interpretation module interpreted each segment’s specific features and properties and the relationship of face parts. Thus, that module determined each feature’s degree of membership in a specific class, thereby creating an image description. To evaluate overall system performance, we used precision and recall.