H2PTM (2007) Cheniti-Belcadhi

Ontology based assessment personalization

---

 
 

Titre Ontology based assessment personalization Auteurs Lilia Cheniti-Belcadhi(i), Rafik Braham(i) et Nicola Henze(ii) Affiliations (i)PRINCE Research Group, ISITC, University of Sousse 4011, HammamSousse ISITC, Tunisia lilia.belcadhi@infcom.rnu.tn (ii)IVS-Semantic Web Group, University of Hannover, Appelstraße 4, 30167 Hannover, Germany henze@l3s.de Dans actes du colloque H2PTM 2007 Hammamet publié dans H²PTM07 : Collaborer, échanger, inventer

Résumé

Le travail présenté dans cet article concerne le développement d´un système d´évaluation personnalisée sur le Web sémantique. Nous présentons d´abord les principaux problèmes en rapport avec l´évaluation sur le Web et décrivons ensuite les principales fonctionnalités de notre système. Nous explorons par la suite les moyens d´utiliser une ontologie de domaine. Les ressources sont annotées avec des langages du Web sémantique et avec les standards LOM et IMS/QTI. Un formalisme en logique du premier ordre a été défini pour l´évaluation personnalisée puis implémenté en utilisant TRIPLE, un langage pour raisonnement sur le Web sémantique. Nous présentons finalement l´architecture du système, composée de services Web. La communication entre ces services est réalisée avec les technologies du Web sémantique. Ce système a été intégré dans le système Personal Reader.

Mots-clés 

Web sémantique, Ontologie, Service Web, évaluation basée sur le Web, personnalisation.

English description

Abstract 

In this work we are interested in building a personalized access to learning and assessment materials in educational environments based on domain ontology. The work reported in this paper relates to the development of a web-based Semantic web assessment personalization system (SWAP). We first expose the main issues related to web based assessment and describe then the features of our SWAP system. Our approach for personalization of assessment is based on a domain ontology, which enables the access to resources based on the concepts contained in these resources. In this paper we first investigate the way of building our domain ontology. Resources used in this system are annotated according to the e-learning standards LOM and IMS QTI and using the domain ontology. In this paper we also show how resources have been annotated and present briefly a logic-based approach for description of assessment personalization using First Order Logic and for reasoning over Semantic web using TRIPLE. The system is implemented using web services and has been integrated as part of the Personal Reader Framework. Assessment personalization is then performed through services communication using Semantic Web technologies.

Keywords

Semantic Web, Ontology, Web Service, Web based assessment, Personalization.

 

Introduction

In the context of education, a "test" is usually a series of questions. Typically, the sequence of questions presented in a test to a learner is independent from his progress in the learning process. Thus, the need to maintain and provide individualized teaching and learning support for students is a growing concern for higher education. This work is concerned with the design and development of a personalized Web-based assessment system that takes into account the individual performances of each learner in knowledge acquisition. Our objective is to provide the learners with personalized tests that are appropriate to their needs and prior knowledge to guarantee an efficient learning process. Current research in the Semantic Web (Berners-Lee, 2001) could be used to achieve this objective thanks to the notion of ontology, which plays a role in facilitating the sharing of meaning and semantics. The Semantic Web architecture is powerful enough to satisfy the computer assisted assessment requirement. Assessment material is semantically annotated and for a new assessment demand it may be easily combined in a new test. According to his/her level of knowledge, the learner can be easily presented with appropriate tests. The process is based on semantic querying and navigation through assessment materials, enabled by the ontological background. Languages such as RDF (RDF, 2007), RDFS (RDF 2, 2007) can be used for metadata representation and reasoning. In this work we have developed a system, which embodies personalized Web services that deliver assessment and generate learning resources proportionally to domain ontology, learner's requirements and interaction with the learning environment. The remainder of this paper is structured as follows: In section 2 we present the main issues related to assessment systems implementation. In section 3 we shall illustrate the features of the assessment system that we develop and present in section 4 its implementation. We, finally give in section 5, our concluding notes and a statement of inferences for future work.

Issues Related to assessment systems implementation

Further to the review of an important number of assessment systems, we have identified these main issues related to their implementation.

• Lack of interoperability

Of the assessment systems existing currently, few mention any technical aspects of how their systems work. Furthermore, none of these systems examined describes with precision what format is used to store question content. Generally speaking, they either use a proprietary format or do not demonstrate any ability to share question content with other systems.

• Application domain of assessment systems

The majority of assessment tools cannot be applied to general situations. They are typically intended for a particular setting in which assessment commonly takes place. In their limited application areas, most of the tools have proven useful to some extent in educational institutions. There are some interesting tools available at various stages of development, with varying functionality and of varying quality. Some products are strong in a few areas but none can be considered to meet every educational need; indeed, there are specific and fundamental shortcomings of the currently available tools and the approaches they take.

• Component based architecture

The architecture of many Web based assessment systems incorporates relatively many components which are very dependent. These systems are sensitive to change. A change in the output or in the implementation of one of the components will often cause the whole system to break. We need to replace the current models of assessment architecture design with a more flexible architecture. An encapsulation of the assessment functionality in web service based architecture helps to reduce its complexity and facilitates its interoperability and extensibility. The idea of combining Web services with assessment tools has been addressed only in few systems.

Features of Semantic Web Assessment Personalization System (SWAP System)

In this work we propose a Semantic Web Assessment Personalization System (SWAP system) that permits the assessment personalization using semantic web technologies. The proposed architecture is domain-independent and can be used for a variety of courses. We combine Semantic Web technologies, learning standards and Web services, so that we can enhance interoperability of resources. The system fosters the implementation of an ontology based adaptive assessment scenario, thereby enabling the adaptive selection and generation of questions and online courses to acquire a good estimation of learner's level of knowledge. It also reduces the number of questions required to estimate the learner's knowledge standard, resulting in a less tedious assessment process

Domain ontology

Our first step towards the solution of providing personalized assessment on the semantic web is to rethink the way of knowledge modeling. Our objective is to have an independent knowledge model, one that can be used to generate personalization of learning in a very flexible way. This model will be used for modeling learning dependencies, describing the actual knowledge state of a particular user and for making inferences on the base of user observations. The concepts in the knowledge model should define a controlled vocabulary for describing the knowledge of the application domain and are used for making the metadata annotation of used learning and assessment resources (Henze, 2001) and can be presented in an ontology. In our work, we have developed an ontology for C++ and basic object oriented concepts. This ontology has been built by using the language grammar as well as the main concepts related to object oriented programming that need to be represented in our knowledge base. The ontology for C++, that we established, is used for the resource description of the object oriented programming course on which we propose to test our SWAP system. We first have set up the classes of concepts in this programming language based on its grammar. We then searched for subclass relations between these classes, which helped us define an “is_a” hierarchy of classes. In general, A is a subclass of B if every instance of A is also an instance of B. Specific domain information is usually described by concepts and their mutual relationships in a domain. In our figure we show just a fragment of a domain knowledge base covering some C++ programming concepts, and including the “is_a” (“subConceptOf”) relationship between these concepts. Figure 1 depicts the Programming types concept with its subconcepts: object_oriented_programming, structured_programming modular_programming procedural_programming and unstructured_programming. Some of these concepts have also their own sub concepts.

Figure 1. Part of the C++ ontology

By generating the ontology as a shared conceptualization we had to find a way to support different views on the categorization of concepts which we want to illustrate by an example. “constructor” is a concept related to the method in a class responsible on the construction of instances and should therefore be part of the member specification of a class. It is also considered to be one of the object oriented concepts (“oo_concept”). Therefore “constructor” can be expected to occur as subconcept of both concepts “member_specification” and “oo_concept”. To solve this problem we allow multiple occurrences of concepts in the ontology. This approach would facilitate the implementation of personalization strategies. Moreover, it is also worth to note that we cannot expect that our ontology will be fixed once and forever (Henze, 2001). New concepts are likely to appear and have to be then added in the proposed ontology.

Resources description

The SWAP system makes use of learning standards LOM (LTSC, 2007)and Dublin Core (Dublin Core, 2007) to support interoperability. It is also designed to be IMS/QTI (IMS, 2007) compliant in order to enhance the exchange of assessment resources. The annotation of the learning content according to e-learning standards facilitates its reuse and personalization to various learners. The concept of learning object introduced in LOM facilitates the selection, composition and delivery of personalized resources in open and distributed learning environments. In particular we have used the “Educational Objective” and the “Prerequisite” attributes from the LOM schema to annotate respectively the objective and the prerequisite concepts of the learning resource with reference to the ontology. The QTI standard used for assessment resources provides a standard format in which assessment information can be interoperable and reusable among different systems. The key data structures proposed in QTI are: Item (a question/ response bloc). Section (a collection of items or other sections), Assessment (a collection of sections). The established C++ ontology is used then for the annotations of learning and assessment resources. As an example, we show in the following part of an assessment resource, which has been annotated according to IMS QTI. We have used the field "objective" to specify the main concepts targeted by the assessment resource with reference to the domain ontology. This annotation would facilitate the selection and presentation of the assessment resource related to the current progress and learning course selection of the learner. The assessment resource described in the following example is an “item”. It is a “MultipleChoice” question and its objective is to check the knowledge of the learner on the concept “inheritance”.

Example; Assessment resource description

Description of assessment personalization

There is a large variety of systems that offer assessment, and therefore assistance to the learner. Nonetheless, the assessment functionality in the majority of these systems is described with reference to a specific environment, which means that the functionality is only described in terms of the system that supports it. So long as the assessment functionality is dependent upon the system's characteristics, it is difficult to give a precise comparison of assessment systems. There is thus far no common language that enables the description and analysis of the assessment functionality. Thus, a formal description of personalized assessment which allows a system-independent characterization of the assessment functionality is needed. This formalization is also helpful in constructing a model of the user, which is a representation of those characteristics on the basis of which assessment personalization takes place. We provided a formalism of the personalized assessment with First order Logic (Cheniti-Belcadhi, 2005). This formalism has been developed based on the formalism of Adaptive educational hypermedia systems (Henze, 2003). The choice of predicate logic to formalize and describe the personalized assessment functionality is driven by the observation that this type of logic provides a precise conception of the ways in which the first-order meaning representations relate to particular information. In the following we give two examples of FOL rules. The first rule embodies all items required for the assessment for a learning object a posterior, and entails the criteria used to select items to Post-tests a given user.

The second rule entails that a learning object is recommended for post-assessment if at least one of its objective concepts from the knowledge base has not been learned by the user.

Given semantically annotated data it is then possible to perform reasoning. Assessment personalization can be processed through reasoning rules which are able to enquire about resources and metadata, and reason over distributed data and metadata descriptions. In order to express this reasoning we need a query language for the Semantic Web which is able to access various resources. In our work we have translated the first-order logic rules in the query language TRIPLE (Sintek, 2002), making it possible to implement them using Semantic Web technologies. These rules can be employed to reason over distributed information sources (ontology, user profile information, resource descriptions). The communication between reasoning rules and the open information environment will take place by exchanging RDF annotations. In the following we give two rules in TRIIPLE representing respectively the translation of the FOL rules above.

We consider that our approach facilitates the personalization of assessment in open document spaces. We have indeed considered the personalization of the assessment functionality as a query in open environments and determined the characteristics of such functionality in order to define useful queries.

SWAP system implementation

The architecture we design is composed of services available for the user, which deliver assessment and generate learning resources with respect to domain ontology, learners' requirements and interaction with the learning environment, without the need for centralized control. The SWAP system embodies mainly the following services: visualization service, connector service, user profile service and assessment service (figure 2). It has been implemented as part of the Personal Reader Framework (Personal Reader, 2007), which is a framework for designing, implementing, and maintaining personalized Web Content Readers. The web services implement specific interfaces in Web Service Description Language (WSDL) (WSDL, 2007). These interfaces declare methods for service invocation and for the provision of meta-information on these services. The interaction between the services is presented in Figure 2.

Figure 2. Web services interaction in the SWAP system

The visualization service is responsible for the display of the resources requested by the learner, i.e. either the learning resources or the assessment resources. It is responsible for user administration, the reception of user queries and inputs and the display of results. Having processed the input data, its main task is to then display to the user those output data received from the "connector service". This latter is a key component of the SWAP architecture and is the mediator between the various services. Communications among all services go through this service. Its main task is to convert the different formats of metadata descriptions used by the assessment service and provide a generic interface for the visualization service. The user profile service is responsible for recording learner interactions (usage monitoring) and updating user profile (user performance management). The assessment service provides a personalized assessment to the learner based on the information provided in his profile. It has two main tasks the construction of tests and the evaluation of learner answers. A test is typically a well-organized sequence of questions selected according to the user profile. Two types of tests can be generated: pre-tests and post-tests. The questions are deployed in a test to evaluate the level of knowledge acquisition by the learner. The construction of Pre-tests and Post-tests is a dynamic process that depends on the learner's current level of knowledge. Furthermore, the navigational behavior of the learner through the content is considered. The assessment procedure takes into account the learning resources that the learner has visited. This type of information is saved in the learner's profile. In the Pre-tests, questions relating to the prerequisite concepts of the selected learning resources are posed. In Post-tests, questions that are relevant to each objective of the current selected resource are presented to the learner. Communication between the different Web services is carried out through RDF documents, with reference to the domain ontology. This mechanism presents the immediate advantage that all components can be independently invoked and developed. The SWAP system allows every learner who uses it to have a personalized interface, which they can access by logging in with their password. Figure 3 is a screen shot showing how learning resources are presented to the learner. In the right part the specific learning resource is displayed. The left part visualizes the results received via the Connector service. Learning resources for which a pre-test can be constructed based on the information in the learner's profile will be marked with a yellow ball. In case a post-test can be created for the selected learning resource a red ball will be displayed.

Figure 3. Screenshot of the SWAP system

We carried out an experiment of the SWAP system with a group of students from our University.The results that emerged from the first experiment are encouraging. A detailed description of this experiment, the hypotheses established and the results discovered can be found in (Cheniti-Belcadhi, 2006)

Conclusion and future work

The system reported in this work offers a particularly promising outlook in the field of e-learning, and more specifically in the area of Semantic Web. The methodology permits flexibility in assessment personalization and can be applied to any domain given its ontology. The present work has only examined the use of objective questions for the provision of personalized assessment over the Semantic Web. Further research may explore the potential to enlarge the types of questions provided and integrate other types of questions such as open-ended enquiries. The architecture we designed was composed of Web services that handle all the necessary communication and reasoning for the assessment of a given learner. We suggest that further investigation of the algorithms and techniques for services compositions would be very useful to enlarge the scope of application of the system we developed.

References

[Berners-Lee, 2001] ↑ Berners-Lee T., Hendler J. et Lassila O., « The Semantic Web », Scientific American, 2001.

[Cheniti-Belcadhi, 2005] ↑ Cheniti-Belcadhi L., Braham R., Henze N. et Nejdl W., « A Generic Framework for Assessment in Adaptive Educational Hypermedia », IADIS International Journal on WWW/Internet, vol. 3, n° 1, 2005, p. 17-28.

[Dublin Core, 2007] ↑ Dublin Core, « Metadata Initiative »
En ligne : http://dublincore.org/

[Henze, 2003] ↑ Henze N. et Nejdl W., « Logically Characterizing Adaptive Educational Hypermedia Systems », Proceeding of the Adaptive Hypermedia Workshop, World wide Web Conference, Budapest, Hungary, 20-24 May 2003.

[Henze, 2001] ↑ Henze N., « Towards an open adaptive hypermedia », Proceeding of the ABIS Workshop (LLWA01), October 2001, Dortmund, Germany, 2001.

[IMS, 2007] ↑ IMS Question and Test Interoperability Specification, V1.2
En ligne : http://www.imsglobal.org/question/index.cfm.

[LTSC, 2007] ↑ Learning Technology Standards Committee, « Towards an open adaptive hypermedia », Learning Object Metadata
En ligne : http://ltsc.ieee.org

[Personal Reader, 2007] ↑ Personal Reader Framework, {{{texte}}}
En ligne : http://www.personal-reader.de

[RDF, 2007] ↑ Resource Description Framework, {{{texte}}}
En ligne : http://www.w3.org/RDF/

[RDF 2, 2007] ↑ Resource Description Framework, {{{texte}}}
En ligne : http://www.w3.org/TR/2000/CR-rdf-schema

[Sintek, 2002] ↑ Sintek M. et Decker S., « TRIPLE- An rdf query, inference, and transformation language for the semantic web », Proceeding of the 1st International Semantic Web Conference, Sardinia, Italy, 10-12 June 2002.

[WSDL, 2007] ↑ WSDL: Services Description Language, (version 2.0)
En ligne : http://www.w3.org/TR/2004/WD-wsdl20-20040803/