SciELO - Scientific Electronic Library Online

 
vol.19 número1Balancing Energy and Performance in Dense Linear System Solvers for Hybrid ARM+GPU platformsPICTOAPRENDE: Design and evaluation of a tool that contributes to the personal autonomy of children and youth diagnosed with autism spectrum disorder in Ecuador índice de autoresíndice de materiabúsqueda de artículos
Home Pagelista alfabética de revistas  

Servicios Personalizados

Revista

Articulo

Links relacionados

Compartir


CLEI Electronic Journal

versión On-line ISSN 0717-5000

CLEIej vol.19 no.1 Montevideo abr. 2016

 

Assessing a Methodological Proposal for the Design, Creation and Evaluation of Learning Objects Oriented to Educators with Diverse Educational and Technological Competencies

Jorge J. Maldonado
University of Cuenca, Computer Science Department,
Cuenca, Ecuador, 010203,
jorge.maldonado@ucuenca.edu.ec
Jorge L. Bermeo
University of Cuenca, Computer Science Department,
Cuenca, Ecuador, 010203,
jorge.bermeo@ucuenca.edu.ec
Guillermo Pacheco
University of Cuenca, Language Institute,
Cuenca, Ecuador, 010203,
guillermo.pacheco@ucuenca.edu.ec

Abstract

This paper describes a methodological proposal for the design, creation and evaluation of Learning Objects (LOs). This study arises from the compilation and analysis of several LO design methodologies currently used in Ibero-America. This proposal, which has been named DICREVOA, defines five different phases: analysis, design (instructional and multimedia), implementation (LO and metadata), evaluation (from the perspective of both the producer and the consumer of the LO), and publishing. The methodology focuses not only on the teaching inexperienced, but also on those having a basic understanding of the technological and educational aspects related to LO design; therefore, the study emphasizes LO design activities centered around the Kolb cycle and the use of the ExeLearning tool in order to implement the LO core. Additionally, DICREVOA was used in a case study, which demonstrates how it provides a feasible mechanism for LO design and implementation within different contexts. Finally, DICREVOA, the case study to which it was applied, and the results obtained are presented.

Abstract in Spanish

El presente artículo describe una propuesta metodológica para el diseño, creación y evaluación de Objetos de Aprendizaje (OAs). Este estudio surge de la recopilación y análisis de varias metodologías de diseño de OAs, actualmente utilizadas en Iberoamérica. Esta propuesta, que se denomina DICREVOA, define cinco fases diferentes: análisis, diseño (instruccional y multimedial), implementación (OAs y metadatos), evaluación (desde la perspectiva tanto del productor como del consumidor de los OAs), y la publicación. La metodología se centra no sólo en los que carecen de conocimiento en el tema, sino también en aquellos que tienen un conocimiento básico de los aspectos tecnológicos y educativos, relacionados con el diseño de OAs; por lo tanto, este estudio hace hincapié en el diseño de OAs centrados en el ciclo de Kolb y en el uso de la herramienta ExeLearning, con el fin de implementar el núcleo del OA. Adicionalmente, DICREVOA fue implementado en un estudio de caso, que demostró como esta metodología proporciona un mecanismo viable para el diseño e implementación de OAs, en contextos diferentes. Finalmente, se presenta DICREVOA, el estudio de caso, y los resultados obtenidos durante la investigación.

Keywords: Learning Objects, Design Methodologies, Kolb Cycle, Digital Educational Materials
Keywords in Spanish: Objectos de Aprendizaje, Metodologías de Diseño, el Ciclo de Kolb, Materiales Digitales Educativos
Received: 2015-10-30 Revised: 2016-02-14 Accepted: 2016-03-01
DOI: http://dx.doi.org/10.19153/cleiej.19.1.3

1 Introduction

Digital Educational Materials (DEM) have made their way to the traditional classroom - audiovisual media, such as an electronic presentation or a video, have gradually replaced printed texts and become content mediators in the learning process. This has enriched educational processes with a variety of materials and resources available for students. Conducted research [1] concluded that computer-based instruction is a better motivator, and students who use computers learn more than those who learn using traditional materials. This would require educators to be prepared to configure new learning scenarios for students, and they would also need to expand their educational material collections with new, digital materials that are good enough as additional learning materials, beyond the classroom. To achieve this, educators make use of various strategies that are directly dependent on their knowledge (pedagogy, didactics, technology) [2] and whose purpose is obtaining specific learning results. However, the strategies that have been adopted by educators (in some cases, digital immigrants) to produce DEM in contexts where students are digital natives, have been insufficient [1]. Digital immigrants are the result of a digital migration process that involves moving towards a highly technological environment created by Information and Communications Technologies (ICTs) - while students (digital natives) have grown up with technology and are therefore innately prepared for the digital jargon and environment, this is not the case of their educators, the digital immigrants [1]. There is therefore a need for strategies and guidelines that help educators design and create DEMs.

The Learning Objects (LOs) paradigm offers a strategy to achieve this. A LO can be understood as ”an independent digital didactic unit designed to achieve a specific learning objective and to be reused in different Teaching and Learning Virtual Environments (TLVEs) as well as different learning contexts. It must also include metadata that allow locating and contextualizing it.´´ [3]. The creation of DEMs through the LO paradigm requires the application of several disciplines for their design, development, production and storage, such as: a) Instructional Design, b) Computer Science, c) Library Science, and d) Human-Computer Interaction. Instructional Design guides educators in the design of learning strategies that will help students achieve the learning objectives proposed. Computer Science provides mechanisms that will allow adapting and converting the contents to be taught into digital materials supported by the use of digital formats; it also provides communication channels with users. Library Science allows cataloging digital materials by using labels that include a description of the corresponding digital material to facilitate their storage, localization, and retrieval. Human-Computer Interaction is aimed at improving the interaction between LO users and LOs themselves, with the purpose of improving information exchange, strengthening the stimulus received by students to engage and motivate them.

However, the concept of LO is not a unified one [4, 5, 6, 7, 8, 9, 10, 11, 12] which has resulted in a wide variety of interpretations that propose a number of LO design methodologies [13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 2, 24, 25, 26, 27] that have mostly been conceived to cover specific educational material production needs within each university space, from various perspectives, in specific environments that are adapted to the needs of the case at hand, and which require educators with mid-to-advanced computational knowledge to focus on the use of a given software tool or, alternatively, requiring educators to master specific instructional methods to propose meaningful activities to facilitate learning.

In this paper, we will introduce and describe a methodological proposal for the design, creation and evaluation of LOs; this proposal is the result of a through compilation and analysis of LO design methodologies in Ibero-America. Our proposal, which we have called DICREVOA, has 5 phases - analysis, design, implementation, evaluation and publication. This methodological proposal is conceived to be used by educators with any level of educational or technological knowledge. Therefore, it focuses on the design of LO activities around Kolb Cycle (educational) and the use of the ExeLearning tool (technological) to implement the LO structures. Then we present a case study in which the methodology was assessed.

This article contributes to the research on Learning Objects and methodologies in two ways. First, it extends the literature on Learning Objects by providing a solid concept about what is and what isn´t a LO. Second, it provides a practical methodology intended to those teachers who are willing to drive the entire process related to the design, creation and evaluation of LO through educational and technological sides. In this way the evaluation is intended in two ways, one is under the eye of the teacher as producer of DEM and the other one under the eye of the students as a consumer.

The article is organized as follows: section 2 describes the state of the art in relation to methodologies and methodology proposals for the design of learning objects; section 3 discusses the phases of the DICREVOA proposal; section 4 present a case study in which the methodology was assessed by students and educators; section 5 discusses the results obtained; and section 6 presents the conclusions.

2 Related Work

In this section, thirteen methodologies and methodology proposals that are used for designing and creating LOs are analyzed. For the analysis, a tool that provides an analysis framework for selecting LO design methodologies, called MASMDOA [28], is used. This tool establishes three dimensions that allow identifying the most relevant aspects, both educational and technological, of each methodology being considered. The results obtained are the basis for developing the methodology proposal presented in this paper.

2.1 Analysis of Learning Objects Design Methodologies

MASMDOA allows analyzing methodologies, proposed methodologies, and heuristics used to implement LOs. In this frame, MASMDOA proposes certain criteria, obtained by systematically reviewing bibliography, on agreements on the conceptual definition of a LO by the most relevant authors in the last 17 years (1998-2015). These criteria are closely related to the characteristics a LO should have, but they also include technological and pedagogical aspects. Criteria are grouped under three dimensions: a technological dimension, an educational dimension, and a general dimension. This allows analyzing them from various perspectives, and knowing the orientation of each methodology without actually implementing it; for instance, knowing if it is aimed at dealing with aspects that are more on the technological side than the educational or vice versa, or if there is a balance between both aspects.

From the technological perspective, the criteria used are: 1) Locatable, 2) Guidelines/Techniques, 3) Reusable, 4) Publication, 5) Interoperable; from the educational perspective, the criteria used are: 6) Educational Design, 7) Assembly Possible, 8) Learning Objects Components; from the general perspective, the criteria used are: 9) Definition Included, 10) Methodology User, 11) Author Licenses. For more detail on each of the dimensions and their criteria, see [28]. Table 1 compares the technological and educational aspects covered by LO Design Methodologies.

The methodologies described in Table 1, have been conceived at different points in time and for specific contexts, and were aimed at meeting existing needs in the production of LOs. Some of these methodologies, such as MEDEOVA, MIDOA, ISDMELO, UBOA, and UAT, suggest that a multidisciplinary team would be required to adopt them (coordinator, programmers, content reviewers, content experts, etc.) which, for an educational institution, would be expensive given the number of professionals required. On the other hand, the Techno-Pedagogical and LOCOME methodologies are oriented to educators with knowledge of Computer Science, that is, educators with technical training in Computer Science, while the Standard Technologies, Pattern-Based, UPV, and ISDOA proposals are oriented to educators with basic Computer Science knowledge. Unlike the others, CROA and AODDEI propose their own instructional design model and are oriented to educators who do not have experience in the development of LOs; however, they do not explain how the LOs are assembled.

Figure 1 summarizes the number of criteria in each of the three perspectives defined in MASMDOA that are covered by each of the methodologies. Eight of these, namely CROA, ISDMELO, ISDOA, Techno-Pedagogical, LOCOME, Standard Technologies, AODDEI and Pattern-Based, cover most of the criteria linked to the technology dimension. However, only CROA, ISDMELO, UAT and UPV encompass all of the criteria considered for the educational dimension.


PIC
Fig. 1: LO Design Methodologies Compared

The Polytechnic University of Valencia [13] in its proposal for the creation of LOs, uses, in its content model to produce the Learning Module, something that is very similar to CISCO’s content model, which groups between 5 and 9 Reusable Information Objects (RIOs) to produce a Reusable Learning Object (RLO), using an introduction or overview, a summary and an evaluation. However, in order to create LOs using this proposal, educators must be familiar with, or at least have basic knowledge of, the computer tools required.

The Pattern-Based proposal [14] considers LO production issues from a strategic and cognitive view that consists in using predefined LO patterns. When building LOs, patterns are used to identify and select the processes to develop, such as learning dimensions or various types of cognitive activities; these, together with computer supplies (readings, images, audio, video) and the adaptation of instructions, will make up a LO.

The MEDEOVA methodology [15] was conceived with the purpose of providing guidelines for the design, development and publication of Virtual Learning Objects (VLOs) that can be used to support the teaching and learning process at the various courses offered by the Saint Thomas Aquinas University at Tunja, Colombia.

The MIDOA methodology proposal [16] presents a conceptual model for the creation of LOs that is based on the use of a Software Engineering development methodology and the creation of production rules under an instructional design. It proposes modeling the process based on the use of either of the following Software Engineering methodologies: 1) Evolutionary Prototyping, 2) Extreme Programming.

The ISDMELO methodology [17] is focused on the design and development of educational content and is based on the instructional design model. ISDMELO considers that, given their expertise, instructional designers are the ones responsible for assembling LOs. This methodology is strongly supported by learning theories and widely encompasses the LO-supported instruction design process. ISDMELO recommends analyzing student learning styles; however, it does not relate LO design aspects to the analysis of these styles, meaning that there is no link between the profile of the students from the standpoint of their learning style and the design of the LOs from a pedagogical point of view. It is oriented to multidisciplinary teams.

The LOCOME methodology [18] comes from the need for a robust methodology for building LOs. Design is based on necessary standards and mechanisms that can guide the process of building software objects using the RUP (Rational Unified Process) methodology. This methodology is fully iterative, which means that, when inconsistencies, failures or weaknesses are found in any of the lifecycle phases, several iterations can be run on that phase, or even go back to previous phases.

The UBoA methodology [19] establishes its theoretical and pedagogical foundation in accordance to the virtual pedagogical model of the University of Boyaca. The pedagogical model is used to build the basis for defining requirement levels to be taken into account when building LOs. These include conceptualization, design, production and distribution phases designed to answer the questions What does it teach?, How does it teach it?, and What is assessed, and how?. The UBOA methodology is organized in five phases, each with their respective activities and results specification.

The Techno-Pedagogical proposal [20] combines knowledge from the areas of Education, Software Engineering, and Human-Computer Interaction. Human-Computer Interaction deals with all aspects pertaining to interface development (it should motivate students), Education is used to describe the teaching and learning process to be carried out, emphasizing learning scenarios, and Software Engineering dictates how to implement the learning process on the computer.

The ISDOA methodology proposal [21] is a problem-based LO design proposal with well-defined characteristics in relation to the design of the problem, design of the graphic interface, software architecture pattern, self-evaluation process, and metadata structure that must be in agreement with the needs and learning styles of the target audience and deployment platforms defined in non-functional requirements. The lifecycle for the development of LOs proposed by ISDOA by means of Software Engineering is supported on two fundamental pillars - test planning and quality assessment. This proposal considers that both pillars must be run in parallel for all LO creation phases.

The CROA methodology [22] resorts to a number of questions in each of its phases; these questions guide both the analysis and the design of LOs. Upon completion of each phase, certain output deliverables are obtained; these deliverables become the documentation that supports the LO. This methodology is oriented to any type of educators.

The UAT methodology [23] establishes a methodological link between pedagogy and its theories (constructivist, cognitive, interaction and communication) and the technological development of the LOs. It takes aspects from Software Engineering, Dick and Carey’s instructional design. The LOs that can be obtained are of one of two types - oriented to the educator or oriented to multidisciplinary teams.

The LO design proposal that is based on Standard Technologies [2] proposes the design and implementation of LOs under parameters that facilitate interoperability, reusability and maintenance, taking into account the characteristics of learners and using standard architectures that support e-learning, such as Learning Technology Systems Architecture (LTSA), IMS Content Packaging specification and LO metadata (IMS-METADATA).

The AODDEI methodology [24] is based on L’Allier’s definition [4] and it was proposed by the Autonomous University of Aguascalientes. This methodology is aimed at solving some of the issues that face educators with no experience in LO development.

The AODDIE methodology is based on the ADDIE instructional model [29] , which guides the pedagogical aspects of the methodology and provides educators with a series of templates that they can use to collect information.


TABLE 1: Comparison of Related Design Methodologies
--------------------------------------------------------------------------------------------------------------------------------                                 Technological and educational aspects covered by LO design proposals and methodologies       Methodology                    -----------------------------------------------------------------------------------------------------------------                     Metadata      Publication   Interope -    Educational  De-  Assembly     LO  Components       Methodology   User     -------------------------------------------rability-------sign---------------------------------------------------------------------       UPV    Pro-   LOM           LOR          Not conside-  Student profile     Yes          Introduction, theory,  Educator with knowl-       posal                                     red                                          learning activity and  edge of Computer Sci-     ---------------------------------------------------------------------------------------evaluation------------ence----------------       Pattern Pro-   LOM/DC        Not conside-  SCORM        Educational   pat-  Not  conside- Media  content, infor- Educator with knowl-       posal                        red                        terns              red          mational content, pat- edge of Computer Sci-     ---------------------------------------------------------------------------------------tern-----------------ence----------------       MEDEOVA       LOM   CO      LOR    and   Not conside-  Student   training  Not  conside- Objective,  hierarchi-  Multidisciplinary                                   TLVE         red           needs             red          cal content, learning  team    (coordinator,                                                                                            activities, evaluation  pedagogical  advisor,                                                                                                                 subject matterexpert,                                                                                                                 designer,   computer     ------------------------------------------------------------------------------------------------------------science-specialist)-----       MIDOA         Not specified   Not conside-  Not conside-  Own   instructional Not  conside- Content,   activities,  Multidisciplinary                                   red           red           design             red          contextualization      team  (analyst,  de-                                                                                            elements             signer, author, devel-                                                                                                                 oper, user, analyst,     ------------------------------------------------------------------------------------------------------------educator)------------       ISDMELO        LOM           LOR    and   SCORM        Student profile  +  Yes          Instruction units with Multidisciplinary                                   TLVE                      learning    theories               a specificgoal,content  team and educators                                                             and  styles  +  in-               and activity                                                             structional  design     --------------------------------------------------------(ADDIE-)----------------------------------------------------------------       LOCOME        LOM           Not conside-  SCORM        Training needs      Yes          Digital units         Educator with knowl-                                   red                                                                            edge of Computer Sci-     ------------------------------------------------------------------------------------------------------------ence----------------       UBoA          DC            TLVE         SCORM        Learning strategies  Not  conside- Competencies,   sce-  Interdisciplinary team                                                             + own instructional red          nario, content, learn- and educator                                                             design model                    ing  activities,  tech-                                                                                            nology    innovation,     ---------------------------------------------------------------------------------------evaluation--------------------------------       Techno-        LOM           LOR    and   SCORM        Instructional   de-  Not  conside- Objectives,     con-  Computer educator       Pedagogical                  TLVE                      sign model         red          tents,     activities,     ---------------------------------------------------------------------------------------self- evaluation----------------------------       ISDOA          SCORM         Not conside-  SCORM        Student  learning  Not  conside- Objectives,     con-  Educator with basic                                   red                        styles             red          tent,  activity,  self-  knowledge  of Com -     ---------------------------------------------------------------------------------------evaluation------------puter Science--------       CROA          LOM/DC        LOR    and   SCORM        Own   instructional Yes          Specific    objective,  Educators                                   TLVE                      design                          content, activities and     ---------------------------------------------------------------------------------------self- evaluation----------------------------       UAT           Not specified   LOR    and   Not conside-  Dick  &   Carey’s  Yes          Learning   objective,  University  educator                                   TLVE         red           instructional design              informational content, /    multidisciplinary                                                             model                          learning activities and  teams     ---------------------------------------------------------------------------------------evaluation--------------------------------       Standard       IMS -         LOR    and   IMS -CP       Felder    learning  Not  conside- Contents, activities   Educator with knowl-       Technologies    Metadata      TLVE                      styles             red                               edge of Computer Sci-     --Proposal---------------------------------------------------------------------------------------------------ence----------------       AODDEI        Not specified   LOR          SCORM        ADDIE     instruc-  Not  conside- Objective,     infor-  Educator with no ex-                                                             tional model       red          mational    content,  perience in the devel-                                                                                            activities, evaluation  opment of LOs     ---------------------------------------------------------------------------------------of-learning--------------------------------       DICREVOA      LOM  -DC      LOR    and   SCORM        Own  Instructional Yes    (new  Specific    Objetive,  Educator   with   or                                   TLVE                      Design  center in  version)      Contents,  activities,  without experience     --------------------------------------------------------KOLB---------------------------self assessment---------------------------

2.2 Comparison of Considered Methodologies

Some of the methodologies and proposals analyzed here have been conceived to be used in contexts with specialized teams available for designing and creating LOs (designers, programmers, style correctors, educators, teachers, etc.), while others require educators themselves to tackle the entire LO design and creation process, which would imply that educators are required to have all the technological and educational skills needed for implementation. In this sense, it should be noted that educators coming from non-specialized technological contexts (e.g., Educational Sciences, Philosophy) will have difficulty when attempting to use the tools that support LO creation. On the other hand, those educators that come from specialized technological contexts (e.g., Engineering, Architecture) might have difficulties when tackling the educational and instructional design aspects of the LOs. As regards instructional design, it should be noted that the educational material that is designed may be suitable for students with a certain profile or learning style but not for other students. It would be interesting then to consider how to propose LOs that can motivate students with different student profiles or different learning styles.

In this context, DICREVOA is intended to be used by teachers who do not have a multidisciplinary team in their institutions of higher education, and it is they who can carry out the whole process of design, development and evaluation of LOs. This methodological approach emphasizes the proposed instructional design, introducing the cycle Kolb as its centerpiece of the activities. It also recommends the use of eXeLearning tool for designing the main structure of LO and at the same time it facilitates the incorporation of metadata into the LO, whether LOM or Core Dublin used. In addition to this it proposed evaluating LOs from the perspective of students and teachers.

Thus, the following questions gain relevance in relation to LO design and creation: RQ1: How should LO requirements be analyzed? RQ2: How can LO design be tackled from both an educational and a technological perspective for educators from different contexts? RQ3: What types of activities are appropriate for the different student learning styles and which of these can be designed for inclusion in LOs? RQ4: What are the tools that facilitate LO implementation and annotating LO metadata? RQ5: How can LO quality be assessed from the perspective of the educator as producer and from the perspective of the student as consumer? The proposal presented in this article is aimed at answering each of these research questions.

3 PHASES OF THE METHODOLOGICAL PROPOSAL FOR THE DESIGN, CREATION AND EVALUATION OF LEARNING OBJECTS

In this paper, we present a 5-phase methodological proposal called DICREVOA, which is the result of reviewing 29 proposals, heuristics and methodologies [30] used in Ibero-America for designing and creating LOs that are stored in Learning Object Repositories (LORs).

The phases in DICREVOA are as follows: A) Analysis, B) Design, C) Implementation, D) Evaluation, and E) Publication. The purpose of these phases is guiding educators in the design of LOs; to this end, a number of guidelines are provided to allow educators to make significant decisions in relation to LO-facilitated learning planning. This methodological proposal is aimed at educators who do not have access to a multidisciplinary team to provide the necessary support for the creation of LOs, which means that the educators themselves are responsible for the entire process of designing, creating and evaluating LOs. See Figure 2.


PIC

Fig. 2: Phases in DICREVOA


3.1 Analysis

In this phase, the requirements behind the need to develop a LO are surveyed, as well as its relation to the target audience profile. The following questions are answered: What is it that the target audience needs to learn? What is the topic to be developed? Who is the target audience of the material? To this end, the template shown in Table 2, can be used. It includes a description of each of the issues that must be considered.


TABLE 2: Learning Objects Needs Analysis
------------------------------------------------ -LO--Topic----LO-Topic-Identification.-------------  LO     De -  LO Contents Literal Description -scription---------------------------------------  Level        Sample group educational level identi-               fication (Elementary, High school, Uni- --------------versity,-Others)---------------------  Student’s    Set up the student’s profile based on  profile       learning styles, remarking on whether               or not the LO favors or supports one or --------------more-learning-styles.----------------  Estimated    Given time in minutes for students to  time      to  accomplish a complete LO.  cover   the -LO---------------------------------------------  Educational  Identify the elements and factors that  context      favor the LO teaching-learning process,               for instance the use of a LO in a TLVE --------------or in-face- to-face-class.--------------  Type  of Li- Determinethetypeoflicensetobe used -cense--------in the-LO-development.-------------  LO    Non -  Identify the LO  working technical re-  functional    quirements such as operating system,  Require-     browser, mobile devices and the re- -ments--------quired-plugins.---------------------

As regards students’ profiles, it is important to know what the prevailing learning styles are among students. This could be key for the creation of digital educational materials, and its purpose is to adapt to students’ preferences to help them learn.

Learning styles are the cognitive, affective and physiological features that act as relatively stable indicators of how students perceive, interact with, and respond to their learning environments [31]. The cognitive features are linked to preferences students have for perceiving and processing the information they learn; affective features are related to student motivations and expectations for learning, and physiological features are connected to biotype and biorhythm aspects. For Kolb [32], learning styles are the preferred abilities to learn, which are determined by hereditary factors, previous experiences, and the demands of the current environment in which the individual acts. Kolb’s perspective is a constructivist one, hence his emphasis on the experiential nature of learning, understanding it as a knowledge-building process with a creative tension among the four modes mentioned.

Alonso and Honey, the same as Kolb, defend that the best learning occurs when students go through four cyclical phases, and they defined the following learning styles based on such phases: Active, Reflective, Theoretical, and Pragmatic [31].

To survey student profiles, the CHAEA questionnaire [33] is recommended. It allows taking a snapshot of the learning style of a single student or a group of students. This is significant information for future decisions that have to be made during the design phase, when the designer of the LO will decide if the predominant learning styles are to be favored, or if activities will be designed to take into account all four learning styles. Figure 3 shows the learning style profile model of a group of students.


PIC

Fig. 3: Learning style profile model for a group of students


3.2 Design

During this phase, the LO is designed both from a pedagogical perspective as well as a technological perspective. Design aspects that affect the educational and technological components of the LO are developed as shown in Table 3.


TABLE 3: Template for the Learning Objects Design Stage
-------------------------------------------- -Template--for-the-Learning-Objects-Design--- -1.-Instructional-Design-------------------- -1.1.-Internal structure-of-the-LO--------------- -a) Learning Objective----------------------- -b) Contents-------------------------------- -c) Activities-------------------------------- -d) Self- evaluation--------------------------- -2.-Multimedial-Design---------------------- -2.1.-Interface design------------------------- -2.2.-Layout-structure------------------------- -2.3.-Path-----------------------------------

To be able to better understand the criteria used in the design of the latter template and for the sake of its use, two main components have been taken into account, 1) Instructional Design and 2) Multimedial Design.

3.2.1 Instructional Design

Instructional design involves defining aspects pertaining to the internal structure of the LO. In accordance with the definition adopted in this article, an LO is mainly made up of four elements, a learning objective, contents, activities, and a self-evaluation.

Learning Objective: a LO should have a single learning objective. Taxonomies that allow identifying progressive complexity levels of cognitive operations are used to define LOs, using verbs that are associated to the cognitive category that is to be described. One of the recommended taxonomies to do this is the one proposed by Anderson and Krathwohl, which is an adaptation of Bloom’s taxonomy levels [34], and which defines the following 6 taxonomic levels:

  1. Remember: recognize, list, describe, identify, recover, locate.
  2. Understand: interpret, summarize, infer, classify, compare, explain, and translate.
  3. Apply: execute, implement, perform, use
  4. Analyze: differentiate, distinguish, select, integrate, structure, outline.
  5. Evaluate: review, critique, experience, judge, test, detect, monitor.
  6. Create: design, build, produce, conceive, trace, elaborate, generate, invent.

To formulate the learning objective, one of the verbs listed is used in combination with the specific contents that will be covered by the LO, and then the conditions or circumstances in which the student is expected to perform are added. Figure 4 shows an example.


PIC

Fig. 4: Learning Objective Formulation


Contents: As regards the contents of a LO, the following should be determined: What contents will be used? How should they be organized? How should they be presented? The following aspects should be considered [22]:

  1. Content selection: it allows defining what is going to be included in the LO and what is not. Not all knowledge related to the topic covered by the LO is relevant for inclusion; the learning objective defined at the beginning of the instructional design, and its degree of specificity in particular, determines the answer to the question ”What should be included?”.
  2. Organization: it allows identifying the appropriate way of organizing the various types of contents (conceptual, procedural, conditional). In the case of conceptual contents, it is recommended to organize them hierarchically, starting from the more general concepts in the topic, then moving to more specific ones, and finally introducing specific examples, always following a deductive logic. See Figure 5.


    PIC
    Fig. 5: Example of Deductive Logic


    In the case of procedural contents (processes, techniques, skills, etc.), it is recommended to first introduce the task or end result of the procedure so as to provide context for the information, and then go into the details for each of the steps, in the corresponding order. See Figure 6.


    PIC
    Fig. 6: Example of Procedural Contents


    Conditional contents are the criteria used by experts to make decisions - what, when and how to use certain pieces of knowledge (conceptual/procedural) to solve a case, situation or problem. When emphasis is on solving problems, it is recommended that the problem or situation to be solved be used as structural platform around which contents are organized, in order to favor content transfer and application. See Figure 7.


    PIC
    Fig. 7: Example of Conditional Contents


  3. Content presentation: How contents are presented is also key, especially when dealing with digital materials. Mayer’s SOI (Select, Organize, Integrate) model [35] is proposed, which was designed to emphasize these cognitive processes; see Figure 8.
    PIC
    Fig. 8: Contents Presentation


Activities: Activities should be selected based on: learning objective, type of contents, target audience, purpose of the activity, and learning processes it stimulates. There are four different types of activities [22]:

  1. Diagnostic: They are aimed at identifying previous knowledge, learning about what students know and what they do not know, and activating what they do know before the learning process begins.
  2. Motivational: They are aimed at generating reasons to learn.
  3. Comprehension, application, transfer: They are aimed at processing, organizing, storing, and using the information.
  4. Integration: They are aimed at promoting the interrelation, integration, and summarization of the various topics studied.

When adjusting the design of the LO to the specific needs of the target students and using Kolb-Honey and Mumford proposal in relation to learning styles, focus is on the activities that must be included to accommodate the different learning styles. The learning styles model created by Kolb assumes that, to learn something, the information received must be used or processed. To Kolb, the optimal learning is the result of working with the information in four phases: acting, reflecting, theorizing, and experimenting. See Figure 9.


PIC

Fig. 9: Kolb’s Cycle


These are possible starting points:

  • A direct and concrete experience: active student.
  • An abstract experience, which is the kind of experience obtained when reading about something or hearing someone talk about it: theoretical student.

All experiences, both concrete and abstract, become knowledge when they are elaborated upon in either of these ways:

  • By reflecting upon and thinking about them: reflective student.
  • By actively experimenting with the information received: pragmatic student.

It is recommended to introduce a brief initial activity as a learning trigger that favors the active style, then a reflective activity that allows developing a deeper learning, followed by a third activity that provides students the tools needed to theorize over the topic, and finally a fourth activity that encourages the application of the new knowledge in a contextualized situation or problem.

Self-evaluation: Self-evaluation allows establishing if the learning objective has been achieved, generating a reflective instance over what has and what has not been learnt. For the self-evaluation stage, the following criteria should be taken into account:

  1. Analyzing the difficulty of the items proposed taking into account the level of knowledge and understanding.
  2. Designing random open or closed questions and tasks; these can be configured considering number of attempts, time available to provide an answer, additional time bonus, etc.
  3. Preparing instructions, questions and options for students to have an advantage that is not linked to knowledge.

As regards student feedback, when they provide a correct answer, additional information should be provided and they should be redirected to other sources. If they provide an incorrect answer, hints or clues should be provided to guide new attempts, the reasons why the answer is wrong should be explained, and the students should be invited to review certain topics or they should be redirected to other sources.

3.2.2 Multimedial Design

Multimedial design involves defining aspects pertaining to the structure of the interfaces that will bring the LO to life. These features are the following:

Interface Design: the design of the interface for the LO must be specified; templates that apply metaphors in relation to the topic can be used. This will be dependent on the tool that is later on used to implement the design of the LO. It is recommended to use Cascading Style Sheets (CSS) or templates developed to support Web pages (HTML).

Layout Structure: LO layout structure must be specified; one of the four designs mentioned in this work must be selected, such as:

  1. Navigation block on the left
  2. Navigation block on top
  3. Navigation block to the right
  4. Combined block

Considering this, interface design can be based on a structure that shows the navigation block on the left. See Figure 10.


PIC

Fig. 10: Graphic interface design with the navigation block on the left


Path: content organization should follow a sequence going from known to unknown, from immediate to medium-term, from concrete to abstract, and from easy to difficult. For instance, a simple structure can be used that allows organizing target knowledge and skills into a hierarchy; or a branched linear structure where users can follow different routes based on their interests or previous knowledge.

3.3 Implementation

The purpose of this phase is implementing the LO that was designed in the previous stage. Therefore, it is important to take into account the following references to implement the ”core´´ of the LO and to label it.

Computer tools: it is recommended to use authoring tools that allow integrating each of the elements considered in the design. In this proposal, the eXelearning tool allows implementing the designed LO. Table 4 shows the review of 6 tools that allow creating ”learning activities´´ that can be used and integrated into eXelearning.


TABLE 4: Computer Tools Comparative Chart
-------------------------------------------------------------------- -Software---License----Publishing-Format----Compressing--standards-- -Raptivity---commercial-html,-html5, flash, zip-SCORM,-AICC,-Tin-Can--- -Storyline----commercial-html,-html5, flash, zip-SCORM------------------ -Captivate---commercial-html5, flash----------SCORM,--AICC,-Tin-Can--- -Ardora-----libre-------html,-zip-------------SCORM------------------- -Camtasia---commercial-html,-avi, iTunes-----SCORM------------------- -Cuadernia--libre-------zip------------------SCORM-------------------

Metadata: after implementing the LO, its metadata card must be filled in; this information will be used to describe and then locate the LO in a Learning Objects Repository (LOR). To fill in the metadata, the eXelearning 2.0 tool allows generating them using the LOM (Learning Object Metadata) standard or the DCMI (Dublin Core Metadata Initiative) standard. Figure 11 shows a metadata input card using the DCMI standard.


PIC

Fig. 11: Metadata card in eXelearning


3.4 Evaluation

This phase is focused on evaluating the LO considering two perspectives:

  1. From the standpoint of the user as consumer of the LO (student): users participate to determine if the LO really helped them learn. The CUSEOA (LO Student Satisfaction Questionnaire) questionnaire was used to gather this information; this questionnaire asks about the global reaction to the LO and then identifies its collaboration from a pedagogical and technical perspectives [36].
  2. From the standpoint of the producer of the LO (educator): a digital educational material evaluation signature called CODA is used to gather the minimum set of criteria from LO quality models from a didactic and technological perspective [37]. It should be noted that, at this point, an evaluator can be an academic peer who is not necessarily familiar with the LO target topic.

3.5 Publication

The purpose of this phase is to package the LO using a standard that allows publishing it on different TLVEs. To achieve this, it is recommended to use a formal standard and to consider that the LO can be stored in a LOR so that it can be searched for and retrieved from. Deploying the LO in a TLVE and Publishing the LO in a LOR will be now explained in further detail:

Deploying the LO in a TLVE: to deploy a LO in a TLVE, the following steps must be carried out: a) Package the LO using a standard that can be recognized by a Web environment (SCORM, IMS). Reload Editor is recommended to package LOs. b) Deploying the LO in a TLVE that is supported by the standard used to package it. See Figure 12.


PIC

Fig. 12: Steps for packaging and deploying a LO on a TLVE


Publishing the LO in a LOR: the final step after producing and evaluating the LO is publishing it and making it available to the students through a LOR.

4 Case Study

To assess DICREVOA, a case study was used, since it is an empirical method that allows analyzing phenomena in their own context. This method is used when there is no marked boundary between phenomenon and context, or when there is a lack of experimental control and information is gathered from a few entities [38]. This case study was carried out with 3 groups that were trained on the design and creation of LOs using the DICREVOA methodology. Each group took 16 hours of training - Group 1 included educators from a university and several high schools from the same city; Group 2 and Group 3 included several educators from high schools and universities from various cities, in the context of two post-grade courses offered by 2 universities. Table 5 shows the characteristics of each group.


TABLE 5: Groups
--------------------------------------------------------------------------------------------- --Group-#----#-of-participants--Type--of student-------------------Professional-profile-------- --1----------14----------------Secondary-and-tertiary-level educators-Technological and-educational --2----------30----------------Secondary-and-tertiary-level educators-Mostly-educational---------- --3----------30----------------Secondary-and-tertiary-level educators-Mostly-technological---------

Group 1 members were secondary- and tertiary-level educators; they had little theoretical knowledge about LOs and almost no experience in the design and creation of LOs. Group 2 members were secondary- and tertiary-level educators; they had little theoretical knowledge about LOs and no experience in the design and creation of LOs. None of the participants had professional degrees related to the area of Computer Science; however, they did in the area of Educational Sciences. Group 3 members were secondary- and tertiary-level educators and they neither had theoretical knowledge about LOs nor experience in their design and creation. All of the members in this group had a professional degree - either a bachelor degree or a degree in computer engineering.

The case study was used to determine the advantages of using DICREVOA in contexts where users have different types of profile (technological and educational). It would also allow analyzing the feasibility and possibilities offered by the methodology developed to help educators design and create LOs and incorporate them to the teaching and learning process. To carry out the case study, a protocol containing the activities was developed:

  1. Case definition
  2. Case design and planning
  3. Preparation for data collection
  4. Data collection
  5. Analysis and interpretation of the data collected
  6. Results report

For the data collection stage, two questionnaires were administered at two different points in time (before training and after it) in each of the groups that took the training course (Group 1, Group 2 and Group 3) and passed it. The questions included in the questionnaire were reviewed and validated by academic peers that are content experts. Answers were collected using Google’s online survey system. The questionnaires used in this research can be found at:

Pre-course questionnaire:

  • http://ow.ly/YfABj

Post-course questionnaire:

  • http://ow.ly/YfB9a

As regards the questions asked, the following results were obtained:

  • Is the definition of LO provided by the DICREVOA methodology appropriate?
    • For Group 1: 86.6% of the participants considered that the methodology provides an appropriate definition for LOs.
    • For Group 2: 92.5% of the participants considered that the methodology provides an appropriate definition for LOs.
    • For Group 3: 100% of the participants considered that the methodology provides an appropriate definition for LOs.
  • About the concept of LOs.
    • For Group 1: in the diagnostic questionnaire, 90% of the educators indicated that they did not know what LOs were; however, after taking the course, 90% of the participants were able to define them.
    • For Group 2: in the diagnostic questionnaire, 100% of the educators indicated that they did not know what LOs were; however, after taking the course, 85.71% of the participants were able to define them.
    • For Group 3: in the diagnostic questionnaire, 100% of the participants indicated that they did not know what LOs were; however, after taking the course, 93.3% of the participants were able to define them.

    It can be inferred then, that before taking the course, almost no one knew what LOs were but this paradigm significantly shifted after they took the course. See Figure 13.


    PIC
    Fig. 13: Results about familiarity with the concept of LOs before and after the course


  • About the instructional design of the LO
    • For Group 1: 80% can define it, 16.6% can define it to a certain extent.
    • For Group 2: 64.3% can define it and 35.7% can define it to a certain extent.
    • For Group 3: 80% can define it and 16.7% can define it to a certain extent.
  • To what extent do you consider that this is a recommendable methodology for designing and creating LOs?
    • For Group 1: 33.3% highly recommendable and 66.7% recommendable.
    • For Group 2: 57.1% highly recommendable and 42.9% recommendable.
    • For Group 3: 83.3% highly recommendable and 16.7% recommendable.

    It can be inferred that the methodology had a positive impact on participants, since no one was indifferent to it or found it to be little recommendable or not recommendable at all. See Figure 14.


    PIC
    Fig. 14: Results about potentially recommending the DICREVOA methodology


  • Which is your global score for the methodology used?
    • For Group 1: 60% considered it to be useful and 40% considered it to be very useful.
    • For Group 2: half of the participants (50%) considered it was useful and the other half considered it was very useful.
    • For Group 3: 16.7% considered it to be useful and 83.3% considered it to be very useful.
  • How do you assess the balance between the theoretical and the practical components developed in this course?

    The purpose of this question was to avoid the influence that the trainers might have when teaching the methodology; the ends of the scale were ”very theoretical” and ”very practical”.

    • For Group 1: results showed that the training course was very practical for 33.3% of the participants, not practical enough for 20% of the participants, and balanced for the remaining 46.7% of the participants.
    • For Group 2: results showed that the training course was very practical for 28.6% of the participants, not practical enough for 35.7% of the participants, and balanced for the remaining 35.7% of the participants.
    • For Group 3: results showed that the training course was very practical for 80% of the participants, not practical enough for 3.3% of the participants, and balanced for the remaining 16.7% of the participants.

5 Discussion

Based on the gathered results from the case study, it is possible to state that the definition adopted about what an LO means, allows us to understand its nature, and most importantly, it allows us to make the creation of digital educational material, based on this paradigm, feasible, through DICREVOA.

In Figure 14, it can be clearly seen that the percentages obtained from the three groups are uneven, in relation to what extent DICREVOA is recommended to design and create LOs. Group 1 states that DICREVOA is strongly recommended, as seen in its 33.30%. On the other hand, groups 2 and 3 show an increase in their percentages. They strongly recommend it in a 57.10% and in an 83.30%, respectively. The variability of the percentages can be explained by the context predominantly (educational or technological) of the coming course participants of each group and the orientation (techno-pedagogical) DICREVOA has.

Regarding the results of how practical the training that group 3 (83.3% - more technological context) as well group 2 (28.6% - more educational context) had, it could be thought that a strong feature of DICREVOA is the instructional proposal presented when designing the LO. It is noteworthy that DICREVOA, is the result of the analysis and incorporation of the best practices that have taken a number of methodologies to create LO during the last 15 years.

The paradigm of LO is still a matter of discussion and despite of not reaching agreement within the scientific community in its definition, several proposals for its creation have been generated, which try to be de facto. A problem that is evident in these proposals and that has been addressed in this work, is the orientation they usually have, since depending on the context they come from, they focus on exploiting either more technological implementation of LO as software artifact leaving out consistent educational design which the LO should have to fulfill the purpose for which it was created. In other cases, the focus is on instructional design detail so that it is not possible its implementation through software tools. In addition, the size a LO should have is another opinion that varies among different proposals, where larger LOs are less reusable in different educational contexts, and smaller Los tend to be more reusable in different educational contexts and this feature also allows them to be combined with others to form larger units.

However, despite the amount of work done in relation to this paradigm, there are still some questions that should be addressed in the future. One has to do with the way LOs should be assembled to generate larger units and the second question addresses the fact that the proposed instructional design of this methodology can be scaled to design e-learning courses, such as MOOCs.

6 Conclusions

The production of digital educational material under the Learning Object paradigm is still a challenge on tertiary education both for educators as well as production teams, since creating LOs involves design logics for technical and educational reutilization. When facing the design of a LO, it is important to have a practical definition as starting point that allows understanding the characteristics of LOs. From a technological point of view, the LO must be considered as a software artifact. It must also be updatable, both in relation to software as well as contents. The LO must be stored in a LOR (published) and it must be described by means of metadata (locatable). It must be possible to deploy it on different TLVEs (technological reutilization) using packaging standards (interoperable).

From an educational point of view, the LO must be considered as a teaching and learning aide, with an educational intent (educational design to favor learning) aimed at facilitating comprehension, the representation of a concept, a theory, a phenomenon, etc., and promoting the development of skills, abilities and competencies in individuals. The LO must also have an internal structure or components (objective, content, activity, evaluation) (granularity) to favor the possibility of integrating the LO into larger collections, such as lessons, courses, etc. (generativity).

The DICREVOA methodological proposal presents 5 phases aimed at guiding educators through the design and creation of LOs. To achieve this, the analysis phase allows surveying the needs that will be covered by the LO; the design phase is aimed at providing the necessary guidelines to consider both the educational and technological perspectives; the implementation phase allows creating the LO using the eXeLearning authoring tool that facilitates both the creation of the LO and the provision of its metadata; the evaluation phase allows evaluating the LO from the point of view of the educator as producer and the student as consumer; and the publication phase allows deploying the LO on a LOR or a TLVE.

The case study presented allowed testing the feasibility and usability of the DICREVOA methodology, which provides an accessible mechanism to help educators make appropriate decisions when creating LOs that are of good quality but, above all, that are suitable for their needs.

Finally, this study makes evident that the creation of LOs from a ”de facto´´ definition provides teachers understanding of this paradigm from a techno-pedagogical position to create digital educational material.

Through the methodology presented in this paper, DICREVOA, teachers will have in their hands a tool for analyzing initial requirements that cause the need to create a LO, seeking to balance the design of LO in both the educational and technological scope, trying to meet various learning styles through the cycle of Kolb and bringing it into practice through the use of the eXelearning tool for structuring the conceptual basis of a LO, easing the loading of metadata. The given templates CUSEOA and CODA allow teachers a first approach to the evaluation of this type of MED in order to be evaluated by teachers and students.

As future work, we expect to demonstrate the efficacy of DICREVOA through new experiments. To achieve this, there will be groups of students that will use the LO and will assess if it helped them achieve the learning objective proposed or not. We are also planning to promote DICREVOA to be used by educators with various competencies, be these educational and/or technological.

7 Acknowledgments

This study has been funded by the Research Direction of the University of Cuenca (DIUC) through research project ”Assessing the impact of a methodology to be developed for the creation of quality digital educational material when applied to educators at the School of Engineering of the University of Cuenca´´ and project ”Assessing a flipped learning model based on Learning Objects (LOs) to reduce university student failure,” for which we are truly thankful. We also thank all the individuals and universities that participated in the training course given as part of this methodological proposal and who answered our surveys.

References

[1]   F. García, J. Portillo, and M. Benito, “Nativos digitales y modelos de aprendizaje.” SPDECE, 2007.

[2]   D. Álvarez, J. Mendoza, and J. Kelsy, “Diseño e implementación de objetos de aprendizaje basado en tecnologías estándares,” ., . [Online]. Available: http://aves.edu.co/ovaunicor/webroot/recursos/1/index_DIOTE.pdf

[3]   G. Astudillo, C. Sanz, and P. Willging, ”Análisis del estado del arte de los objetos de aprendizaje. Revisión de su definición y sus posibilidades”. Universidad Nacional de La Plata, La Plata, Buenos Aires., 2011. [Online]. Available: http://sedici.unlp.edu.ar/ARG-UNLP-TPG-0000002954/12061.pdf

[4]   J. L’Allier, “NETg’s precision skilling: The linking of occupational skills descriptors to training interventions,” netg, 1998. [Online]. Available: http://www.netg.com/research/pskillpaper.htm

[5]   C. Barritt, D. Lewis, and W. Wieseler, “Cisco systems reusable information object strategy. definition, creation overview, and guidelines,” Cisco Systems, Inc., 1999.

[6]   IEEELTSC, “Draft standard for learning object metadata,” New York: Institute of Electrical and Electronics Engineers, Inc., 2002.

[7]   D. Wiley, “Connecting learning objects to instructional design theory: A definition, a metaphor, and a taxonomy,” The Instructional Use of Learning Objects: Online Version, 2000. [Online]. Available: http://reusability.org/read/chapters/wiley.doc

[8]   D. Wiley, S. Waters, D. Dawson, B. Lambert, M. Barclay, D. Wade, and L. Nelson, “Overcoming the limitations of learning objects,” Journal of Educational Multimedia and Hypermedia, vol. 13, no. 4, pp. 507–521, 2004.

[9]   L. García Aretio, “Objetos de aprendizaje. características y repositorios,” Bol. Electrónico Not. Educ. Distancia BENED, 2005. [Online]. Available: http://www.uned.es/catedraunesco-ead/p7-04-05.htm

[10]   M. de Educación Nacional Colombiano, “Objetos virtuales de aprendizaje e informativos,” MEN, 2006. [Online]. Available: http://www.colombiaaprende.edu.co/html/directivos/1598/article-172369.html

[11]   A. Chiappe, Y. Segovia, and Y. Rincon, “Toward an instructional design model based on learning objects,” Educational Technology Research and Development, vol. 55, no. 6, pp. 671–681, 2007.

[12]   J. Silva Sprock, C. Ponce Gallegos, and Y. Hernández, “Estado del arte de las metodologías para el desarrollo de objetos de aprendizaje,” Conferencias LACLO, vol. 4, no. 1, 2013.

[13]   S. Martínez Naharro, P. Bonet, P. Cáceres, F. Fargueta, and E. García, “Los objetos de aprendizaje como recurso de calidad para la docencia: criterios de validación de objetos en la universidad politécnica de valencia,” in IV Simposio Pluridisciplinar sobre Diseño y Evaluación de Contenidos Educativos Reutilizables (SPDECE 2007), 2007.

[14]   J. Delgado, J. Valdivia, R. Morales, S. González, and M. Chan Núñez, “Desarrollo de objetos de aprendizaje a basado en patrones,” Presentado en Virtual Educa Brasil, 2007. [Online]. Available: http://e-spacio.uned.es/fez/eserv/bibliuned:19220/n03delgado07.pdf

[15]   J. Monsalve Pulido and F. Aponte Novoa, “MEDEOVAS-metodología de desarrollo de objetos virtuales de aprendizaje,” Conferencias LACLO, vol. 3, no. 1, 2012.

[16]   A. Barajas Saavedra, J. Muñoz Arteaga, and F. Álvarez Rodríguez, “Modelo instruccional para el diseño de objetos de aprendizaje: Modelo MIDOA,” Presentado en Virtual Educa, Brasil, p. 11, 2007.

[17]   L. Blondet Baruque and R. Nascimento Melo, “Learning theory and instruction design using learning objects,” Journal of Educational Multimedia and Hypermedia, vol. 13, no. 4, pp. 343–370, 2004. [Online]. Available: http://apan.net/meetings/busan03/materials/ws/education/articles/Baruque.pdf

[18]   M. Medina Balda and M. López López, LOCoME: Metodología de Construcción de Objetos de Aprendizaje. Universitat de Oviedo y REDAOPA. Actas del III Simposio Pluridisciplinar sobre. Diseño, Evaluación y Descripción de Contenidos Educativos Reutilizables (SPDECE), 2006. [Online]. Available: http://spi03.sct.uniovi.es/moodle_cv/mod/resource/view.php?id=233

[19]   R. Correal Cuervo, C. Montañez Torres, and L. Bernal Zamora, “UBoa - metodología para la creación de objetos de aprendizaje de la universidad de boyacá,” Universidad de Boyacá, 2009.

[20]   Y. Hernández Bieliukas and A. Silva Sprock, “Una experiencia tecnopedagógica en la construcción de objetos de aprendizaje web para la enseñanza de la matemática básica,” Eduweb, vol. 5, no. 1, pp. 57–72, 2011.

[21]   C. Castro, M. Serna, and R. Taborda Botero, “Ingeniería de software para desarrollar objetos de aprendizaje - ISDOA,” in EATIS ’12: Proceedings of the 6th Euro American Conference on Telematics and Information Systems. New York, NY, USA: ACM, 2012, pp. 347–353.

[22]   C. Sanz, F. Barranquero, and L. Moralejo, “Curso de doctorado metodología CROA,” Universidad Nacional de la Plata, 2014.

[23]   G. Padilla and M. Hernández, “Metodología -UAT: Una metodología para el diseño de oa, en avances en objetos de aprendizaje. experiencias de redes de colaboración en méxico, editado por f.j. Álvarez y jaime muñoz.” Eduweb, vol. 5, no. 1, pp. 47–70, 2011.

[24]   J. Muñoz, B. Osorio, F. Álvarez, and P. Cardona, “Metodología para elaborar objetos de aprendizaje e integrarlos a un sistema de gestión de aprendizaje,” Revista Apertura del Sistema de Universidad Virtual, Universidad de Guadalajara, México. ISSN 1665 - 6180, 2006. [Online]. Available: http://ingsw.ccbas.uaa.mx/sitio/images/investigaciones/13TEMunozArticulo.pdf

[25]   V. Štuikys and R. Damaševičius, “Towards knowledge-based generative learning objects.” Information technology and control, vol. 36, no. 2, 2015.

[26]   J. Maldonado, J. P. Carvallo, and J. Siguencia, “Metodologías y propuestas metodológicas para el diseño de objetos de aprendizaje: Un estado del arte en iberoamérica,” Conferencias LACLO, vol. 6, no. 1, 2015.

[27]   Y. Collaguazo, L. Chamba-Eras, and A. Padilla, “Propuesta de un modelo genérico para el diseño y valoración de objetos de aprendizaje basado en estándares e-learning,” Conferencias LACLO, vol. 6, no. 1, 2015.

[28]   J. Maldonado, A. Fernández-Pampillón, and C. Sanz, “Analysis framework for tailored selection of learning objects methodologies.” The International Workshop on Collaboration and e-Learning, CTS 2015, Atlanta, Georgia, Estados Unidos. (In Press), 2015.

[29]   P. L. Smith and T. J. Ragan, “Instructional design,” Wiley New York, p. 3, 1999.

[30]   J. Maldonado, C. Sanz, and A. Fernández-Pampillón, “Desarrollo de un marco de análisis para la selección de metodologías de diseño de objetos de aprendizaje (oa) basado en criterios de calidad para contextos educativos específicos,” Universidad Nacional de La Plata, La Plata, Buenos Aires., pp. 79 – 110., 2015. [Online]. Available: http://hdl.handle.net/10915/45063

[31]   C. Alonso, D. Gallego, and P. Honey, “Los estilos de aprendizaje. procedimientos de diagnóstico y mejora,” Ediciones Mensajero. Universidad de Deusto. Bilbao, 1994.

[32]   A. Y. Kolb and D. A. Kolb, “The kolb learning style inventory,” www.learningfromexperience.com, Recuperado el 4 de Julio de 2014, 2005.

[33]   P. Honey and C. Alonso, “Baremo cuestionario chaea de estilos de aprendizaje.” [Online]. Available: http://www.estilosdeaprendizaje.es/menuprinc2.htm

[34]   L. Anderson and D. Krathwohl, “Revision of bloom’s taxonomy of educational objectives. en a taxonomy for learning, teaching and assessing.” En A Taxonomy for Learning, Teaching and Assessing. Longman, New York., 2001.

[35]   R. E. Mayer, “Diseño educativo para un aprendizaje constructivista,” in Diseño de la instrucción: teorías y modelos: un nuevo paradigma de la teoría de la instrucción. Santillana, 2000, pp. 153–172.

[36]   S. Massa, “Objetos de aprendizaje: Metodología de desarrollo y evaluación de la calidad,” Ph.D. dissertation, Facultad de Informática, 2013.

[37]   A. M. Fernández-Pampillón, E. Domínguez Romero, and I. D. Armas Ranero, “Herramienta de evaluación de la calidad de objetos de aprendizaje universitarios COdA: Guía para la producción y evaluación de materiales didácticos digitales. v. 1.1,” ., 2011.

[38]   F. Benbasat, D. Goldstein, and M. Mead, “The case study research strategy in studies of information systems.” MIS Quaterly 11(3), pp. 369–386, 1987.

Creative Commons License Todo el contenido de esta revista, excepto dónde está identificado, está bajo una Licencia Creative Commons