The research infrastructures in question are strongly interlinked with the GLAM (Galleries, Libraries, Archives, and Museums) sector, especially with libraries and archives, which are often essential partners of and contributors to eHumanities and eHeritage infrastructures. To reap the benefits of research infrastructures, (aspiring) digital humanities and cultural heritage scholars (users) as well as professionals working at institutions that are research infrastructures themselves, need to acquire additional theoretical knowledge and practical skills so they can effectively make use of digital humanities and digital heritage research infrastructures in supporting the various phases and facets of the research life cycle (see Figure 1). To enhance the awareness and knowledge of using and building research infrastructures, in order to assist and accelerate digital research methods and tools in the Humanities and Cultural Heritage Studies, the PARTHENOS project (an acronym for “Pooling Activities, Resources and Tools for Heritage e-Research, Optimization and Synergies”) funded by the European Commission’s Horizon 2020 framework, has a strong focus on training and education, which is coordinated by the PARTHENOS Training team.1
Given the resource restrictions within the project, PARTHENOS has focused from the beginning on asynchronously available materials for self-learning and ‘train the trainers’ opportunities (utilizing Web technologies) to maximize its impact as outlined in the overarching guiding principles for the development of the PARTHENOS Training Plan. In order to complement and promote its online training portfolio accessible via a dedicated portal,2 in the first half of 2018 PARTHENOS developed and delivered its eHumanities and eHeritage Webinar Series (hereafter: PARTHENOS Webinar Series) that focussed on enhancing awareness and knowledge of using digital research infrastructures. The author has been closely involved in conceiving, organizing, and conducting the series and also acted as one of the webinar trainers.
A webinar is “a live online educational presentation during which participating viewers can submit questions and comments;” it derives from the words “web + seminar” (both quotes: Webinar, n.d.). The PARTHENOS Webinar Series aimed to overcome some significant barriers to the increased uptake and exploitation of digital research infrastructures for the humanities and heritage community, that can be described in short as a lack of understanding of what these research infrastructures are, do, and why they might be a useful topic to engage with in more depth in the first instance. The first exploratory run of the PARTHENOS Webinar Series took the form of five one-hour webinars during February and April 2018 and they attracted a total of 161 participants (and 235 registrations) from 27 countries, mainly from Europe and beyond (Australia, Cameroon, Nigeria, South Africa, and the United States).
The paper presents the intellectual, educational, and practical context in which the PARTHENOS Webinar Series was conceived, the analysis of the data collected during the PARTHENOS Webinar Series (attendance and feedback from trainees and trainers), and it frames the post-processing activities related to its accompanying resources within the movement towards Open Educational Resources (OER). It concludes with a discussion of the potentials of webinars in the broader context of infrastructure training. As infrastructure training contributes to world-class research, the PARTHENOS Webinar Series is a timely training intervention into the role of research infrastructures in the research life cycle. The author is convinced that the findings are not only of relevance for those directly involved in infrastructure training but also for information professionals in research libraries and information centres who are considering adding webinars to their training portfolio or are already conducting webinars.
The paper has four aims:
- To describe the intellectual, educational, and practical context of the PARTHENOS Webinar Series as an eLearning format aimed at the professional development of humanities and cultural heritage researchers and practitioners related to infrastructural topics.
- To present an outline of the actual PARTHENOS Webinar Series as course materials, including learning objectives and details about the course design and structure.
- To discuss the analysis of the data collection exercises conducted during the first five webinars on how the PARTHENOS Webinar Series met the professional development and capacity building training needs of (digital) humanities and cultural heritage scholars and information professionals in the context of the PARTHENOS project.
- To serve as a basis for a broader discussion of the effectiveness, improvement opportunities, and further potentials of webinars in the wider context of research infrastructures.
2. General Intellectual and Practical Contexts
2.1. Why is it Important to Provide Training in the use and Potential of eHumanities and eHeritage Research Infrastructures in the Research Process?
Research infrastructures undeniably have an increasing presence in Humanities and Cultural Heritage research (e.g., Benardou, Champion, Dallas, & Hughes, 2017; Duşa, Nelle, Stock, & Wagner, 2014; ESF, 2011; ESFRI, 2016). They contribute to a transformation of methodology and epistemology in these disciplines and offer new possibilities in all phases of the research process (see Benardou et al., 2017). To maximise the benefits of using and cooperating with research infrastructures in the different phases of the research life cycle, humanities and cultural heritage scholars and information professionals as the main kind of users and providers of research infrastructures need to be aware of relevant research infrastructures, their potential to aid research and to acquire additional theoretical knowledge as well as practical skills. To cater to these needs, from the very beginning PARTHENOS has had a strong focus on training and education. The overarching principles behind PARTHENOS Training (led by Trinity College Dublin) are described in the ‘Initial Training Plan’ (Edmond et al., 2016) and are based on an internal training requirements analysis (see Drude et al., 2016, esp. p. 243–276).
2.2. The PARTHENOS Training Approach and Overarching Principles
The initial training requirements analysis revealed that many research infrastructures in the arts and humanities already offer training events for researchers (e.g. Summer and Winter Schools) and develop, provide, and contribute to platforms for online training materials (e.g. #dariahTeach, CLARIN Videolectures).3 However, many of the training events and materials focus on the activities and products of individual research infrastructures and therefore are highly project or topic specific. PARTHENOS identified a gap concerning introductory topics that serve to expand the understanding and uptake of research infrastructures in general. Therefore, the first overarching principle of PARTHENOS Training during the first project phase (until month 24) was to focus on research infrastructure related knowledge and skills on a macro scale/meta level.4 During the second phase, this approach was to be complemented by including and thus supporting the uptake of more specific outputs of PARTHENOS and the PARTHENOS cluster partners by a broader audience.
Another overarching principle is the distinction between “knowledge-what” and “knowledge-how,” in other words, the difference between awareness-raising and skills building. While skills-based training activities that focus on either a specific new tool or research environment are undeniably useful for furthering the uptake of these specialised products, they often “miss out” the first step on the user side, that is attracting less experienced audiences to engage with digital research infrastructures in the first place by informing them why they are relevant to them. Therefore, PARTHENOS Training seeks to develop appropriate and accessible training interventions that foster general knowledge about research infrastructures in order to attract and incentivise learners to seek out more specific information about specialized services and tools.
The third guiding principle of PARTHENOS Training acknowledges that research infrastructures in particular involve a broader ecosystem of actors than scholars only. They often build on and expand established physical cultural heritage research infrastructures and institutions as owners and thus providers of the relevant primary research data (see ESF, 2011, p. 8–14). Many digital collections, created and managed by librarians and other information specialists, are already part of eHumanities and eHeritage research infrastructures (e.g. Europeana, EHRI, and Cendari)5 and they are used by researchers globally to produce world-class research. To further enhance the visibility, accessibility, and re-use of research infrastructures, increased collaboration and cooperation combined with awareness-raising interventions concerning user requirements and expertise on the researchers’ and the information professionals’ side are needed and addressed by special interest groups such as the LIBER Digital Humanities & Digital Cultural Heritage Working Group (LIBER, n.d.). Therefore, besides scholars, information professionals working in the field of research libraries and archives (within PARTHENOS Training mainly referred to as “cultural heritage practitioners”) were identified as a highly relevant audience group of PARTHENOS Training.
Due to limited financial and human resources within the project, the fourth guiding principle for PARTHENOS Training is to concentrate on asynchronously available materials for self-learning, ‘train the trainers’ approaches, and partnering with other projects and initiatives in order to achieve maximal impact. Thus, PARTHENOS focused during the first phase of the project (year 1–2) on online materials to maximize its impact and developed a training portfolio consisting of several thematic modules (e.g. “Introduction to Research Infrastructures,” “Management Challenges in Research Infrastructures,” and “Collaborations within Research Infrastructures”) that can be accessed via the PARTHENOS Training Suite Portal.6 These modules consist of a variety of multimedia and multimodal training materials. Additionally, a few concerted face-to-face training interventions such as lectures and workshops were conducted.
2.3. The Founding Idea of the PARTHENOS Webinar Series
At the beginning of the second project phase (years 3–4), the PARTHENOS ‘Initial Training Plan’ was assessed and updated (see Oltersdorf et al., 2016; Spiecker et al., 2017). During the assessment concerns were raised regarding potential barriers to the uptake of the online materials and it was recommended to conduct more face-to-face sessions including hands-on activities,7 to focus more on the life cycle of engagement in research infrastructures, the uptake of the PARTHENOS Training modules in general, and to establish a stronger link between theoretical work and existing projects especially for novice audiences (see Spiecker et al., 2017, esp. p. 19–20, p. 28). Therefore, PARTHENOS decided to offer a one-week workshop at the European Summer University in Digital Humanities (ESU) 2018 as a mix of live webinars held by subject specialists accompanied by hands-on sessions led by two tutors on site (as a kind of blended learning approach). The innovative mix of webinars and live sessions at the ESU 2018 was intended to involve a great variety of subject specialists as trainers while acknowledging practical obstacles such as the lack of financial and time resources to bring all trainers to the workshop site by recognising that the virtual presence of most of the trainers would suffice. We also decided to run all webinars twice (in Spring 2018 and during the ESU 2018) and to record them. This would offer a possibility to refine the webinars based on the participants’ feedback and be a great opportunity to reach and engage a broader audience with PARTHENOS Training via publishing the recorded webinars.8 Thus, the “PARTHENOS eHumanities and eHeritage Webinar Series” was born.
3. Intellectual Context of the PARTHENOS Webinar Series (Design, Scope, Content)
3.1. Why are Webinars a Pertinent Training Approach?
As webinars were an addition to the already existing training portfolio of PARTHENOS, the author further investigated their specific factors, focussing especially on their educational potential and possible pitfalls (especially regarding the ‘live’ element), and their appropriateness in the context of elucidating digital humanities and digital cultural heritage methods and tools.
Currently, webinars are gaining increasing popularity in various educational contexts and are offered quite regularly as professional development activities by (European) research infrastructures and initiatives (e.g. by LIBER, RDA, Foster, and the Helmholtz Association).9 Webinars, as live seminars on the Internet (or virtual classrooms), seem to offer an interesting possibility to reach a geographically and otherwise diverse audience and to cater to the user need for face-to-face training while taking into account obstacles such as limited financial and human resources on the organisers’, trainers’, and users’ side to deliver and participate in training events by translating them into a virtual context. Thus webinars have the advantage of being economical, time efficient, and allowing an (almost) unlimited number of participants regardless of their location.
Even though webinars are gaining popularity in various educational contexts, it is difficult for the uninitiated (non-eLearning experts) to locate practical or theoretical information about conducting webinars in a research support context. To the author’s knowledge, there is scant literature available that offers practical, hands-on information about conducting webinars in a non-profit educational context. While being useful, the existing literature often has a more commercial background and requires adaptation in several areas (see Bett, 2013; Herrmann-Ruess & Ott, 2014). The short case study by Wyatt (2006) shares experiences from a webinar series aimed at Northern American librarians as online professional development and underlines the potential of webinars to reach geographically separated audiences that otherwise do not have the time or the money to participate in face-to-face events. Allan (2013, p. 82–83) mentions webinars as a useful instrument for library training and professional development; Kümper (2011, p. 20) and Kamenzin (2018) explore briefly the possibilities and potential of webinars for teaching (digital) medieval studies. Therefore, during the development phase, besides desk research, the author herself participated in several webinars and consulted webinar practitioners and e-Learning experts about practicalities (webinar systems, organisation, etc.) and the specific challenges of teaching online in a webinar setting.
As an online format, webinars require specific teaching skills of the trainers. In a webinar setting, the participants mainly see the presentation slides (and as in our case a small live stream/picture of the trainers) and can only actively interact with the trainers and other participants via the chat and by contributing to interactive polls (this was also the setting we opted for). Depending on the chosen technical environment and the group size, other interaction methods are also open to the participants, e.g. working together on a shared document, drawing collectively on the virtual whiteboard or using their audio and video. For participants, it is hard to follow lessons on a screen because there is less contact with the trainers, less social interaction, and more distraction. Also for the trainers, it is challenging not being able to see the participants’ reactions. In theory, of course, all participants could use their audio and video (at least for parts of the webinar), but in reality this can lead to technical problems (disruptive noises such as feedback), technical exclusion (if a participant has no microphone or webcam), and social exclusion (if the participant cannot use audio and video due to his learning environment or does not wish to be filmed). Also, it is almost impossible for the trainer to ‘register’ the facial expressions of a larger group based on minute webcam images.
An inadequately engaging webinar threatens not only the achievement of the learning objectives by the participants who are likely to leave the webinar prematurely and may not participate in any future versions, but also prevents the trainers from learning something from the participants. In order to achieve a high level of participant engagement, the teaching style needs to be adapted from a ‘traditional’ on-site training situation to the context of the virtual classroom (see Bett, 2013, p. 4–6). This requires additional teaching skills on the part of the webinar trainers that enable a bidirectional communicative situation that justifies a live event instead of simply recording a lecture without an audience (see Kerres, 2001, p. 292). There are different means available to make a webinar engaging and interactive, e.g. using live polls to ask the participants about their experiences or opinion related to different topics, rotation of different speakers, providing visually attractive slides, and active encouragement of the participants to contribute to the chat discussion (see Bett, 2013, p. 7–14).
Despite their advantages, webinars as a synchronous eLearning format are not uncontested. While on-site training events mostly ensure that the participants are in a suitable learning environment and have the time to learn and discuss uninterruptedly, learners participating from home or their workplace cannot always make sure that they have the time and technical requirements to participate at the moment set for the webinar. If webinars are conducted like lectures without follow-up immersion activities that enable participants to actively deal with the content taught, they mainly address the lowest level of Blooms taxonomy of educational objectives (see Gertler, 2013) and are therefore mainly considered as a suitable format to transport general knowledge, even though they can also be used for the mediation of more concrete skills if the trainers share their screen while demonstrating tasks, with the advantage that participants can directly ask questions while testing out what they just have seen (see Kamenzin, 2018). Additionally, the importance of interactive moments (so-called online-highlights) to prevent the dropout of participants by creating a group feeling and sense of contributing that enhances the virtual learning experience has been underlined (Herrmann-Ruess & Ott, 2014). It is argued that in order to activate learners more, ideally webinars should be enriched with additional exercises or be used in a flipped classroom setting (see Handke, 2015, p. 176; Kerres, 2001, p. 291–2), but have valuable potential for knowledge transfer on an introductory level, which justified testing their potential as addition to the existing PARTHENOS Training portfolio.
3.2. Syllabus Design
In the light of the context outlined above and further internal project discussions, the design of the initial PARTHENOS Webinar Series had to meet the following requirements:
- General message: Research infrastructures empower research(ers)! (=User-centric perspective: What’s in it for me?)
- Audience: Humanities and cultural heritage scholars and information professionals that are not yet eHumanities and eHeritage research infrastructure users and those who need more knowledge on how to engage with them and benefit from them to reap their benefits (beginners’ to intermediate level of knowledge).
- Learning objectives: 1) Participants will gain a nuanced overview of the role, value, and issues related to using eHumanities and eHeritage research infrastructures throughout the whole research process; 2) Participants will be able to identify research infrastructures and tools especially valuable for their research and data; 3) Participants know where to find more information about the respective topics, especially on the PARTHENOS Training Suite Portal.
- Provide a well-defined educational framework that would also be suitable for the envisaged one-week ESU 2018 PARTHENOS Workshop (1 week=5 webinars)
- The individual webinars must be consumable independently as we cannot expect that learners will be able to follow all of them if they are spread over several weeks.
- To broaden their relevance, all webinars will not be too PARTHENOS specific, but they will contain pointers to relevant achievements within PARTHENOS and its cluster partners.
Departing from these requirements and based on insights into relevant fields of knowledge and skills that are needed to enhance the awareness and knowledge of using research infrastructures to assist different phases of the research process that had already been gained within PARTHENOS (see Edmond et al., 2016; Spiecker et al., 2017), the author conducted additional exploratory desk research into comparable training activities and translated the findings into a draft syllabus.10
As the basis for the above mentioned educational framework, we opted for the research life cycle (see Figure 1) because its distinctive phases provided a clear user-centric lens and an inherent logical order. In the literature, diverse approaches to conceptualizing the research process in the form of a research life cycle can be found. They all have in common that they are attempts to visualise a rather complex activity and vary in their constituting phases and order (compare with various incarnations of the so-called research data life cycle). For the PARTHENOS webinars, we opted for a simplified research life cycle model with the following phases: developing research questions, planning of research, carrying out research, analysing of data, and publishing of results. We merged the two most technical phases of the research life cycle (carrying out research & analysing of data) because we considered them as being less suitable and relevant for a novice audience and added a general introductory webinar (see Table 1).
|Create Impact with your eHumanities and eHeritage Research|
|Juliane Stiller (Humboldt-University Berlin) and Klaus Thoden (Max Planck Institute for the History of Science, Berlin)|
|Research Life Cycle Phase “Publish Results”|
|On successful completion of the Webinar, the participants will be able to:
|How to work together successfully with eHumanities and eHeritage research infrastructures: The Devil is in the Detail|
|Marie Puren (Inria) and Klaus Illmayer (Austrian Academy of Sciences)|
|Research Life Cycle Phase “Plan Research Project”|
|On successful completion of the Webinar, the participants will be able to:
|eHumanities and eHeritage Research Infrastructures: Beyond Tools|
|Steven Krauwer (CLARIN) and Stefan Schmunk (State and University Library Göttingen)|
|No Research Life Cycle Phase, but General introduction|
|On successful completion of the Webinar, the participants will be able to:
|Make it Happen – Carrying out Research and Analysing Data|
|George Bruseker (Foundation for Research and Technology) and Carlo Meghini (Consiglio Nazionale delle Ricerche)|
|Research Life Cycle Phases “Carry out Research & Analyse Data”|
|On successful completion of the Webinar, the participants will be able to:
|Boost your eHumanities and eHeritage research with Research Infrastructures|
|Darja Fišer (University of Ljubljana) and Ulrike Wuttke (University of Applied Sciences Potsdam)|
|Research Life Cycle Phase “Develop research questions”|
|On successful completion of the Webinar, the participants will be able to:
4. The Initial PARTHENOS Webinar Series
The PARTHENOS Webinar Series syllabus was aimed at beginners, required no prerequisite knowledge, and was tailored to the needs and requirements of a user-centric perspective (‘What’s in it for me?’). The series consisted of five webinars, each with its own set of learning outcomes (see Table 1).11 Each webinar was delivered in a single one-hour session (see Table 2). Due to practicalities, the webinars were conducted in a different order from the order that the research life cycle would have dictated, but as dependencies between the webinars were minimised, each webinar could be followed independently.
|00:00–05:00||–||Greeting of participants and trainers||Moderator|
|–||Explanation of webinar room functions and rules (Housekeeping)|
|–||Introduction of PARTHENOS|
|–||Title of the webinar & Introduction of Trainers|
|05:00–35:00||–||Greeting of Participants||Trainers|
|–||Warm Up Question|
|–||“Setting of the scene” and/or “Hell Story” > Aims of the webinar|
|–||Lecture with interactive highlights (polls etc.)|
|–||Take home messages|
|35:00–40:00||–||Introduction of discussion (Questions & Answers)||Moderator|
|–||Announcements of next PARTHENOS webinar and relevant resources produced by PARTHENOS|
|40:00–55:00||–||Questions & Answers||Moderator, Trainers, Participants (via chat)|
|55:00–60:00||–||Announcement of feedback survey||Moderator, Trainers|
|–||Thank you and good bye|
The syllabus provided a lens through which a more nuanced understanding of the role of Digital Humanities and Digital Cultural Heritage research infrastructures for each phase of the research process could be obtained. The syllabus was also built around use cases with clear relevance for scholars and information professionals. Topics included: theoretical and practical reflections on the role of digital and analogue research infrastructures in Humanities and Cultural Heritage research; opportunities and challenges of eHumanities and eResearch; finding, working and contributing to digital research infrastructure collections; the role of standards; the FAIR principles and research data management; ontologies; digital tools and Virtual Research Environments (VREs), and; new publication and dissemination types.
4.2. The Individual Webinars
For each of the five webinars, a length of 60 minutes was foreseen. While there are valid arguments to run shorter webinars (see Hanke, 2018), 60–90 minutes is a common webinar length (see Bett, 2013, p. 7) that allows the webinar trainers to elaborate on the topic and seemed suitable for our target audience (beginners’ to intermediate level of knowledge).
Each webinar followed a similar outline that is exemplified in Table 2 (note that this outline does not correspond exactly to what happened in each webinar). As we expected some of our participants to be new to webinars as a teaching format and also unfamiliar with PARTHENOS, each webinar began with a short outline of the main functionalities of the virtual classroom and a short introduction of PARTHENOS. Also during each webinar what was to follow was situated within the wider context of the series, e.g. by pointing out the respective phase of the research life cycle that was specifically being addressed in the webinar. Throughout each webinar, the participants were actively encouraged to ask questions to the webinar trainers.
Regarding the presentation slides, a special feature of the PARTHENOS Webinar Series was a moderator’s slide set (including Housekeeping, Introduction to the Trainers, Discussion Time, etc.) that helped to brand the webinars and provided the general framing structure also for the trainers. The slide sets from the PARTHENOS Webinar Series, that are always a combination of the moderator’s slide set and the trainers’ slide set, are made available online.12 It also contributed to the general learning objective that each webinar pointed participants to relevant achievements within PARTHENOS and to additional resources, especially to modules of the PARTHENOS Training Suite, since the moderator always included one slide to announce relevant highlights from PARTHENOS before the discussion. This “commercial break” provided the trainers with time to catch their breath and gave the participants an opportunity to gather their thoughts for the discussion.
As PARTHENOS Training aims at producing materials for both trainers and self-learners, we developed a publication and archiving strategy that facilitates the independent uptake and re-use of the webinar materials as Open Educational Resources (OER). The original definition of OER from the UNESCO underlines the use of open licences (see UNESCO, n.d.). Later definitions added the importance of the absence of technical barriers (see Muuss-Merholz, 2018; Weitzmann, 2014). Therefore, the presentation slides were published with an open license (CC-BY), technically barrier-free (various formats), and findable (various publications channels, rich metadata). Due to the high user demand, we also made the webinar recordings available.
The publication strategy included:
- Variety of materials: presentation slides, webinar recordings, website texts
- Additional information: webinar descriptions and Wrap-Ups of the webinars
- Several formats (HTML, PPTX, PDF, Flash movie, MP4)
- Various publication channels: PARTHENOS Training Suite, PARTHENOS Webinars Community on Zenodo, HAL, YouTube, Adobe Connect
- Promotion via PARTHENOS website and PARTHENOS newsletter, social media etc.
The information on all available resources for each individual webinar is bundled in a dedicated Wrap-Up. The Wrap-Up serves as a ‘landing page’ that helps the learners to identify all available webinar resources at a glance and serves as documentation: it contains a summary of the webinar with links, screenshots of the polls, and a summary of the Question & Answer section. The Wrap-Ups and webinar descriptions are accessible as an independent module within the PARTHENOS Training Suite13 and can be used for self-study and integrated into training courses. We also have integrated thematically relevant webinar recordings into other PARTHENOS Training Suite modules.
5. Attendance and Feedback
Methods of evaluation can only provide insights into the success of any endeavour and invaluable data for reporting and future improvements if they are planned well in advance, especially as in some cases they have an influence on the practicalities of the webinars. For example, we decided to set up a registration with the online event platform Eventbrite14 and to collect information about the professional background and level of knowledge of the participants during the registration process to monitor the success of outreach endeavours and to gain insight into the composition of the webinar audience (target group analysis), despite the risk that being required to register would put off participants (therefore some webinar organisers work without registration). The anonymized registration data not only helped the trainers to tailor their webinar presentation according to the expected audience, but also indicated if more promotion was needed.
Across the initial PARTHENOS Webinar Series, a total of 161 individuals from the eHumanities and eHeritage community attended the five webinars (235 registrations), 127 of whom were researchers, 52 practitioners, 16 from the executive level, 12 developers and technicians, and 28 had chosen none of the aforementioned options to identify their professional background (see Figure 2). The geographical distribution of the webinar participants was very diverse. All in all, 27 countries were represented, with a peak of 90 participants from Germany, which can be explained by the fact that we used several German mailing lists to promote the webinars, especially to librarians (InetBib) and the German Digital Humanities community (DHd-mailing list). A significant amount of participants came also from France (15), the Netherlands (13), the United States (12), Greece (11), Italy (9), Ireland (8), Croatia (7), and Austria (7). Besides the United States and Croatia, PARTHENOS has project partners from all these countries, which also supported the outreach. Notably, the series reached beyond the European Union (Australia, Cameroon, Canada, Russia, and South Africa) which shows its potential for further iterations.
Regarding their familiarity with research infrastructures, 84 participants identified themselves as beginners, 96 as being at an intermediate level, and 55 as being at an advanced level (see Figure 3). As the webinars were mainly targeted at participants with a beginners’ to intermediate level of knowledge, the high number of participants with an advanced background might be explained by the interest the series created amongst project members of PARTHENOS itself and members of related research infrastructures.
5.2. Participants’ Feedback on the Individual Webinars (Online Survey)
We collected direct feedback on the individual webinars from the participants using Quamp,15 an online survey tool licenced by the University of Applied Sciences Potsdam that is compliant with German data protection regulations. It was anticipated that this data would help the organiser assess whether the webinars had been properly tailored to the participants’ background and level of knowledge and met the stated learning outcomes. Rebecca Sierig (at that time working for PARTHENOS based at the University of Leipzig) developed the questions for the online feedback survey together with the author of this paper. An individual survey was set up for each webinar (see Table 3).
|1. Tell us a bit more about your background:|
|[1.1.] Researcher (DH, CH)|
|[1.2.] Cultural Heritage Institution Practitioner (e.g. information specialist, content specialist)|
|[1.3.] Developer or technician|
|[1.4.] Executive Level: Managers and Policy Maker|
|[1.5.] None, of these… [free text]|
|2. Have you ever participated in a webinar before?|
|[2.2] Yes. Number of times participated (estimation): [Numerical field]|
|3. How familiar would you rate yourself with the topic of the webinar?|
scale: very familiar, more or less familiar, not familiar at all [one choice]
|Relevance of content|
|4. How relevant would you rate the topic for researchers in general?|
scale: very relevant, more or less relevant, irrelevant [one choice]
|5. How relevant would you rate the topic for your own research or work?|
scale: very relevant, more or less relevant, irrelevant [one choice]
|Structure and amount of information|
|6. The amount of input was|
scale: too high, appropriate, too low [one choice]
|7. The webinar was structured logically.|
scale: strongly agree, neutral, strongly disagree [one choice]
|8. Where there any technical problems during the webinar (e.g. no sound, no internet, no video)?|
|8.2. Yes, the following: [free text]|
|9. How could the webinar be improved (in terms of delivery, structure, content or other aspects)?|
|10. This feedback survey is anonymous. However, if you are interested in giving us direct feedback, you can tell us your e-mail-address here:|
The feedback survey was announced during each webinar with the link shown on a slide and posted into the participants’ chat, additionally, the link to the feedback survey was also sent to all registered participants in a follow-up e-mail (mostly together with announcing the availability of the presentation slides). Despite these efforts, the number of respondents declined significantly during the course of the PARTHENOS Webinar Series while the satisfaction level stayed equally high (based on live feedback received during the webinars, on Twitter, and the answers of those who responded). As we know that many participants attended more than one webinar, we presume that they got weary of answering the survey questions. We are considering some short form of live feedback during further webinars. Nevertheless, we were able to collect valuable feedback data during the exploratory run.
We were able to collect feedback from 65 participants on the five webinars of the initial series via the feedback survey. In the following, the answers are grouped for all webinars to indicate trends for the series as a whole. The most important results of the participants’ feedback survey for the trainers and the organisers of the series were the answers to the question concerning the adequacy of the input. We correlated the answers to this question with the answers to the questions concerning the familiarity of the participants with the respective webinar topic (see Figure 4). We see that participants with a beginners’ to intermediate level of knowledge generally considered the amount of input as being appropriate, but the participants with an advanced level of knowledge were less content.
The participants were also asked to use free text fields for individual feedback on the webinars. We asked participants to highlight any problems experienced during the webinar and to provide suggestions for improvement. The responses to the free text questions were grouped by theme to identify clusters. The following clusters could be identified from the answers to the question regarding problems:
- Connection or sound problems on the side of the presenter (n=6) (note: this was the webinar where we almost totally lost the connection with one of the webinar trainers)
- Connection or sound problems on the side of the participants (n=3)
- Adobe Connect doesn’t work on Linux (n=2)
- Problems with the live poll (n=2) (note: this was the webinar where a mistake was made in handling the live poll, the poll had to be repeated)
- Difficulties typing into the chat (n=1)
The responses to the question regarding improvements show a greater variation. The following clusters could be identified:
- General responses to the amount of input (too low or too high) (n=6)
- Requests to explain more details (n=5)
- Request to give more concrete examples (n=4)
- General positive feedback (n=3)
- Requests for preparatory materials, the slides, and additional materials (n=3)
- More activation of the participants (n=2)
- Requests for the trainers to speak slower (n=1)
5.3. Trainers’ Feedback on the Individual Webinars (Structured Interview)
We also developed a structured interview to gather verbal or written feedback from the trainers. This additional evaluation method provided us with valuable insights from the trainers’ perspective for future versions. We asked for feedback on the preparation phase, the execution, and any suggestions for the future (see Table 4).
|Tell us a bit more about your background as a teacher/previous teaching experiences/formats/experiences with online formats (esp. webinars)?|
|The Webinar experience|
|How did you experience the preparation support for the webinar by the moderator?|
|How did you experience the preparation for the webinar personally? In how far was it different from preparing a live lecture?|
|How did you experience the webinar and the interaction with the participants in general?|
|What would you do differently?|
|Any general suggestions?|
|What are your ideas for an online representation of the webinar (e.g. where to upload it, include quiz material)? Additional features? Related to ESU workshop (hands-on)?|
|After this experience would you be happy to be a webinar trainer again (not specifically for PARTHENOS)?|
From the trainers’ responses, the following trends could be identified even from the small sample size:
- The support by the moderator as a guide through the process (incl. educational and technological aspects) and the rehearsal aspect were highly valued, the latter would be even better with a test audience
- The preparation took more time than expected, but less time was needed for travelling
- A webinar needs to be structured differently than a face-to-face lesson, trainers need to get acquainted with the different possibilities of interacting with the audience
- Interaction with an invisible audience is challenging, one may consider granting speaking rights to the participants during the discussion, but the chat seems to make the participants egalitarian
- Areas for improvement: more interaction with the participants, less content in order to stick to timing and to have more room for discussion, more and attractive visuals (keep script separate in order to not (mis-)use the slides as memory aid/documentation)
- The possibility of playing videos was missing (note: this is in general possible, but we did not use it for the first series of webinars)
Taken together these three sets of data point to the following provisional findings and recommendations:
- The targeted promotion of the webinars is key to reach the intended audience groups; especially beginners are difficult to entice to spend time on a topic whose relevance for their own research is unclear
- The strong interest of information specialists in the webinars highlights that webinars are a compelling tool to conduct training for this audience group; this correlates with findings on the barriers to attendance experienced by this group in relation to other training offers in this area (e.g. a Round Table discussion organised by PARTHENOS during DH Benelux 2018)16
- It is extremely important to state the target group for each individual webinar clearly in the advertisement; advanced learners were slightly disappointed with our webinars tailored to a beginners’ or intermediate level of knowledge – they were simply the ‘wrong’ audience; however given the overall interest from more advanced learners, we consider targeting webinars at this group
- Support for the trainers by a moderator is invaluable, especially to point out teaching methods specific to webinars and if the trainers are novices to the format
- It is important to stick to timing to allow enough room for discussion and also advisable to experiment with shorter, more focused, webinar formats
- Live polls and other forms of activation were valued highly by participants and trainers and gave interesting insights into user requirements and contributed to building peer-to-peer networks
6. Lessons Learned
From our experience, we have divided the webinar life cycle of the PARTHENOS Webinar Series into three phases: 1) preparation phase, 2) execution phase, and 3) post-processing phase. This simple tripartite webinar life cycle is not based on a complex learning or teaching theory. It mirrors the everyday teaching praxis that applies to live and virtual face-to-face sessions alike and seemed useful to the author to order relevant observations. In the following, particular attention will be paid to these three essential phases of the webinar life cycle and the main insights for each phase are summarised:
6.1. Preparation Phase
6.1.1. Don’t Underestimate the Need for Promotion
Although webinars are more accessible than face-to-face events, at the same time they lack the incentive of travelling to an attractive location and being able to network. They also compete with the prospective participants’ regular tasks as well as with other webinars. We cooperated with the outreach teams at PARTHENOS, cluster partners, and external partners for the strategically wide promotion of the webinars. Information about the PARTHENOS Webinar Series in general and the individual webinars was posted on the PARTHENOS main site, announcements were sent out via mailing lists, and social media channels were deployed (e.g. Twitter, using the hashtag #PARTHENOSwebinar). We were able to monitor the success of the outreach activities via the registration tool and to send timely reminders about promotion to internal and external project partners where needed. All in all, for the webinar organiser the amount of time required for outreach and other preparation activities compares to that required to organise a small face-to-face event.
The first five PARTHENOS webinars were mainly aimed at communities of humanities researchers and cultural heritage practitioners (e.g. at libraries, archives, museums) with a beginner to intermediate level of knowledge about research infrastructures. Although they attracted a very satisfactory amount of registrations from these groups, we experienced difficulties in attracting our main target audience – beginners. We suspect that people belonging to this group are unlikely to be involved in networks that PARTHENOS and its cluster partners have already established links to (eHumanities and eHeritage infrastructures and research communities), do not know that this topic might be relevant for them, and may also be reluctant to participate in such a format. For future versions, we will concentrate more on community and language specific mailing lists that our target group presumably uses for information, choose more ‘speaking’ webinar titles (search engine optimization), and optimize the webinar description, e.g. by including additional information for those who are inexperienced with webinars (this information could, for example, be provided in the form of a short YouTube video).
6.1.2. Allow Overbooking
For the first five webinars registration was capped at around 50 participants, as this seemed to trainers and organizers a manageable group size. However, our experience was that many registrations were not converted into participants. It was difficult to predict how many of the registrations would actually translate into participants: sometimes only half of the registered participants showed up because they had registered out of curiosity, wanted to ensure that they would receive links to the webinar resources and recording or were prevented at the last minute from participating (also MOOCs seem to have a drop-out ratio of up to 60% in their very first phase, see Hochschulrektorenkonferenz, 2014, p. 17, see also note 7). For further webinars we would allow more registrations from the start.
6.1.3. Provide Guidance and Documentation for the Trainers
Based on the information gained during the desk research and the consultation with eLearning experts, the author drafted a ‘script’ for the webinars that served to guide all parties smoothly through the process as well as a document that contained background information for the prospective webinar trainers about practical and pedagogical aspects. Based on lessons learned during the initial PARTHENOS Webinar Series, these documents were updated regularly. It is intended to develop them independently from this case study into a “Webinar Toolkit.”
6.1.4. Share Responsibilities
During a live webinar, a lot is happening at the same time. It is almost impossible for (inexperienced) webinar trainers to present their lecture, to keep an eye on the participants’ chat and to interact with the participants, and to be responsible for the technical execution all at the same time. Therefore our webinars had a moderator whose role was to brief the trainers about the general educational framework, the specifics of online teaching, and special features of the virtual webinar room, in our case Adobe Connect, to take care of the overall organisation (including promotion and post-processing), but most importantly, to actively participate in each webinar to support the trainers’ interaction with the webinar participants and to take care of any technical issues that might arise. This ensured that the experts could fully concentrate on their role as trainers and feel confident in the virtual lecture space.
6.1.5. Choose your Webinar Tool Wisely
After evaluating several webinar tools, we chose Adobe Connect as webinar tool because it fulfilled all our requirements regarding desired functionality (interaction with participants; accommodating the anticipated number of participants; the possibility of downloading chats and the webinar recordings, and pricing). Adobe Connect is a replete webinar solution that is free of use for organisations that are a member of the DFN (German National Research and Education Network).17 However, this choice was not uncontested. We received feedback that some prospective participants were unable (or unwilling) to participate because Adobe Connect requires the installation of Flash which can be considered as a security problem and because it does not support Linux. At the moment we are looking into alternative systems.
6.1.6. Practice and have a Plan B
As in our case with inexperienced webinar trainers, it is important to practice in the virtual webinar room in advance. During rehearsals, the trainers can get acquainted with the virtual classroom and test out its different functionalities. This is also the moment to go through the webinar script and agree on roles and speaker changes (e.g. Who will cover which part of the webinar? Who will do which live poll? How will the participants’ questions be collected and handled? Do the trainers wish to be visible during their presentation? etc.). If the moderator’s and trainer’s view of the virtual classroom differs from the participants’ view (as in the case of Adobe Connect) it is advisable to prepare being able to monitor the delivery with a second screen. Also, a LAN-connection is strongly recommended for the trainers and the moderator, as the latter is, in general, more stable and robust than a Wi-Fi one (and in fact is also advisable for the participants). However, all these precautions cannot prevent unpredictable technical problems, such as a trainer suddenly losing connection to the virtual classroom for a while, which have to be dealt with creatively.
6.2. Execution Phase
6.2.1. The Importance of a Final Rehearsal
On the day of the webinar we met with the trainers in advance in the virtual classroom to go through the webinar script and for a final technical check (Do all cameras and microphones work? Are all presentations uploaded? Is the second screen to monitor the delivery set up?). During the webinar, we monitored the delivery with this second screen and stayed in contact with the trainers via a separate trainer chat channel within the virtual classroom software (Adobe Connect). The latter function was especially helpful to solve technical problems and to make quick adjustments.
6.2.2. Less is more
The most important lesson we learned is that “less is more.” This refers not only to less text on the slides but also to the webinar content in general. Most of our webinars were quite ‘packed’ with content and highly theoretical. Even though we had practised the webinars in advance and the moderator kept an eye on the clock during the webinars, the timing proved difficult. If the training lectures run over time, it meant cutting the discussion time, which is disappointing for the participants. To avoid this, for the future, we will adjust for even more discussion time and more condensed content.
Being new to the format, so far we have not made use of the possibility of showing short videos/screencasts or attempting to include hands-on activities but mainly relied on lectures livened up with interactive elements (polls, surveys, etc.). For future runs, we will consider experimenting with shorter webinar formats and to address more practical aspects by having the trainers share their screen to show something live, although it may depend on the number of participants (and the stability of the webinar system) to make the later a satisfying experience for the learners.
6.2.3. Make the Webinar Bidirectional
By using interactive elements such as live polling and encouraging the participants to be active in the chat discussion, we managed to create a bidirectional communication situation in which the webinar participants acted as a peer-to-peer network (when participants contributed additional information), and to use the webinars to learn about the participants’ experiences and issues related to certain topics and to investigate user requirements (see Figure 5), because once the participants’ questions get going, they are the most valuable contribution to the webinar!
6.3. Post-Processing Phase
6.3.1. Plan to Publish the Materials from the Outset
As we intended from the beginning to publish the webinar slides under a CC BY licence, we were able to tell this to the trainers in advance and draw their attention to the importance of applying correct licences to their materials. We also announced to the participants beforehand that the webinars would be filmed and made available online so that they could choose their username (which appears in the chat) accordingly. We developed a detailed publishing strategy for the webinar materials as OER with a module on the PARTHENOS Training Suite as their main entry point (see above) and promoted their availability to the webinar participants (via a follow-up email) and in general via the PARTHENOS social media channels and the project’s website.
7. Next Steps
The exploratory run of the PARTHENOS Webinar Series was a success for PARTHENOS Training. All webinars attracted a significant amount of participants (see Drenth & Wuttke, 2018). Also, the general feedback on the webinar content by the participants was positive and encouraging with minor points for improvement as outlined above. This success, measured by the number of participants and their positive feedback, is an indicator that the PARTHENOS webinars are filling a gap. Theoretical considerations are suitable webinar topics, but to convey the more practical aspects of training one needs to employ options for screen sharing or screencasts, whereby the results may depend on participant group size and the stability of the webinar system. Therefore, we are currently exploring the possibility of running further webinars by actively soliciting the PARTHENOS partners to suggest topics and by investigating user needs for webinar topics during live (face-to-face) training events and via social media.
We will also be actively fostering the re-use and uptake of this format not only via a better integration of the existing webinar materials into the PARTHENOS Training Suite but also by aligning future webinars with upcoming new (sub-)modules and PARTHENOS work package and cluster partner activities and products.
Last but not least, during the presentation of the PARTHENOS Webinar Series at DH Benelux 2018 at Amsterdam (see Wuttke, Neuroth, & Sierig, 2018) and during an internal project discussion of the PARTHENOS Webinar Series at the PARTHENOS General Assembly at Prato in June 2018, it was suggested that we should facilitate the uptake of the webinar content by non-English speakers by adding English subtitles (transcript of the English presentation) and/or by adding subtitles in different languages. Although this would be a great addition and contribute to their accessibility and contribute to language diversity, it also requires a huge amount of additional human effort on the part of the PARTHENOS Training team and therefore needs to be weighed against other upcoming tasks. We are considering the option of outsourcing or crowdsourcing this task to interested parties within PARTHENOS and beyond (and indeed already have an interested party for another PARTHENOS webinar that includes the offer to provide localizations).
8. Conclusion and Outlook
This case study has described how the initial PARTHENOS Webinar Series was conceived and delivered. It has presented the results of the analysis of data collection exercises that were conducted as part of the overall delivery strategy and also briefly considered potential future developments. The PARTHENOS Webinar Series’ success has confirmed that there is a demand amongst eHumanities and eHeritage researchers and practitioners for this format and the infrastructural topics that were covered. The webinars have proven to be a valuable addition to the existing training portfolio of PARTHENOS, to have high impact in directing attention to the existing PARTHENOS Training Suite elements, as well as to increase interest in PARTHENOS and its aims and results in general, which shows the potential of concerted pedagogical activities for overall project impact. As training is vital to achieving world-class research in the fast-moving field of eHumanities and eHeritage for researchers and practitioners alike, the PARTHENOS Webinar Series can therefore be considered a timely intervention in underpinning the role of research infrastructures in supporting the research life cycle.
Based on these insights from the PARTHENOS Webinar Series, the author would like to argue that webinars are a valuable addition to the training portfolio of (digital) research infrastructures, for example as a vehicle to deliver introductory training to various audiences and to attract them to engage with more in-depth learning resources and training interventions. To reap their full potential and to enhance their learning effect, the webinar content could be enriched by additional (digital) exercises and suggestions for further reading/additional learning resources that can be used during face-to-face training events or by self-learners; in the end they should comprise a full learning unit: lecture plus materials for an immersion phase (worksheets, assignments, sample solutions) (see Handke, 2015, p. 176).
As webinars seem to offer a cost-effective solution to reach a geographically and otherwise diverse audience and are currently trending amongst research infrastructures, it seems timely to discuss their greater potential amongst existing and prospective webinar providers for infrastructure training in general. The author would like to argue that an increased professionalization is required to produce educationally innovative material that produces satisfactory impact and results, meaning that the development of this kind of material should be tackled by subject experts in cooperation and collaboration with eLearning specialists. Often subject specialists are left to deal with this task alone due to natural project resource constraints. Such webinars could also gain from a further professionalization within research infrastructures based on a better theoretical pedagogical grounding garnered from the eLearning community, as well as from a concerted exchange of experiences and an ongoing analysis of their effectiveness between research infrastructure and other webinar providers. If they exceed digital ex-cathedra teaching, they can be utilized to gain insights in user requirements and new developments and also possess considerable potential for building peer-to-peer networks and community creation, thus contributing to the sustainability and life cycle of engagement with research infrastructures in general on the part of users.
There are many paths to travel on the road to professionalization, possibilities suggested here include organizing concerted mechanisms to share knowledge and experiences amongst webinar organizers and trainers, considering webinar skills as part of the professional development of research infrastructure staff involved in training, devising relevant training interventions and materials (e.g. a “Webinar Toolkit”), designing adequate mechanisms to measure the pedagogical dimensions of webinars (evaluation, quality assurance), and to promote the greater inclusion of education professionals in this discussion. Last but not least, we hope to inspire others to take up this format and share their experiences.