View source for Journal:University-level practical activities in bioinformatics benefit voluntary groups of pupils in the last 2 years of school

{{Infobox journal article
|name         = 
|image        = 
|alt          = <!-- Alternative text for images -->
|caption      = 
|title_full   = University-level practical activities in bioinformatics benefit voluntary groups of pupils in the last 2 years of school
|journal      = ''International Journal of STEM Education''
|authors      = Barker, Daniel; Alderson, Rosanna G.; McDonagh, James L.; Plaisier, Heleen;<br />Comrie, Muriel M.; Duncan, Leigh; Muirhead, Gavin T.P.; Sweeney, Stuart D.
|affiliations = University of St. Andrews, University of Manchester, Kilgraston School, Forfar Academy, Portlethen Academy
|contact      = Email: db60@st-andrews.ac.uk
|editors      = 
|pub_year     = 2015
|vol_iss      = '''2'''
|pages        = 17
|doi          = [http://doi.org/10.1186/s40594-015-0030-z 10.1186/s40594-015-0030-z]
|issn         = 2196-7822
|license      = [http://creativecommons.org/licenses/by/4.0/ Creative Commons Attribution 4.0 International]
|website      = [http://www.stemeducationjournal.com/content/2/1/17 http://www.stemeducationjournal.com/content/2/1/17]
|download     = [http://www.stemeducationjournal.com/content/pdf/s40594-015-0030-z.pdf http://www.stemeducationjournal.com/content/pdf/s40594-015-0030-z.pdf] (PDF)
}}

==Abstract==
'''Background:''' [[Bioinformatics]] — the use of computers in biology — is of major and increasing importance to biological sciences and medicine. We conducted a preliminary investigation of the value of bringing practical, university-level bioinformatics education to the school level. We conducted voluntary activities for pupils at two schools in Scotland (years S5 and S6; pupils aged 15–17). We used material originally developed for an optional final-year undergraduate module and now incorporated into 4273''π'', a resource for teaching and learning bioinformatics on the low-cost Raspberry Pi computer.

'''Results:''' Pupils’ feedback forms suggested our activities were beneficial. During the course of the activity, they provide strong evidence of increase in the following: pupils’ perception of the value of computers within biology; their knowledge of the Linux operating system and the Raspberry Pi; their willingness to use computers rather than phones or tablets; their ability to program a computer and their ability to analyse DNA sequences with a computer. We found no strong evidence of negative effects.

'''Conclusions:''' Our preliminary study supports the feasibility of bringing university-level, practical bioinformatics activities to school pupils.

'''Keywords:''' Bioinformatics; Computational biology; Secondary school; Raspberry Pi; Open access teaching material; Case study

==Findings==
===Introduction===
Progress in Science, Technology, Engineering, Mathematics and Medicine (STEMM) subjects is increasingly dominated by computational analyses. In biological sciences, for example, the exceptional pace of recent advances in technology for DNA and genome sequencing has created a demand for computationally able researchers, to analyse the large amounts of data produced. A field specialising in application of computation to biological problems has emerged, known as bioinformatics. The development of bioinformatics is discussed by Hogeweg<ref name="HogwegTheRoo">{{cite journal |title=The roots of bioinformatics in theoretical biology |journal=PLOS Computational Biology |author=Hogeweg, P. |volume=7 |issue=3 |pages=e1002021 |year=2011 |doi=10.1371/journal.pcbi.1002021 |pmid=21483479 |pmc=PMC3068925}}</ref>, and university-level bioinformatics education has been reviewed by Magana et al.<ref name="MaganaASur">{{cite journal |title=A survey of scholarly literature describing the field of bioinformatics education and bioinformatics educational research |journal=CBE – Life Sciences Education |author=Magana, A.J.; Taleyarkhan, M.; Alvarado, D.R.; Kane, M.; Springer, J.; Clase, K. |volume=13 |issue=4 |pages=607–23 |year=2014 |doi=10.1187/cbe.13-10-0193 |pmid=25452484 |pmc=PMC4255348}}</ref>

DNA sequences and related data are available at low cost (for new sequencing work) or free in online databases such as GenBank (Benson et al.<ref name="BensonGen">{{cite journal |title=GenBank |journal=Nucleic Acids Research |author=Benson, D.A.; Clark, K.; Karsch-Mizrachi, I.; Lipman, D.J.; Ostell, J.; Sayers, E.W. |volume=43 |issue=Database issue |pages=D30–D35 |year=2015 |doi=10.1093/nar/gku1216 |pmc=PMC4383990}}</ref>), Ensembl (Cunningham et al.<ref name="CunninghamEns">{{cite journal |title=Ensembl 2015 |journal=Nucleic Acids Research |author=Cunningham, F.; Amode, M.R.; Barrell, D., et al. |volume=43 |issue=Database issue |pages=D662-D669 |year=2015 |doi=10.1093/nar/gku1010}}</ref>) and hundreds of others (Galperin et al.<ref name="GalperinThe2015">{{cite journal |title=The 2015 ''Nucleic Acids Research'' Database Issue and Molecular Biology Database Collection |journal=Nucleic Acids Research |author=Galperin, M.Y.; Rigden, D.J.; Fernández-Suárez, X.M. |volume=43 |issue=Database issue |pages=D1-D5 |year=2015 |doi=10.1093/nar/gku1241 |pmid=25593347 |pmc=PMC4383995}}</ref>). Software for bioinformatics research is usually free, for example the very widely used sequence database search software, BLAST (Altschul et al.<ref name="AltschulGap">{{cite journal |title=Gapped BLAST and PSI-BLAST: a new generation of protein database search programs |journal=Nucleic Acids Research |author=Altschul, S.F.; Madden, T.L.; Schäffer, A.A.; Zhang, J.; Zhang, Z.; Miller, W.; Lipman, D.J. |volume=25 |issue=17 |pages=3389-3402 |year=1997 |doi=10.1093/nar/25.17.3389 |pmid=9254694 |pmc=PMC146917}}</ref>). Free resources are also available for bioinformatics teaching and learning, for example 4273''π'' (Barker et al.<ref name="Barker4273n">{{cite journal |title=4273''π'': Bioinformatics education on low cost ARM hardware |journal=BMC Bioinformatics |author=Barker, D.; Ferrier, D.E.K.; Holland, P.W.H.; Mitchell, J.B.O.; Plaisier, H.; Ritchie, M.G.; Smart, S.D. |volume=14 |pages=243 |year=2013 |doi=10.1186/1471-2105-14-243 |pmid=23937194 |pmc=PMC3751261}}</ref>), Bioinformática na escola (Marques et al.<ref name="MarquesBio">{{cite journal |title=Bioinformatics Projects Supporting Life-Sciences Learning in High Schools |journal=PLOS Computational Biology |author=Marques, I.; Almeida, P.; Alves, R.; João Dias, M.; Godinho, A.; Pereira-Leal, J.B. |volume=10 |issue=1 |pages=e1003404 |year=2014 |doi=10.1371/journal.pcbi.1003404 |pmc=PMC3900377}}</ref>), GOBLET (Corpas et al.<ref name="CorpasTheGOB">{{cite journal |title=The GOBLET training portal: a global repository of bioinformatics training materials, courses and trainers |journal=Bioinformatics |author=Corpas, M.; Jimenez, R.C.; Bongcam-Rudloff, E.; et al. |volume=31 |issue=1 |pages=140–142 |year=2015 |doi=10.1093/bioinformatics/btu601 |pmid=25189782 |pmc=PMC4271145}}</ref>), [http://www.nbic.nl/nl/education/high-school-programmes/bioinformaticsschool Bioinformatics@school] and the [http://evoed.evolutionsociety.org EvoEd Digital Library]. These publicly available data, software and materials present excellent opportunities for relatively low-cost teaching. 

There has been a recent, encouraging increase in exposure of school pupils to bioinformatics (e.g. Gallagher et al.<ref name="GallagherAFirst">{{cite journal |title=A first attempt to bring computational biology into advanced high school biology classrooms |journal=PLOS Computational Biology |author=Gallagher, S.R.; Coon, W.; Donley, K.; Scott, A.; Goldberg, D.S. |volume=7 |issue=10 |pages=e1002244 |year=2011 |doi=10.1371/journal.pcbi.1002244 |pmid=22046118 |pmc=PMC3203055}}</ref>; Lewitter and Bourne<ref name="LewitterTeach">{{cite journal |title=Teaching bioinformatics at the secondary school level |journal=PLOS Computational Biology |author=Lewitter, F.; Bourne, P.E. |volume=7 |issue=10 |pages=e1002242 |year=2011 |doi=10.1371/journal.pcbi.1002242 |pmc=PMC3203059}}</ref>; McQueen et al.<ref name="McQueenDesign">{{cite journal |title=Design and implementation of a genomics field trip program aimed at secondary school students |journal=PLOS Computational Biology |author=McQueen, J.; Wright, J.J.; Fox, J.A. |volume=8 |issue=8 |pages=e1002636 |year=2012 |doi=10.1371/journal.pcbi.1002636 |pmid=22956895 |pmc=PMC3431290}}</ref>; Kovarik et al.<ref name="KovarikBio">{{cite journal |title=Bioinformatics education in high school: implications for promoting science, technology, engineering, and mathematics careers |journal=CBE – Life Sciences Education |author=Kovarik, D.N.; Patterson, D.G.; Cohen, C.; Sanders, E.A.; Peterson, K.A.; Porter, S.G.; Chowning, J.T. |volume=12 |issue=3 |pages=441–59 |year=2013 |doi=10.1187/cbe.12-11-0193 |pmid=24006393 |pmc=PMC3763012}}</ref>; Machluf and Yarden<ref name="MachlufInt">{{cite journal |title=Integrating bioinformatics into senior high school: design principles and implications |journal=Briefings in Bioinformatics |author=Machluf, Y.; Yarden, A. |volume=14 |issue=5 |pages=648-60 |year=2013 |doi=10.1093/bib/bbt030 |pmid=23665511}}</ref>; Wood and Gebhardt<ref name="WoodBio">{{cite journal |title=Bioinformatics goes to school – new avenues for teaching contemporary biology |journal=PLOS Computational Biology |author=Wood, L.; Gebhardt, P. |volume=9 |issue=6 |pages=e1003089 |year=2013 |doi=10.1371/journal.pcbi.1003089 |pmid=23785266 |pmc=PMC3681668}}</ref>; Marques et al.<ref name="MarquesBio" />; Toby and Pope<ref name="TobyBio">{{cite journal |title=Bioinformatics tools available for K-12 students to engage in research |journal=Aviation, Space, and Environmental Medicine |author=Toby, I.; Pope, A. |volume=85 |issue=4 |pages=484-485 |year=2014 |doi=10.3357/ASEM.3953.2014 |pmid=24754215}}</ref>). Genomics and associated topics have started to appear in many official school curricula, for example in Scotland (see “Discussion”, below), the Netherlands (College voor Examens, p. 17<ref name="CollegeBio">{{cite web |url=http://www.examenblad.nl/examenstof/syllabus-2016-biologie-vwo-nader/2016/vwo/f=/biologie_vwo_2016_def_voor_hervaststelling.pdf |format=PDF |title=Biologie VWO. Syllabus Centraal Examen 2016 |author=College voor Examens |pages=59 |year=April 2014 |accessdate=21 October 2015}}</ref>) and the USA (Wefer and Sheppard<ref name="WeferBio">{{cite journal |title=Bioinformatics in high school biology curricula: a study of state science standards |journal=CBE – Life Sciences Education |author=Wefer, S.H.; Sheppard, K. |volume=7 |issue=1 |pages=155–162 |year=2008 |doi=10.1187/cbe.07-05-0026 |pmc=PMC2262119}}</ref>). From a different angle, computer science is now a major part of the [https://www.gov.uk/government/publications/national-curriculum-in-england-computing-programmes-of-study primary school curriculum for England]. This is in line with a “back to basics” approach to computing currently emerging, as opposed to more traditional [[information]] and communications technology (ICT). In the UK, this change has been particularly associated with the low-cost [http://www.raspberrypi.org Raspberry Pi computer], which is suitable for educational projects in electronics and engineering as well as general use and has [http://www.wired.co.uk/news/archive/2015-02/18/raspberry-pi-5-million sold over 5 million units]. However, a practical link between computers and STEMM — which we will refer to as computational science, as opposed to computer science — still does not feature strongly on the UK school curriculum. DNA sequencing has a pervasive and increasing influence across traditionally disparate subject areas, including biochemistry, biomedical research, clinical medicine, evolutionary biology, ecology, neuroscience and anthropology. DNA sequencing is used to diagnose genetic and infectious diseases, discover drugs, characterise environments, monitor the progress of cancers, identify species and reveal evolutionary patterns. We consider increased amounts of practical bioinformatics at school to be a priority. 

Motivated by the increasing importance of bioinformatics to the life sciences and its appearance on school curricula, we conducted a preliminary investigation of the benefits of bringing university-level bioinformatics teaching material to voluntary groups of children in the last 2 years of school in Scotland (S5 and S6; pupils aged 15–17). The material was originally developed for an optional, final-year undergraduate module at the University of St Andrews, [https://www.st-andrews.ac.uk/coursecatalogue/ug/2015-2016 BL4273 Bioinformatics for Biologists]. To better match bioinformatics as it is actually used in research at universities, institutes and industry, the material uses the Linux operating system, in this case a variant of Rasbpian Linux running on low-cost Raspberry Pi hardware. This material has been released under an open access licence, as part of [http://4273pi.org 4273''π''] (Barker et al.<ref name="Barker4273n" />). Our proposition was that school pupils can benefit from practical, undergraduate-level bioinformatics teaching material. Compared to the undergraduates for whom this material was originally developed, school pupils are less experienced and knowledgeable about biology in general. However, their levels of practical bioinformatics experience are broadly similar: zero in the case of the school pupils, and approximately ten actual contact hours among undergraduates at the time of starting the module. 

Many of the skills developed in our activities, and 4273''π'' or bioinformatics in general, are generic skills in computational science. For example, although the programming language taught in the “INTRO” component — Perl — is particularly widely used in bioinformatics (e.g. Stajich et al.<ref name="StajichTheBio">{{cite journal |title=The BioPerl toolkit: Perl modules for the life sciences |journal=Genome Research |author=Stajich, J.E.; Block, D.; Boulez, K.; et al. |volume=12 |issue=10 |pages=1611–1618 |year=2002 |doi=10.1101/gr.361602 |pmid=12368254 |pmc=PMC187536}}</ref>; Stabenau et al.<ref name="StabenauTheEns">{{cite journal |title=The Ensembl core software libraries |journal=Genome Research |author=Stabenau, A.; McVicker, G.; Melsopp, C.; Proctor, G.; Clamp, M.; Birney, E. |volume=14 |issue=5 |pages=929–933 |year=2004 |doi=10.1101/gr.1857204 |pmid=15123588 |pmc=PMC479122}}</ref>), it is structurally similar to other programming languages widely used in science, including C, Fortran, Java, Python and R. Use of the command-line, emphasised in 4273''π'', is also essential in computational physics, computational chemistry and, indeed, computer science. Although computational chemistry is not yet part of the Higher qualification in Chemistry, several simulations are suggested by the Scottish Qualifications Authority.<ref name="ScottHigher">{{cite web |url=http://www.sqa.org.uk/files_ccc/CfE_CourseUnitSupportNotes_Higher_Sciences_Chemistry.pdf |format=PDF |title=Higher Chemistry Course Support Notes |author=Scottish Qualifications Authority |pages=98 |year=2015 |accessdate=20 October 2015}}</ref> Computational skills, as taught in 4273''π'', will be valuable to students taking chemistry, physics and other STEMM subjects at university.

Judged by pupil self-assessment forms, our preliminary trial was a success, though caution is required due to the small sample size. We will continue developing peer-reviewed bioinformatics material, targeted at school pupils and/or undergraduates, and applying it in practice. This will simultaneously lead to expansion of the 4273''π'' resource and the gathering of larger, more complex and conclusive educational data at a future date. 4273''π'' itself, and links to relevant social media groups, may be found at http://4273pi.org. 

===Methods===
Two activities were carried out, each using a voluntary group of seven pupils studying science from a single school in Scotland. One group was from Kilgraston, an independent girls’ school, and the other was from Forfar Academy, a comprehensive school. In the case of Kilgraston, five pupils were at S5 and two were at S6 level, and instruction and assistance were provided by D.B., M.M.C., G.T.P.M. and H.P. In the case of Forfar Academy, all pupils were at S5 level, of whom two where girls and five were boys, and instruction and assistance were provided by R.G.A., D.B., L.D., J.L.M. and S.D.S. Generally, university staff or PhD students provided detailed instruction on the bioinformatics activity, and school staff highlighted links to material already taught and the curriculum. With a combination of university staff or PhD students and school staff, students were guided through the practical material of two components (modules) of 4273''π'' Bioinformatics for Biologists. At Kilgraston, the event was held at the school, occupying an entire day on which no other classes were scheduled. With Forfar Academy, the event was held at the University of St Andrews, where students participated in an afternoon and evening session, primarily held in the same room used, at other times, by undergraduates on the BL4273 module. Refreshment breaks were included, using the school’s usual facilities (Kilgraston) or the [http://www.st-andrews.ac.uk/museum/bellpettigrew Bell Pettigrew Museum] (St Andrews). In total, the teaching time was approximately 4 h. 

Raspberry Pi Model B hardware was used, one per student (plus one connected to a projector for demonstration). Prior to the first event, at Kilgraston, tasks were selected from existing material in discussion between D.B. and H.P. (familiar with 4273''π'') and M.M.C. (familiar with the school curriculum). For both groups of pupils, the first task corresponded to the “INTRO” component of 4273''π'' Bioinformatics for Biologists, providing an introduction to the Raspberry Pi computer hardware, the Linux command-line, BLAST sequence similarity search software and Programming in the Perl language. The second task corresponded to the “DNA” component, involving an introduction to the FlyBase database (Dos Santos et al.<ref name="DosSantosIntro">{{cite journal |title=FlyBase: introduction of the Drosophila melanogaster Release 6 reference genome assembly and large-scale migration of genome annotations |journal=Nucleic Acids Research |author=Dos Santos, G.; Schroeder, A.J.; Goodman, J.L.; Strelets, V.B.; Crosby, M.A.; Thurmond, J.; Emmert, D.B.; Gelbart, W.M.; The FlyBase Consortium |volume=43 |issue=Database issue |pages=D690-7 |year=2015 |doi=10.1093/nar/gku1099 |pmid=25398896 |pmc=PMC4383921}}</ref>) and genome annotation with BLAST (Altschul et al.<ref name="AltschulGap" />), GeneWise (Birney et al.<ref name="BirneyGene">{{cite journal |title=GeneWise and Genomewise |journal=Genome Research |author=Birney, E.; Clamp, M.; Durbin, R. |volume=14 |issue=5 |pages=988–95 |year=2004 |doi=10.1101/gr.1865504 |pmid=15123596 |pmc=PMC479130}}</ref>) and SNAP (Korf<ref name="KorfGene">{{cite journal |title=Gene finding in novel genomes |journal=BMC Bioinformatics |author=Korf, I. |volume=14 |issue=5 |pages=59 |year=2004 |doi=10.1186/1471-2105-5-59 |pmid=15144565 |pmc=PMC421630}}</ref>). Hard-copy handouts were provided. The handouts for Kilgraston and Forfar Academy were identical in content apart from date, location of the event, staff details and location of files (~/kilgraston or ~/forfar_academy). For the record, the specific handouts used are available as Additional file 1 (Kilgraston) and Additional file 2 (Forfar Academy), but with names and contact details redacted. The latest, open access versions of these will be found in [http://4273pi.org 4273''π'']. 

Hard-copy, paired, “before” (prior to the use of the computers) and “after” questionnaires were used for pupils and school staff, involving questions on a 1–5 Likert scale for self-assessment of attitudes and free text (Table 1; Additional file 3). In preparing the questionnaires, for those questions on a Likert scale, the sequence of questions was randomised and the sense of each question (“1” corresponding to “good” on our subjective scale, vs “1” corresponding to “bad”) was randomised. The same sequence and sense were used for each questionnaire handed out; within each group (pupils or staff), the sequence of paired questions was the same “before” and “after”. Results of the questions on the Likert scale were summarised per question as a bar chart, and as a likelihood ratio sign test for evidence of systematic change over the course of the activity. We apply a likelihood approach to statistical inference (Birnbaum<ref name="BirnbaumOnThe">{{cite journal |title=On the Foundations of Statistical Inference |journal=Journal of the American Statistical Association |author=Birnbaum, A. |volume=57 |issue=298 |pages=269-306 |year=1962 |doi=10.1080/01621459.1962.10480660}}</ref>; Edwards<ref name="EdwardsLike">{{cite book |title=Likelihood |author=Edwards, A.W.F. |location=Baltimore, MD |publisher=Johns Hopkins University Press |pages=296 |year=1992 |isbn=9780801844430}}</ref>; Royall<ref name="RoyallStat">{{cite book |title=Statistical evidence: a likelihood paradigm |author=Royall, R. |location=Boca Raton, FL |publisher=Chapman and Hall/CRC |pages=191 |year=1997 |isbn=9780412044113}}</ref>; Barker<ref name="BarkerSee">{{cite journal |title=Seeing the wood for the trees: philosophical aspects of classical, Bayesian and likelihood approaches in statistical inference and some implications for phylogenetic analysis |journal=Biology & Philosophy |author=Barker, D. |volume=30 |issue=4 |pages=505–525 |year=2015 |doi=10.1007/s10539-014-9455-x}}</ref>). In common with other approaches to statistical inference, this provides no absolute threshold beyond which evidence is considered conclusive. By convention, we define “strong” evidence as a log (ln) likelihood ratio, Δℓ, of at least 2, or a likelihood ratio of at least 8 (Edwards, pp. 199–202<ref name="EdwardsLike" />; Royall<ref name="RoyallStat" />). Were Δℓ converted to a p value under the assumptions of a likelihood ratio test (Wilks<ref name="WilksTheLar">{{cite journal |title=The large-sample distribution of the likelihood ratio for testing composite hypotheses |journal=Annals of Mathematical Statistics |author=Wilks, S.S. |volume=9 |issue=1 |pages=60-62 |year=1938 |doi=10.1214/aoms/1177732360}}</ref>), then for one free parameter Δℓ ≥ 2 corresponds to p ≤ 0.046, approximately the traditional threshold for statistical significance prior to any correction for multiple testing (i.e. p < 0.05). Calculations were performed in R (R Development Core Team<ref name="RTheR">{{cite web |url=https://www.r-project.org/ |title=R: A language and environment for statistical computing |author=R Development Core Team |publisher=The R Foundation for Statistical Computing |date=2015 |accessdate=20 July 2015}}</ref>). 

{| 
 | STYLE="vertical-align:top;"|
{| class="wikitable" border="1" cellpadding="5" cellspacing="0" width="70%"
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" colspan="8"|'''Table 1.''' Analysis of before and after questions for pupils on a Likert scale (1 “strongly disagree” to 5 “strongly agree”). For each pair of responses for each question, for each pupil, the change from before to after was noted, if any, and was reduced to a binary variable, indicating increase or decrease on the Likert scale. The proportion of changes that were increases and the proportion that were decreases constitute our maximum likelihood (ML) estimates for the probability of increase and probability of decrease, conditional on a change occurring. Whether the majority of changes are in the direction of improvement is indicated, with the direction indicating improvement being a subjective judgement by the authors in the context of this study. Assuming a binomial distribution, for each question separately, the likelihood of the observed changes was calculated, firstly, assuming the ML estimates for probability of increase and decrease obtained from the data; and secondly, assuming an extrinsic hypothesis that the probability of increase and probability of decrease are equal (0.5). From these likelihoods, the likelihood ratio and its natural logarithm (Δℓ) were calculated. Rows in italics show strong evidence of overall change in a specific direction over the course of the activity (Δℓ ≥ 2). N = 12 pupils submitted both “before” and “after” questionnaires
 |-
  ! Question
  ! Increases
  ! Decreases
  ! Increases proportion
  ! Decreases proportion
  ! Most changes were improvements?
  ! Likelihood ratio
  ! Δℓ
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1. I will end up working in science.
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|No
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0
 |- 
  | style="background-color:white; padding-left:10px; padding-right:10px;"|2. I think computers are useful within biology.
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Yes
  | style="background-color:white; padding-left:10px; padding-right:10px;"|32
  | style="background-color:white; padding-left:10px; padding-right:10px;"|3.47
 |- 
  | style="background-color:white; padding-left:10px; padding-right:10px;"|3. I have heard of Linux.
  | style="background-color:white; padding-left:10px; padding-right:10px;"|10
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Yes
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1024
  | style="background-color:white; padding-left:10px; padding-right:10px;"|6.93
 |- 
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4. I do not expect to enjoy [after: did not enjoy] the activity today
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|3
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.57
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.43
  | style="background-color:white; padding-left:10px; padding-right:10px;"|No
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1.07
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.07
 |- 
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5. I know more about computers than most adults do.
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|2
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.67
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.33
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Yes
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1.40
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.34
 |- 
  | style="background-color:white; padding-left:10px; padding-right:10px;"|6. I know more about computers than my teachers do.
  | style="background-color:white; padding-left:10px; padding-right:10px;"|2
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Yes
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1.39
 |- 
  | style="background-color:white; padding-left:10px; padding-right:10px;"|7. I am not intending to go to university.
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0
  | style="background-color:white; padding-left:10px; padding-right:10px;"|n/a
  | style="background-color:white; padding-left:10px; padding-right:10px;"|n/a
  | style="background-color:white; padding-left:10px; padding-right:10px;"|n/a
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|8. I have not heard of the Raspberry Pi.
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|10
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.09
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.91
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Yes
  | style="background-color:white; padding-left:10px; padding-right:10px;"|71.78
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4.27
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|9. I think computers are useful within sciences other than biology.
  | style="background-color:white; padding-left:10px; padding-right:10px;"|2
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Yes
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1.39
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|10. I would rather not use a computer, I would prefer to use a phone or a tablet.
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0
  | style="background-color:white; padding-left:10px; padding-right:10px;"|3
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Yes
  | style="background-color:white; padding-left:10px; padding-right:10px;"|8
  | style="background-color:white; padding-left:10px; padding-right:10px;"|2.08
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|11. I cannot program a computer.
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|8
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.11
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.89
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Yes
  | style="background-color:white; padding-left:10px; padding-right:10px;"|22.17
  | style="background-color:white; padding-left:10px; padding-right:10px;"|3.10
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|12. I am not interested in biology.
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0
  | style="background-color:white; padding-left:10px; padding-right:10px;"|No
  | style="background-color:white; padding-left:10px; padding-right:10px;"|2
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.69
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|13. I do not enjoy using computers for fun.
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|2
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.33
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.67
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Yes
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1.19
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.17
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|14. I am good at using a computer to analyse DNA sequences.
  | style="background-color:white; padding-left:10px; padding-right:10px;"|8
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.89
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0.11
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Yes
  | style="background-color:white; padding-left:10px; padding-right:10px;"|22.17
  | style="background-color:white; padding-left:10px; padding-right:10px;"|3.10
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" colspan="8"|''n/a'' not available 
 |-
|}
|}

===Results===
Questionnaire responses, with paired before and after questionnaire answers, are available as Additional file 3 and summarised in Table 1. The spread of answers to pupil questionnaires, on the Likert scale, is presented in Fig. 1. Because individual pupils may apply different criteria for each specific question (i.e. categories are simultaneously pupil-specific and question-specific), we regard the separation into five categories as unsuitable for either a continuous approximation or ranking. However, the direction of change on the Likert scale is comparable throughout the data, on the weak assumption that each pupil applies his or her own criteria for a given question consistently both before and after. Apart from changes over the course of the activity, whether the weight of answers is in the “disagree” categories (1 and 2) vs the “agree” categories (4 and 5) may also have some general meaning. 

[[File:Fig1 Barker IntJourSTEMEd2015 2.jpg|778px]]
{{clear}}
{| 
 | STYLE="vertical-align:top;"|
{| border="0" cellpadding="5" cellspacing="0" width="600px"
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"| <blockquote>'''Figure 1.''' For the pupil questions on a Likert scale (Table 1), the summary of answers before the activities (white) and after (black). N = 14 (before), N = 12 (after). Dashed arrows indicate the direction of change considered to be “improvement”. Solid arrows, below these, indicate the observed direction of most changes. Brief mnemonics (in italics) summarise question content. For the full text of questions, see Table 1.</blockquote>
 |- 
|}
|}

To an extent, all our questions are expected, if anything, to improve as a result of the activities. For example, a significant part of the first component (“INTRO”) is devoted to programming. Hence, it would represent either disaster for the educational approach, or perhaps humour on the part of pupils, if pupils tended to agree more with “I cannot program a computer” after the event than before. (The question has some similarity to a “control” in a laboratory experiment). Fortunately, in line with the educational approach being sound, evidence of disastrous effects was weak or absent.

With Question 1, “I will end up working in science”, most pupils agree and there is no systematic change during the activity. This is as expected, because pupils with no interest in science would be unlikely to volunteer to take part. Question 2, “I think computers are useful within biology”, is more specifically related to the activity itself. Again, most pupils agree both before and after, but here we see strong evidence of improvement (log likelihood ratio, Δℓ = 3.47). The usefulness of computers in other sciences (Question 9, “I think computers are useful within sciences other than biology”) shows little change as a result of the activity. As part of the volunteers’ positive mentality, we also see that most pupils expected to enjoy the activity and, in practice, did so (Question 4, “I do not expect to enjoy [after: did not enjoy] the activity today”). For this question, there is a fairly high level of change over the course of the activity, with four pupils reporting changes on the Likert scale in the direction opposite to “improvement”, and three in the direction of “improvement” (Table 1). All pupils have a strong desire to continue studies at university, which was entirely unchanged by participating in the activity (Question 7, “I am not intending to go to university”). Both before and after, they have a strong interest in biology (Question 12, “I am not interested in biology”). One pupil reports a change in the direction opposite to improvement, whose implications are difficult to assess without a larger sample. All appear interested in computers (Question 13, “I do not enjoy using computers for fun”). 

Questions 3 (“I have heard of Linux”) and 8 (“I have not heard of the Raspberry Pi”) reveal that, before the activity, most pupils have not heard of the Linux operating system or the Raspberry Pi, though more have heard of the Raspberry Pi than Linux. Improvement over the course of the activity is almost guaranteed due to its content, and indeed there is strong evidence for this (Δℓ = 6.93 for Linux, Δℓ = 4.27 for the Raspberry Pi).

More complex and perhaps more educationally relevant (Question 11, “I cannot program a computer”), students show strong evidence of improvement from a position of not regarding themselves able to program a computer, to being positive or at least uncertain (Δℓ = 3.10). From a position of considering themselves generally poor at using a computer to analyse DNA sequences (Question 14, “I am good at using a computer to analyse DNA sequences”), there is strong evidence of improvement towards a more positive assessment of their own abilities (Δℓ = 3.10), although no pupil ended up strongly confident (Fig. 1). We speculate this would improve further with repeated activities.

Questions 5 (“I know more about computers than most adults do”) and 6 (“I know more about computers than my teachers do”) are logically correlated, and both show weak evidence of improvement as a result of the activities. We prefer to interpret this as evidence of increased computational confidence among the pupils; however, heightened perception of adult incompetence cannot be ruled out.

Question 10 (“I would rather not use a computer, I would rather use a phone or a tablet”) indicates a base-line level of openness to the educational method used. Our activity, in contrast to much computational activity by young people, is computer-based, using the Raspberry Pi. Pupils’ views are generally moderate both before and after, but with borderline strong evidence of a strengthening tendency to prefer a computer as a result of the activity (Δℓ = 2.08).

Pupil free-text answers to Q15 (“What was the best part of the activity today, and why”) included “getting the hands-on experience with computers, as it is not something we ever get to do at school”; “I liked the programming—making files and commands—because it was really interesting to see how computers actually work”; “The simpler programming stages as they were at an accomplishable level & well explained”; “Learning how computers are used in biology and getting to try it ourselves because it was interesting and a new experience.”; “The best part was going on flybase because I found it really interesting”. Free-text answers to Q16 (“What was the worst part of the activity today, and why?”) included “Typing everything during programme because it took ages and a tiny mistake or omission could cause problems.”; “The worst part was flybase as it was difficult to understand.”; “I enjoyed the entire day and there wasn’t one part I didn’t enjoy. It was really interesting.”; “typing in all of the commands to the LXterminal as it took a long time and it was easy to make mistakes.”; “I didn’t enjoy typing stuff into LxTerminal because I was awful at it”; “The DNA sequencing on the internet was confusing and poorly explained in the handbook”; and “I don’t think the instructions were very clear” (Additional file 3).

It seems the activities stretched pupils intellectually and practically, which was regarded both positively and negatively among the pupils. This is illustrated, for example, by opposite views on the Flybase database, and may also be reflected in changing views on the enjoyability of the activities (Question 4). Typing programs and using the command-line within the terminal was unpopular. However, this is a crucial part of computational science, which cannot be bypassed, and which remains difficult even for professional bioinformaticians. The practical handouts clearly did not suit all pupils. These attempt a practical balance between brevity and total coverage. In future, these could be improved by inclusion of diagrams and screenshots.

School staff free-text answers to the question “What was the best part of the activity today, and why?” included “The pupils being given the opportunity to contextualise their theory based work in class into a practical application using the software. They gained a great deal of practice in making the connection between proteins + amino acids being the result of DNA sequences.” And: “Watching pupils who had never written code program Pis. Seeing their excitement when the code worked.” Also: “The pupils were all fully engaged & gained a great deal from the seminar. I have also gained an insight into bioinformatics which will be vital for teaching the next part of the new higher on genomics” (Additional file 3).

===Discussion===
Our results were for a small sample and cannot represent the full range of voluntary groups one could draw from schools even in Scotland. But it is encouraging for the approach taken that almost all changes in pupil questionnaires, over the course of the activity, were in the direction that we regard as indicating improvement.

For all pupil questions showing strong evidence of change over the course of the activity (Δℓ ≥ 2), the tendency was in the direction of improvement. Also, free-text responses from pupils were generally positive, though with a suggestion that the handouts could be improved (“Results”, above). On this basis, we judge the preliminary investigation a success. The evidence strongly suggests that, in their own assessment, the pupils benefited from the bioinformatics activities undertaken. This complements other research, involving successful bioinformatics activities developed specifically for school level (e.g. Marques et al.<ref name="MarquesBio" />). 

Gallagher et al.<ref name="GallagherAFirst" />) also reported a successful experience bringing bioinformatics to high school level, but with three challenges. We did not experience any of these. Firstly, some of the students in their study questioned the relevance of computation to biology. Gallagher et al. propose that part of their difficulty was use of computer scientists to teach lessons. In our case, lessons were co-taught by university biologists or biologists and chemists, alongside school staff, lending support to this proposition. However, other factors may be relevant, for example the use of whole-class teaching by Gallagher et al., as opposed to our voluntary groups; and the passage of time since their study. Secondly, some of the students in the study by Gallagher et al. doubted the value of learning about algorithms. Since our material was practical, with little detail on algorithms, the objection would—a priori—be unlikely. Thirdly, Gallagher et al. found some students doubted the relevance of bioinformatics material if this was not covered in official tests. In our case, there is relevance to the syllabus (see “Relevance to the new Higher curriculum for Biology and Human Biology”, below), and this was highlighted during the activities by school staff. With a different audience but similar educational aim, Wood and Gebhardt<ref name="WoodBio" /> also emphasise the importance of relevance to the curriculum. They report successfully updating high school teachers on the topic of bioinformatics via hands-on experience, practical demonstrations and lectures. One of their considerations for those wishing to replicate their approach, “Work within the practical limitations of the classroom”, may be alleviated by hardware such as the Raspberry Pi (see “The Raspberry Pi—a flexible, general-purpose computer”, below). 

How far can we generalise from our results? Further work is warranted, for example using a broader range of teaching material; involvement of a larger number of voluntary groups from a wider range of schools, for a more varied sample and greater statistical power; use of less enthusiastic staff; a lower staff/student ratio; a whole-class activity, not only a voluntary class sample; validation of the assessment instrument; direct measurement of student knowledge acquisition, in addition to self-assessment; interviews with staff and students and field observations. Where there is a surfeit of material available among our university colleagues — for example, introductory teaching material on the widely used program BLAST (Altschul et al.<ref name="AltschulGap" />) — one can imagine educational experiments, allowing an evidence-based decision on which material is more successful, perhaps in different geographical regions, countries, educational systems or types of school or with pupils of different age, academic achievement levels, social background or gender. These are important topics for the future.

Bioinformatics has always been relatively “open” as a subject, with public deposition of DNA sequence data associated with research publications and a wealth of free software. Although the algorithms do not change rapidly, the precise details of the software do. For these two reasons—the tradition of openness within the field and the rapid obsolescence of specific details—we consider the subject particularly amenable to online-only, open access teaching material. This costs nothing to obtain; for those writing the material, it can be relatively easily updated in light of developments in software and data. The material we used was from one open access bioinformatics educational effort, 4273''π'' (Barker et al.<ref name="Barker4273n" />). We hope to expand this resource to cover a wider range of bioinformatics topics, whilst continuing to ensure that all material is tested in practice and peer-reviewed. 

===Relevance to the new higher curriculum for biology and human biology===
The new Scottish Qualifications Authority Curriculum for Excellence Higher Syllabus for both Higher Biology and Higher Human Biology includes DNA technologies. Part of the content focuses on the use of computer software in both Biology (Scottish Qualifications Authority, pp. 10, 16<ref name="ScottHigher2">{{cite web |url=http://www.sqa.org.uk/sqa/48674.html?subject=Biology |title=Higher Biology Course Support Notes |author=Scottish Qualifications Authority |year=2014 |accessdate=20 October 2015}}</ref>) and Human Biology (Scottish Qualifications Authority, pp. 14, 15, 56<ref name="ScottHigher3">{{cite web |url=http://www.sqa.org.uk/sqa/48674.html?subject=Human%20Biology |title=Higher Human Biology Course Support Notes |author=Scottish Qualifications Authority |year=2014 |accessdate=20 October 2015}}</ref>). The suggested learning activity “Use genome data to identify stop and start codons and known protein coding sequences” (Scottish Qualifications Authority<ref name="ScottHigher3" />) is directly addressed by the “DNA” component of 4273π, as used in the current study. Other components are also relevant, for example to phylogeny and comparative genomics in Higher Biology (Scottish Qualifications Authority, p. 16<ref name="ScottHigher2" />). In addition, the new Curriculum for Excellence Advanced Higher Biology has content related to protein structure (covered, to an extent, by the “ENZYME” component of 4273''π'') and scientific investigative approaches (Scottish Qualifications Authority<ref name="ScottHigher4">{{cite web |url=http://www.sqa.org.uk/sqa/48674.html?subject=Biology |title=Advanced Higher Biology Course/Unit Support Notes |author=Scottish Qualifications Authority |year=2015 |accessdate=20 October 2015}}</ref>).

Based on our experiences as outlined in this paper, and in agreement with Gallagher et al.<ref name="GallagherAFirst" />, workshops taught alongside university staff seem an efficient means of professional development for teachers, who may be presented with curriculum content which is new to them and outwith their experience. From the pupils’ point of view, having access to workshops and seminars run by university staff also gives some experience of the university experience, particularly when—as in the current case—the teaching material is precisely at the undergraduate level. 

===The Raspberry Pi — a flexible, general-purpose computer===
The use of Raspberry Pi computers allows for a range of new software and experiences for the pupils, without any major impact on school’s ICT staff and systems. By bringing in complete software plus hardware systems, there is no need for specific software to be installed on school computers (some of which may require existing security restrictions to be relaxed). As the first part of the activity was to assemble the Pi, for the activity held at Kilgraston, the only prerequisite for the school was to [http://4273pi.org/files/2015/10/hardware_4273pi.pdf check the school’s VGA monitors would work with the Pi] and check there were enough power sockets.

Pupils may be more confident using the Raspberry Pi, with the knowledge that if they “broke” the software set-up, instructors could just swap its SD card. This allows a risk-taking approach to learning, and the possibility to learn by ones mistakes in a way that is prevented in managed computer classrooms. (Despite this, in the two school activities in this study, and in 3 years of teaching BL4273 Bioinformatics for Biologists at the University of St Andrews, we have never had to reinitialise an SD card due to such an accident. We plan to incorporate greater software risks in future material). It also gives pupils experience of Linux, the operating system they may well use should they choose to further their career in computational science. 

===Conclusion===
In our current study, we have demonstrated that practical bioinformatics material initially developed for an optional, final-year undergraduate module can be used successfully with school pupils at sixth form level (S5 and S6 in Scotland). Covering practical bioinformatics at this level in all schools may be an attainable goal, with subsequent benefits to higher education and research, in both biology and other sciences. 

==Abbreviations==
'''BLAST:''' Basic Local Alignment Search Tool

'''GOBLET:''' Global Organisation for Bioinformatics Learning, Education and Training

'''ICT:''' information and communications technology

'''ML:''' maximum likelihood

'''SD:''' secure digital

'''STEMM:''' Science, Technology, Engineering, Mathematics and Medicine

'''UK:''' United Kingdom

'''USA:''' United States of America

==Competing interests==
The authors declare that they have no competing interests.

==Authors' contributions==
DB and HP conceived of the school activities. DB, RGA, JLM, HP, MMC, LD, GTPM and SDS carried out the school activities. DB analysed results with assistance from RGA and JLM. DB wrote the manuscript with input from RGA, JLM, HP and other authors. All authors read and approved the final manuscript.

==Additional files==
{| 
 | STYLE="vertical-align:top;"|
{| class="wikitable" border="1" cellpadding="5" cellspacing="0" width="500"
 |-
  ! Additional file 1. Teaching material and blank questionnaires used for the event with Kilgraston School
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Names and contact details have been redacted.

Format: ZIP; Size: 431KB [http://www.stemeducationjournal.com/content/supplementary/s40594-015-0030-z-s1.zip Download file]
 |- 
|}
{| class="wikitable" border="1" cellpadding="5" cellspacing="0" width="500"
 |-
  ! Additional file 2. Teaching material and blank questionnaires used for the event with Forfar Academy
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Names and contact details have been redacted.

Format: ZIP; Size: 478KB [http://www.stemeducationjournal.com/content/supplementary/s40594-015-0030-z-s2.zip Download file]
 |- 
|}
{| class="wikitable" border="1" cellpadding="5" cellspacing="0" width="500"
 |-
  ! Additional file 3. Data from questionnaires (LibreOffice Calc spreadsheet)
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|For details of contents, see the “Notes” worksheet.

Format: ODS; Size: 81KB [http://www.stemeducationjournal.com/content/supplementary/s40594-015-0030-z-s3.ods Download file]
 |- 
|}
|}

==Acknowledgements==
This work was supported in part by the Science and Technology Facilities Council under grant ST/M000435/1 to DB. We thank all participants in our activities for their help, and thank Ian McDonald for technical assistance.

==References==
{{Reflist|colwidth=30em}}

==Notes==
This presentation is faithful to the original, with only a few minor changes to presentation. In several cases citation information was missing and was added to make the reference more useful.

<!--Place all category tags here-->
[[Category:LIMSwiki journal articles (added in 2015)‎]]
[[Category:LIMSwiki journal articles (all)‎]]
[[Category:LIMSwiki journal articles on bioinformatics‎‎]]
[[Category:LIMSwiki journal articles on education]]