d.5.3 vals placements evaluation · their results - virtual alliances for learning society (vals)...

540054-LLP-L-2013-1-ES-ERASMUS-EKA

D.5.3 Report on virtual placements pi lots and eva luation of their resul ts

D.5.3 Report on virtual placements pilots and evaluation of their results

Authors: Francisco J. García-Peñalvo (University of Salamanca), David

Griffiths (University of Bolton), Juan Cruz-Benito (University of Salamanca), Edwin Veenendaal (Raycomm B.V.), Achilleas Achilleos

(University of Cyprus), Scott Wilson (University of Oxford), Georgia Kapitsaki (University of Cyprus)

Project Partners: 1. Universidad de Salamanca 2. Università di Udine 3. Opendirective Ltd 4. University of Oxford – OSS Watch 5. Raycom B.V. 6. University of Cyprus 7. University of Bolton 8. Mindshock S. L.



Contents 1. Introduction ........................................................................................................................... 3

2. Students’ opinion about Semester of Code before the enrolment in the placements ..................................................................................................................................... 4

3. Virtual Placements & Semester of Code evaluation ................................................... 13

3.1 Student’s placements evaluation .................................................................................. 13

3.1.1 Students’ placements evaluation results ............................................................ 15

3.1.2 Comments on students’ placements evaluation .............................................. 23

3.2 Mentor’s placements evaluation .................................................................................. 27

3.2.1 Mentors’ placements evaluation results ............................................................. 28

3.2.2 Comments on mentors’ placements evaluation .............................................. 43

3.3 Academic supervisors’ placements evaluation ......................................................... 46

3.3.1 Academic supervisors’ placements evaluation results .................................... 47

3.3.2 Comments on academic supervisors’ placements evaluation ....................... 63

4. Conclusions ......................................................................................................................... 66

5. Acknowledgements ............................................................................................................ 69

6. References ............................................................................................................................ 69



1. Introduction This deliverable explains the evaluations carried out regarding to the students’ intention to participate in the Semester of Code and VALS project [1-5] and later the evaluation of the placements performed during the Semester of Code [6-10]. The three main stakeholders involved performed these evaluations: students, mentors and academic supervisors.

Apart of the main content of the evaluations and its questionnaires, this deliverable comment the results gathered among the different participants involved in the Semester of Code, explaining the key points and responses. Finally, are presented some conclusions about the evaluation results, comparing the main issues gathered.

This document should be referenced as follows:

García-Peñalvo, F. J., Griffiths, D., Cruz-Benito, J., Veenendaal, E., Achilleos, A., Wilson, S. & Kapitsaki, G. (2015). Report on virtual placements pilots and evaluation of their results - Virtual Alliances for Learning Society (VALS) project & Semester of Code. Salamanca, Spain: Virtual Alliances for Learning Society Project Consortium. University of Salamanca, University of Bolton, Raycomm B.V., University of Cyprus, University of Oxford. doi: 10.6084/m9.figshare.3198691



2. Students’ opinion about Semester of Code before the enrolment in the placements

During the presentations related to the Semester of Code to students across all the Universities that participate in the programme and during the process of applying for virtual placements in the scope of the programme, were proposed to these students the completion of a survey to know the student’s opinion about the virtual placements offered and the Semester of Code, as well as how they were informed about the different possibilities offered by the programme and the tools available for students. The survey was developed using Google Forms platform, and it is available through the following link https://docs.google.com/forms/d/1OxO_sD8xZnIoY276VwdNEpIDcBiyTNgNwYC1lTZ4Fao/viewform The questions included in the survey were:

1. Were you informed of the Semester of Code initiative by members of your institution? a. Yes b. No

2. Did you visit the Semester of Code website? a. Yes b. No

3. Did you browse the list of available projects? a. Yes b. No

4. Did you ask for access to the platform? a. Yes b. No

5. Did you compose a proposal for any project? a. Yes b. No

6. (If applicable) Was a proposal you made accepted by a company? a. Yes b. No

7. (If applicable) Did you start work on an accepted project? a. Yes b. No

8. If NO, why? 9. (If applicable) How would you rate the communication with the company mentor?

a. Very poor b. Poor c. Regular d. Good e. Excellent

10. If you decided not to participate, what was (were) the reason(s)? a. Insufficient details on Semester of Code b. Lackluster presentation by members of your institution c. Lack of commitment from local advisors d. Concerns about projects' difficulty e. Concerns about communicating in English f. Concerns about distance work



g. Too little time for applying h. Inconvenient timeframe for project execution i. Lack of monetary compensation j. Not interested in doing a company placement k. Other

11. Do you have any suggestion for the Semester of Code organization to improve it? 50 students of those reached in the VALS events and in the application process for virtual placements answered the survey. The results of the survey are the following. Question 1: Were you informed of the Semester of Code initiative by members of your institution? This question was answered by all of students, 45 of them affirm that their institution informed them about the Semester of Code previously of the survey, other 5 persons were informed by people that do not belong to their institution.

Question 2: Did you visit the Semester of Code website? In this case, the 40% of the students (20 of the total) answered ‘no’ to the question, while other 60% (30 students) visited the website previously to the survey. This question led us to know an initial view of the students’ engagement with the overall process and its first steps.

90%

10%

Were you informed of the Semester of Code initiative by members of your institution?

Yes

No



Question 3: Did you browse the list of available projects? The number of affirmative responses continues decreasingly: in this case 21 students (43% of the sample) browsed the list of available project, while other 28 (57%) did not browse the list.

Question 4: Did you ask for access to the platform? Is at this point where the process lose an important part of the students initially interested. Only 7 students (14%) asked for a personal account necessary to participate in virtual placements application. The others 42 students (86%) did not asked to have an account (so, they did not continue the process to apply for virtual placements).

60%

40%

Did you visit the Semester of Code website?

Yes

No

43%

57%

Did you browse the list of available projects?

Yes

No



Question 5: Did you compose a proposal for any project? Two students (5% of the sample) responded affirmative to this question, while the other 42 students (95% of the student that answered this question) did not compose any. This descent is as strong as the previously described, because of the 7 students that asked for a personal account, only 2 (28% of them) finally composed an application for participate in one or more of the projects available for performing virtual placements.

Question 6: (If applicable) Was a proposal you made accepted by a company? In this case the students’ responses were a bit confusing. From the previous question, where 2 students declared they applied for virtual placements, in this case 7 students answered about if a company approved their proposal. Of those 7 students, 2 affirm they get a project application accepted by a company: 100% regarding the 2 students that apply for a project regarding the previous question, 29% of the responses about in this question.

14%

86%

Did you ask for access to the platform?

Yes

No

5%

95%

Did you compose a proposal for any project?

Yes

No



Question 7: (If applicable) Did you start work on an accepted project? Similarly to the previous question, there were more students answering this question than those that declared an application for a project available in the Semester of Code Virtual Placement System (2 students, question 5). Regarding strictly the responses, 5 students answered the question, 3 affirming that they did not work on an accepted project (60%) and 2 affirming they work on a project. If we slant the responses regarding those 2 that applied for a project, there were a 100% of people that applied for a project, get accepted to work on it and finally started to work on the project.

Question 8: (If applicable) How would you rate the communication with the company mentor? In this case, 3 students answered to the question, 2 of them (67%) declaring that the communication with the company mentor was ‘excellent’ and another 1 (33%) declaring that this communication was ‘poor’. In this case the number of responses should not be limited to those student that declared they applied for a project or those that got an account in the Virtual Placement System,

29%

71%

(If applicable) Was a proposal you made accepted by a company?

Yes

No

40%

60%

(If applicable) Did you start work on an accepted project?

Yes

No



due in the project’s description there are the contact details for the mentor responsible.

Question 9: If you decided not to participate, what was (were) the reason(s)? 44 students answered to the question, 79 responses were recorded (1,80 responses per student) in 20 different categories of response, 10 of them predefined and other 10 suggested by the students. The reasons indicated by the students for not participating in the Semester of Code were (ordered from most to least frequently chosen):

1. Too little time for applying (20 times chosen) 2. Inconvenient timeframe for project execution (12 times chosen) 3. Insufficient details on Semester of Code (10 times chosen) 4. Lack of monetary compensation (6 times chosen) 5. Concerns about communicating in English (5 times chosen) 6. Concerns about projects' difficulty (4 times chosen) 7. Not interested in doing a company placement (4 times chosen) 8. Concerns about distance work (3 times chosen) 9. Other: I don't have enough time (3 times chosen) 10. Lackluster presentation by members of your institution (2 times chosen) 11. Lack of commitment from local advisors (1 time chosen) 12. Other: Already had a project in mind (1 time chosen) 13. Other: Interested in a near future, not now (1 time chosen) 14. Other: Interested on doing it but not yet. I will ask in a future application

period (1 time chosen) 15. Other: Tied up with studies (1 time chosen) 16. Other: I didn't know (1 time chosen) 17. Other: I am not supposed to do a project yet (1 time chosen) 18. Other: I am still 1 year away from the degree project. I cannot apply (1 time

chosen) 19. Other: Waiting for a professor to help with a proposal (1 time chosen) 20. Other: I don't have clear ideas about that (1 time chosen)

67%

33%

(If applicable) How would you rate the communication with the

company mentor?

Excellent

Poor



So, the five most frequent reasons to not participate in the Semester of Code are “Too little time for applying”, “Inconvenient timeframe for project execution”, “Insufficient details on Semester of Code”, “Lack of monetary compensation”, “Concerns about communicating in English”.

Regarding this evaluation and the students’ opinion can be highlighted that among the 5 main reasons to not participate, there is 1 reason in clear conflict with the main

10

2 1

4 5

3

20

12

6

4

1

3

1 1 1 1 1 1 1 1

0

5

10

15

20

25

If you decided not to participate, what was (were) the reason(s)?



idea behind the Semester of Code: lack of monetary compensation. Despite of there are only 6 out of 44 students (which answered this question) check this issue as a reason, it is among the top 5 reasons to not participate and could be a problem to run and mainstream the Semester of Code. Regarding these 5 top reasons to not participate, there are other 3 related to the configuration of the Semester of Code, these reasons are:

• Too little time to applying (20 out of 44 possible participants answer this) • Inconvenient timeframe for project execution (12 out of 44) • Concerns about communicating in English (5 out of 44)

The reasons shown before can be a good indicator about what aspects can be enhanced in the Semester of Code process to attract better students to participate. The first two reasons are related to the timing proposed by the Semester of Code, possibly improving the timeframes provided to announce, read and apply for the projects these reasons can be less important to not participate. Regarding that, in the two lasts rounds of the Semester of Code, students had more time to apply for projects. Even that, in the last instance of the Semester of Code (and which is alive at the end of the project), there is no time restriction to participate: students can apply for projects when they want, they find out projects to solve and talking to the mentor responsible of the project, they set up the possible timeframes and other conditions before the application process. Regarding the third reason, the Semester of Code propose as the main language for the communication the English, but there is no restriction to communicate among the students, mentors and academic supervisors in other language (if they agree it). This reason is the least important of the 5 main reason (5 students out of 44), but partners involved in VALS and Semester of Code observed that in countries like Spain this issue is an important challenge for the students. Regarding to the other reason that students featured (“Insufficient details on Semester of Code”, checked by 10 students out of 44), it could be a problem of disseminating the project and the timeframes used by the Universities to present the initiative to the students. It has been observed that students need more than one presentation and some period to think about their course schedule and possibilities regarding the participation before finally decide to participate. Due this observation, it is possible that if the students are informed about Semester of Code with no enough time by their universities or VALS staff, they do not decide to participate because they are not convinced about their possibilities to get a project or availability to perform the virtual placement based on their course schedule. In general some of these reasons to not participate (3 at least) can be solved in an easy way. VALS staff and participant universities can disseminate the VALS & Semester of Code procedures previously and in more events. As well as, other solutions adopted in the Semester of Code like no restricting the schedule to perform the virtual placements can be a outstanding improvement, due the students can adapt their personal schedule or the virtual placement schedule to fit in their academic course schedule. The monetary compensation reason is hard to solve due the current set up of the Semester of Code. VALS staff can evaluate it in the future exploitation of the project, opening the possibility that companies/foundations pay an expense to the students.



The reason about communicating in English, despite is one of the top 5 reasons, researchers think that is only a local/personal issue, so they can not solve it directly; should be the students/educational system that need to work on improving the skills in foreign languages.



3. Virtual Placements & Semester of Code evaluation

After the virtual placements, the different stakeholders involved –students, mentors and academic supervisors– evaluated the virtual placements performed and the Semester of Code process itself. In the following subsections are commented the different questionnaires presented to the stakeholders, the conclusions of each one, and the overall conclusions.

3.1 Student’s placements evaluation First of all, in this subsection is presented the questionnaire completed by the students involved in the Semester of Code Virtual Placements. The questionnaire is available to be filled through the URL http://forms.semesterofcode.com/students and for download in http://doi.org/10.6084/m9.figshare.2953720

1. Your first name / family name* 2. Your age* 3. Your gender*

a. Male b. Female

4. Your University* a. Faculty of organizational sciences, University of Belgrade – Serbia b. Frederick University – Cyprus c. Open University of Cyprus – Cyprus d. Universitat Politècnica de Catalunya - BarcelonaTech (UPC) – Spain e. University of Bolton - United Kindgdom f. University of Cyprus – Cyprus g. University of León – Spain h. University of Nicosia – Cyprus i. University of Oxford - United Kingdom j. University of Salamanca – Spain k. University of Udine – Italy l. Università Ca’ Foscari Venezia – Italy m. Université Paris 8 – France

5. The name of the project you worked on* The name of the project as it appears on the Semester of Code Virtual Placement System (VPS)

6. Company/Foundation owner of the project you worked on* 7. Your tutor's name*

The name of your tutor from the project / company 8. Your academic supervisor's name*

The name of the academic supervisor from your University 9. Did you complete the placement?*

Did you complete your work on the assigned project (and its tasks)? a. Yes b. No

10. If you did not complete the assignment, what problems did you experience? 11. Starting date of your placement* 12. End date of your placement*



13. Please summarise the tasks you carried out and the issues you addressed during your placement*

14. Please summarise the learning goals that you achieved while working on your project during the placements*

15. What were the main problems that you encountered in carrying out your project? How did you solve them?*

16. Please rate your overall experience of your Semester of Code placement* Regarding your overall opinion about the whole experience (Scale 1 to 5)

17. Please rate your experience with your tutor during the Semester of Code* How good / bad was your collaboration with your mentor from the project? (Scale 1 to 5)

18. Please rate your experience with your academic supervisor during the Semester of Code* How good / bad was your work with your academic supervisor from your university? (Scale 1 to 5)

19. Please rate the relationship between tasks you carried out during your project and the knowledge and competencies you are acquiring in your University studies* (Scale 1 to 5)

20. Where are the results of your placement (code, etc) hosted?* Please add the links (Github, websites, etc.) where is available publicly the placement results

21. Did your experience on your placement help you in planning your future professional career?* (Scale 1 to 5)

22. I think that my placement experience will help me in getting a job in the near future* (Scale 1 to 5)

23. Would you recommend this foundation / company to other students who are thinking of doing a placement?*

a. Yes b. No

24. Did you get a job offer from the foundation / company that you did the project for?*

a. Yes b. No

25. Did the company/foundation suggest that you continued to collaborate with them as a volunteer on this or other development work?*

a. Yes b. No

26. Did any other company offer you work or collaboration in relation to your experience of the Semester of Code?*

a. Yes b. No

27. Did your placement in the Semester of Code give you a positive feeling about collaborating in FLOSS (Free Libre Open Source Software) projects and FLOSS Community?*

a. Yes b. No

28. How easy did you find it to get involved in the programme (Semester of Code)?* (Scale 1 to 5)

29. What would you suggest to improve the Semester of Code process?* 30. What problems did you encounter?*

Regarding the Semester of Code process 31. Did you enjoy taking part?*

a. Yes



b. No 32. Would you take part again?*

a. Yes b. No

33. Did you learn anything from being part of this programme?* Apart of the learning and competences you acquired from your University studies

34. If you took part again what would you do differently?*

3.1.1 Students’ placements evaluation results There is available a spreadsheet with the results of this questionnaire available in https://docs.google.com/spreadsheets/d/1Tt4y5qJvSaEbVFnI18zj51oBUvr83EM9YJwRsRkgYW4/edit?usp=sharing (all data available except names and other personal information). Some of the results are available also in a visual way in https://juancb.typeform.com/report/DzQC3Y/kJXA 12 students out of 17 that performed virtual placements completed this questionnaire; the results are presented below (excluding all data that can identify individual people):

• The average age of the students was approximately 23 years old.

The students that did not complete the placement, explain it providing the following reasons (to preserve the veracity of the content, all comments made by students, mentors or academic supervisors are presented in the same way they wrote them, in a sic way):

1. “The time and communication between the tutor and I”



2. “My major problem it's the extension of the assignment and the lack of knowledge about it. It was my first time doing academic practice so I don't know how to work. Also the academic supervisor doesn't give me enough advise.”

3. “It's not completed but in progress, due to personal issues the development of the assignment was delayed for several months.”

4. “It was difficult for me at that time to learn and implement React.js” 5. “I had no time to develop it”

About the tasks carried out during the placements, students stated:

• “I had to implement a backened (Java RESTful web services) for managing the subtitles and a front end (web site) for interacting with the editor.”

• “I improved the html parser solving some bugs, I added html5 support, I added some extra features like XML parsing”

• “Functionalities implemented: -Mutable sound system -Sound playback manager -Dynamic music system (sound can change depending on a state with configurable transitions) -Efficient convolution system. -Binaural sound.”

• “Doing a basic WebGL implementation, since there were more devs working on the 2D canvas API.”

• “First I had to put a somewhat big effort to understand the core of the Inkscape codebase, which was not easy due to lack of documentation, legacy architecture (mixed C++ and glib) and at first the lack of good development environment that helps in discovering what code calls what methods (until I got the CMake build working and I could use an IDE). Once I got the environment and got to know how to modify Inkscape render code it was time to add the path offsetting features. Adding new SVG properties to both the core architecture and the user interface was quite easy, but rendering offsetted curves turned out to be really problematic. Supposedly I already had a function that calculated path offsets, but it turned out to be extremely buggy. I spent most of my remaining time debugging it and trying to learn how it was supposed to work. I also read a lot of Math papers about Bézier curves. Turns out approximating offsets of them is a very hard problem. In the end I got to fix some bugs in the algorithm and got the stroke-alignment property to work with a limited number of cases and sent a set of test cases that the current algorithm can't handle and their expected output.”

• “I created interfaces for: o Web page main page o Login Dialog o sign Up dialog o Administrator Panel o Group screen interface o User Home page (Appears when user signs in) o Edit User Profile I wrote AngularJS code for the following functionalities: o Login o Sign up o Group navigation”

• “My main task was fixing rpm packages in the new open mandriva distribution”

• “Read ISO standards documents by my own and try to extract some parts from this”

• “Development and main testing of the APP.” • “I made a simple web interface for a social network” • “Build an application about the colour of some t-shirts.”



About the learning goals achieved, the students explained that: • “Learned how RESTful web services work and how to use them combined

with a front end. Improved my JavaScript skills” • “Learned HTML5,XML, java, working under pressure” • “-Working on an already functioning product, integrating new functions on an

existing architecture -A lot of learning about DSP and how to make it efficiently -Experience in modern C++ (C++11) which I lacked previously”

• “Learning about low level graphics programming, about how browsers work, and learn more Rust.”

• “I learned quite a bit of geometry, but not enough to fully understand the papers. I also learned a bit about glib.”

• “While working on my project I have learned the following programming languages: o AngularJS o AJAX o JQuery o Html5 o RESTful Web Services”

• “I learnt a lot about open source projects, also I learnt about OS distributions and rpm functionality”

• “Know something more about the odf and oxml standards” • “Basically I learned how to develop web APP's (something that for unknown

reasons is not done at my university) and general web developing.” • “Java and Javascript”



Where are the results of the your placement (code, etc.) hosted?

1. https://bitbucket.org/paziz001/online-subtitle-editor 2. http://htmlcleaner.sourceforge.net/release.php 3. Forked repository: https://github.com/DethonUSAL/audaspace Parent

repository: https://github.com/neXyon/audaspace 4. https://github.com/servo and https://github.com/ecoal95/rust-offscreen-

rendering-context 5. https://code.launchpad.net/~ntrrgc/inkscape/inkscape 6. https://github.com/akoumi01/epl363-project 7. “The results of my placement are in some of the rpm packets of open

mandriva distribution, I can´t put that code here.” 8. “They will be uploaded to Github once I finish the overall project and add the

pertinent comments and "how it works" stuff.” 9. https://github.com/xeny/webinterface-RS



What would you suggest to improve the Semester of Code process? • “Adding more projects that relate to university studies” • “Some more visibility in the university's web would be nice. Other than that I

don't remember seeing anywhere in the VALS web that I could negotiate the project start date. Fortunately, my academic supervisor told it to me.”

• “Improve visibility, at least in my University. Improving the project search in the VPS should help too.”

• “I still don't know what's the purpose of the academic tutor.” • “It would be nice if companies provide a more detail description of the

projects specifications” • “Maybe some more information about the projects”

What problems did you encounter?

• “I was never very aware of the process, but just got told to "start working and then we will reach you"”

• “I had limited time to implement the Project” • “My tutor can't help me during the summer”

Did you learn anything from being part of this programme?

• “Experienced what is like working on a project remotely.” • “New languages and working in real projects” • “How open source development Works” • “Totally, I learned about how FOSS development works, and I deeply enjoy

being part of the Servo community.” • “I learned how to approximate Bézier curves as polygons. That was not

enough for my project as the SVG output would have too many points and would not look good if zoomed.”

• “I have corporated with many people from all over the world and acted as a member of the group”

• “I learnt a lot of things, things that a student can't learn at college” • “A bit of project managing and a lot about the self learning process.”



If you took part again, what would you do differently? • “I would do some more technical research about the project that I am

interested in working on.” • “Participate in better project” • “Probably I'd try to organise myself better, since I had some time issues.” • “Not picking a project asking to implement an incomplete specification.” • “Maybe I will try to do a better time schedule for the project, taking into

account possible personal problems.” • I” would devote myself more to the project and try my best to do all tasks”

3.1.2 Comments on students’ placements evaluation Regarding the results of the evaluation of placements made by the students, can be highlighted: Regarding to the students that completed the placements, students declare that only 7 out of 12 projects evaluated (58%) finalized when they answered the questionnaire. This percentage is lower than the percentage of success rate in other similar programmes like the Google Summer of Code, that is according to its website around 80-89.7% in the last 10 years (https://developers.google.com/open-source/gsoc/resources/stats#by_the_numbers_blog_posts).

Despite of the significant difference in the success rate, it is possible to observe that in the Google’s Summer of Code, the trend in the success data is that it is increased over the years, so also can be possible that the Semester of Code also increases its success rate in the following rounds (these metrics stated by the students are regarding the both two finalized instances). Moreover, the differences also among both virtual placements programmes can led to different results magnitude. For example –and as will be explained later- the money could be a powerful motivation source to complete the placements; in this case, Google Summer of Code pays more or less 5000$ to each student that complete the project for which they work.



Regarding the reasons to not complete the placements, students explained that they have no time to develop the project (2 students stated it), the lack of knowledge regarding the technology that must be used to develop the project (another 2), difficulties communicating with the tutor –mentor - (other 2), and a last one that did not complete the placement in time but is in progress to finish it in the following months. Regarding the opinion about the overall experience of the placement, most of the students mark the placement with 4/5 points (50% of them). The average mark regarding the placements is 3.58/5, with only 2 (16%) opinions below the mid mark 3 (1, 2). About the tutors –mentors- students rated them mostly with the maximum possible mark: 5/5, so the experience with the mentors in general was outstanding. Also, the average rating of the opinions is 4.00/5. In the case of academic supervisors, the rate is quite similar. 50% of students (6) give a 5/5 mark to them, with an average rating of 3.92/5. So, regarding the experience with the mentors and academic supervisors, students are mostly very satisfied. In the case of the relationship between tasks carried out in the virtual placement and the knowledge and competencies they are acquiring in their Universities studies, students declare that in general there is a low relationship (most of students -42%- checked the mark 2/5 and the average rate is 2.92, below the medium value 3) between the work carried out in the placement and the knowledge they got/get in their studies. This can occur due several reasons, like the gap between the knowledge acquired in the academia and the knowledge required by companies, the needed specialization in some technologies that require the companies (technologies that students do not learn at the University), or the knowledge level that student owns. Anyway, researchers and evaluation do not provide a clear reason about this low relationship, only minimal thoughts to explain it. Other question interesting to analyse is the question “Did your experience on your placement help you in planning your future professional career?”. In this case, the average rate provided by students is a 3.42/5. There are two mode ratings, 5/5 (33% of students) and 3/5 (other 33%). This outlines that there are some students (33%) truly convinced that the placement will help them in their future professional career (at least planning it) and others (33%) that possibly are sceptical about it. Despite of all, regarding the average rating, the students think that the placement will help them in planning (better) their professional career. Regarding to other questions related to the placement and future professional career, there is another good question to feature: “I think that my placement experience will help me in getting a job in the near future”. This question can be a good indicator of the relevance of the Semester of Code in the education and professional skills development of the students, and also its effectiveness building bridges among academia and business through transferring knowledge and promoting professional attitudes in students. Observing the results provided by students, they think in general that this placement will help them in getting a job in the future (the average mark is 3.17). 75% of students rated it with values 3, 4 or 5 out of 5 possible maximum points. Despite of these values, the average is not very high, but it is a promising value for a pilot programme.



About other opinions on the foundation/companies and the placement performed, the students declare that most of them (83%) recommend the company/foundation to other students to do a placement. Also all of them (100%) did not get a job offer from the foundation/company. Despite of this statistic about getting a job, 42% of students stated that the company/foundation offered to continue collaborating as volunteer developers of the projects. This can occur due many projects and open source companies that participated in this kind of initiatives like Semester of Code or Google’ Summer of Code can be considered as indie, this is, they own a little bit of money or budget, and they cannot commit to contracting new staff for the projects. Regarding the question about if other company/foundation offered work or collaboration in relation with the experience in the Semester of Code, only one student (8%) got it. Regarding the short time passed after the finish of the placement, this value is conclusive and it should be evaluated in the following months/year. About the value that Semester of Code provides to the FLOSS (Free Libre Open Source Software) Community, 75% of students affirm that their participation in the Semester of Code give them a positive feeling about collaborating in FLOSS projects and FLOSS Community. This kind of development of positive feelings can lead to the involvement of these students in future FLOSS projects, which could be considered other of the outcomes that this project give to the society. About the easiness of getting involved in the Semester of Code, the average opinion is that was easy. The average value of their opinions is 4.0 out of 5. Among the reasons that could improve it, students feature projects more related to studies contents, more visibility on universities websites, better information about the projects and improving the Virtual Placement System features to search projects and more information about the programme. Regarding to the problems faced out by students, they stated issues with time (limited time to implement the project), and issues with tutors (“tutor couldn’t help me during the summer” or “tutor said to me: start working and then we will reach you”). About if the students enjoyed participating, 75% of them declared that they enjoy taking part and 58% would like to participate again. This represents a good indicator about the general opinion about the programme.



3.2 Mentor’s placements evaluation As in the previous subsection, first of all is presented the questionnaire completed by the mentors involved in the Semester of Code Virtual Placements. The questionnaire is available using the URL http://forms.semesterofcode.com/mentors and for download in http://doi.org/10.13140/RG.2.1.2838.5047


a. Male b. Female

4. The company / foundation you belong to* 5. The name of the project you mentored on*

The name of the project as it appears on the Semester of Code Virtual Placement System (VPS)

6. The student's name* The name of the student that worked on the project

7. Did the student completed the placement?* Did the student completed your work on the assigned project (and its tasks)?

a. Yes b. No

8. If the student did not complete the assigned project, what do you think the problem was?

9. Starting date of the placement* 10. End date of the placement* 11. Please summarise the issues that the project raised for the student, and the

tasks that they needed to carry out* 12. What problems did the student encounter? How did the student overcome

them, and did you help?* 13. Please rate your overall experience of the Semester of Code mentoring

process* Regarding your overall opinion about the whole experience (Scale 1 to 5)

14. Please rate your experience with the student during the Semester of Code* How well / bad was your work with the student? (Scale 1 to 5)

15. In your opinion, will this placement experience help the student in their future professional career?*

16. Where are the results of the placement (code, etc) hosted?* Please add the links (Github, websites, etc.) where is available publicly the placement results

17. Would you recommend that your company/project/foundation should give paid employment to the student that was involved in the placement*

a. Yes b. No

18. Would you recommend that your company/project/foundation should offer a voluntary role to the student that was involved in the placement* In this or other project

a. Yes b. No

19. Would you recommend that another company/project/foundation should give paid employment to the student involved in the placement?*

a. Yes b. No



20. Do you think that the Semester of Code placement helps students to develop a positive attitude to engaging in FLOSS projects and the FLOSS Community?*

a. Yes b. No

21. Please rate the student's technical ability* (scale 1 to 5) 22. Please rate the student's ability to learn* (scale 1 to 5) 23. Please rate the student's task management skills* (scale 1 to 5) 24. Oral communication skills* (scale 1 to 5) 25. Written communication skills* (scale 1 to 5) 26. Responsibility* (scale 1 to 5) 27. Adaptability* (scale 1 to 5) 28. Creativity and initiative* (scale 1 to 5) 29. Personal involvement* (scale 1 to 5) 30. Motivation* (scale 1 to 5) 31. Receptiveness to criticism* (scale 1 to 5) 32. Punctuality* (scale 1 to 5) 33. Relationship with the work environment* (scale 1 to 5) 34. Capacity for teamwork* (scale 1 to 5) 35. Was getting involved in the Semester of Code programme a smooth

process?* (scale 1 to 5) 36. What problems did you encounter in the Semester of code process?* 37. How would you improve the Semester of Code process?* 38. What mentoring actions did you have to take with your student(s)?*

Please also include any mentoring actions which were not directly related to project development work

39. Did you enjoy taking part?* a. Yes b. No

40. Would you take part again?* a. Yes b. No

41. Did you learn anything from being part of this programme?* 42. If you took part again what would you do differently?*

3.2.1 Mentors’ placements evaluation results There is available a spreadsheet with the results of this questionnaire available in https://docs.google.com/spreadsheets/d/1L5J4hB9IumIj-RKzutQEVgHvnLpCMQnV_oxGKP8i3MY/edit?usp=sharing (all data available except names and other personal information). Some of the results are available also in a visual way in https://juancb.typeform.com/report/xA1v1t/Ndrt 13 mentors (from software companies/foundations) out of 17 that participated in the Semester of Code completed the questionnaires. Regarding the responses provided by the mentors, can be highlighted the following results:



The average age of the mentors is 39 years old.

About the reasons for not complete the placements, they outline (again, all the comments are available in the same way mentors wrote them, sic):

• “The student was not in Europe, therefore she was not doing it for credits and either of the two mentors were remote. Also, though she was interested in contributing she was between moving places and starting new jobs.”

• “No remuneration. Lack of time because studies and remunerated job.” • “I don't know. Maybe lack of time.” • “Personal problems. Lack of proactivity.” • “Lack of economic remuneration” • “Lack of commitment”

Please summarise the issues that the project raised for the student, and the tasks they needed to carry out:

• “Development following the given design, standards and style of existing code. Efficient implementation of given algorithms. Design of an API for the implemented functionality for multiple programming languages.”

• “She had to create different clients to connect to various servers. SunPy counted already with a factory model to connect to different servers, however each new instance need to be added to the factory. The fact that these tools were in a feature branch and not fully tested made the project a big harder to start.”

• “The student had to learn a lot of new technologies, including e-learning standards, platforms, languages and a non finished framework (tsugui). This has been both an issue and a valuable thing for his learning.”

• “Integrate 3d canvas into existing rendering code Implement straightforward WebGL APIs for 3d canvas”

• “Before starting the student told me to leave the project.” • “The student needed to learn the mechanics of the OpenMandriva build farm

to build rpm software packages. This requires ability to use git repositorys and to be familiar with the content of rpm packaging files”

• “First Beta” • “The student attempted to add the proposed SVG/CSS 'stroke-alignment'

property to Inkscape.”



• “He had to learn a lot about making contributions to a public codebase, including respecting existing APIs, documentation, providing full unit tests for their work, and testing patches against a full test suite. I think they learned a lot about the practice of software development. They completed the main piece of work, which was to implement a HTML5 ruleset, and also an additional piece of work, supporting piped commands”

• “He needed to learn html, video playback functionalities, plus the usage of lms softwares and the IMS LTI standard”

• “Apache Corinthia http://corinthia.incubator.apache.org/ is a project that concentrates on generating a data loss free documenter converter as well as a responsive design document editor for mobile devices (touch enabled devices). Office Documents of today usually use either the ODF or the OOXML standards. These standards are very well documented and standardized. However both standards include a larger number of items that are "optional", meaning a given implementation may not support and may even delete it "implementation defined", meaning nobody really knows how it is handled. When organisations tender for new office programs, they trust that asking for ODF/OOXML secures interoperability, sadly enough that is not case. In this project, we aim at providing a compliance sheet divided in 2 parts: Optional features Implementation defined features And made as a list of checkmarks with comments, so the implementor can fill it out, and the organisations can use it to compare vendors. The project is more research than actual programming. The standards are quite big, and not easy to read (for non-programmers), so the challenge is to convert it into something easy understandable. If time permits it would be an added feature to make the sheet available online, e.g. as part of the corinthia web.”

What problems did the student encounter? How did the student overcome them, and did you help?

• “He mostly worked on his own, solving most problems on the way by himself. I mostly only needed to help in terms of guiding the development to fit within the existing design of the software library.”

• “The student raised many questions during the project, most of them answered/discussed by email. Additionally we had a couple of videoconferences with the assigned VALS mentor (Univ. Salamanca) and a few between her and me. Most of the problems were to do with git at the beginning and with how the factory model was working.”

• “Most where technical problems which he solved by himself. My interventions had to do in the design and concept of the solution, for the most part.”

• “Lots of questions about the preexisting code. He spent time reading it and talking to the experts on the team.”

• “Find time to do the project” • “Problems creating de development environment.” • “Unable to set development environment. Difficulties to ask for help when

needed” • “The student had to learn to use our build farm; my colleague pointed him in

the right direction and after that he was largely able to work on his own initiative.”



• “Creating de development environment.” • “Part of the project was to identify problems with the path offsetting

algorithm used by Inkscape as well as problems with how the 'stroke-alignment' property is defined in SVG. This turned out to occupy most of the student's time. To identify and evaluate the various problems, the student created a series of test cases.”

• “The main issue was they hadn't been used to an environment where everything must be fully tested before it can be incorporated into a release; I helped by walking them through the test process and getting them into the habit of never creating patches that haven't been fully tested.”

• “We encountered the normal problems in this kind of project. We approached it with an iterative methodology that has worked fine.”

• “The student doesn't reply the emails”



About where are hosted the results of the placement:

• https://github.com/DethonUSAL/audaspace • https://github.com/tortoiseJess/sunpy/commits/unidown_now • https://code.google.com/p/video-quizz-lti/ • https://github.com/ecoal95/rust-offscreen-rendering-context

https://github.com/servo/servo/pull/5863 https://github.com/servo/servo/pull/5957 https://github.com/servo/servo/pull/6083 https://github.com/servo/servo/pull/6183 https://github.com/servo/servo/pull/6240

• The packages student built after passing through QA were hosted on our main distribution repository site at http://abf-downloads.rosalinux.ru/openmandriva2014.0

• http://bazaar.launchpad.net/~ntrrgc/inkscape/inkscape/changes



• http://htmlcleaner.sourceforge.net/release.php • https://github.com/granludo/video-quizz-lti



What problems did you encounter in the Semester of Code process?

• “No company/organisation backing up the open source project needed to have USAL do that.”

• “On the first season: The student (the above rated) was from outside EU and we got a VALS mentor without problems. On the second season, I just got one application with a very bad proposal which I didn't accept it. I provide an opportunity to the student to update it by giving him feedback, but he never replied.”

• “Just the problem of yet another platform to work with” • “Unclear deadlines. Unclear requirements. Unclear ability to modify

descriptions after submission. Projects that defaulted to being made available in subsequent rounds without our knowledge. Unclear interface for evaluating student proposals.”

• “No motivation of students” • “Lack of motivation by students. I think due a lack of economic

remuneration. Distance also played an important role.” • “Student's difficulty to ask for help. I didn't expect that he must know all

techonologies related to the project, but to ask for help while he was doing the tasks scheduled for the week.”

• “I encountered a problem with this form. It is not always the case that there is oral communication with the student. In Santiagos case I have given him 5 even though I have never spoken with him. Another problem with the form is that asking a free software group whether they should give someone paid employment is the wrong question. The question should perhaps be "Would you 'employ' this student in your group again" or similar wording as groups who produce free software do not necessarily pay their developers.”

• “Student's motivation, lack of responsability.” • “There was a bit of mismatch between the program's organization and the

way a project like Inkscape is organized (e.g. we are not a company or foundation).”

• “The interface on the online application”



• “Lack of commitment from the student” How would you improve the Semester of Code process?

• “Maybe enforce setting individual deadlines at the beginning and add evaluations on the way.”

• “I think it needs a bit more of advertisement and more universities involved.” • “A kick off presential meeting with all the mentors, and a final meeting with

the mentors to gather experiences.” • “Make the steps in the process much clearer. Have better-defined

requirements for project descriptions. Have more coordination between VALS administrators and project mentors/administrators.”

• “Paying projects.” • “Students motivation.” • “Payment system for student. This would help in the motivation.” • “Is there some way that the project owner could get statistics about access

to their projects. This would allow us to judge which projects are likely to prove attractive to students.”

• “Give an economic remuneration to the student.” • “Communication could be improved.” • “1) pay something to the student, maybe some goods a tablet, chocolate or ...

yes money is also good 2) a better app” What mentoring actions did you take with your student?

• “Setting the expected goals, reviewing the progress and results and following up on the correction of errors.”

• “Nothing outside git or python, which I think it's part of the project.” • “Google hangouts meetings, email, code revision, demo and mookups

revisions” • “Regular conversations to determine what Emilio was working on and what

problems he was encountering. Inviting feedback from other experts to address questions he had. Code review.”

• “Emailing” • “Skype meetings” • “Skype meetings for scheduling tasks. Emailing” • “Initially I gave advice as to what he needed to learn but in particular some

advice for conducting online relationships with other developers and users an often neglected part of the development process where bad examples abound.”

• “Skype meetings” • “We exchanged frequent emails.” • “Mostly emails and responding to patches via the issue tracker. We didn't

need to do much more really; we could resolve all issues very simply using text.”

• “Email, code reviews, google hangouts, app testing, mockups”




• “I tried to improve my mentoring of students, but the student was already very good, so not much mentoring was needed.”

• “Yes! thanks to our VALS academic supervisor I learnt about deeper Computer Science concepts that I was not aware of - our project SunPy is being developed by physicists and programs like this help to improve the quality of the project because expert eyes on CS look and play with it. With expert eyes I also meant the students, as their knowledge many times are above ours (the core developers)”

• “I had participated before in the google summer of code, the experience is quite much like it and you always learn something”

• “The importance of clear requirements when putting a program like this together.”

• “Remote work management” • “Distance working” • “Distance working is hard” • “I learnt that todays students are usually more advanced than imagined

despite being young and inexperienced.” • “Yes, defining the behavior of the 'stroke-alignment' property is difficult.” • “I learned just how little students on CS courses are exposed to professional

practices in software development”

If you took part again what would you do differently? • “Communicate more with the student and academic supervisor and set clear

deadlines for the project.” • “I think I would start lying down a list of specific tasks and a estimation of a

due date. This way the student will focus on get them down. However, as I said, this time was a special case as the student was not receiving credits for the work.”

• “I would take two students for the same project, to get redundancy and more points of view.”

• “Preparing better the first weeks in order to facilitate students configuration of the development environment.”



• “I would probably try to get a better idea of the students ability at the beginning.”

• “Be a bit more demanding early on.” • “I would give a more detailed description of the job to be done, this means I

would have to think better in what I need but then the project will be more smooth”

3.2.2 Comments on mentors’ placements evaluation About the evaluation of placements made by the mentors, researchers highlight some thoughts: Regarding to the students that completed the placements, mentors declare that only 7 out of 13 projects (54%) were finalized based on mentors’ evaluation when they answer the questionnaire. This is less (4% less) than the completion rate provided by students. It happens due not all students nor mentors answered the questionnaire and it can produce this variation in the success rate. The mentors indicated the following issues to explain why students did not complete the placements:

• The student was not in Europe and cannot get credits as academic reward for complete the placement; also was starting in another job

• No remuneration in the placement • Combining the placement with other remunerated job or the studies • Lack of commitment and proactivity

The reasons provided reference some of the reasons provided by the student in the section 2 of this document “Students’ opinion about Semester of Code before the enrolment in the placements” about why to not participate. In this case the lack of economic remuneration is a problem regarding to the completion of placements, and the academic rewards can be not enough to finish the project for some students. Also the student’s time management skills are observed like a potential risk in completing the placement. But in general, the lack of motivation and commitment are observed as key factors to not complete the placement. About rating the mentoring process experience, the mentors are pleased in general (average rate of 3.38/5), and the mode rate is 5 points out of 5 possible (31% of mentors). Despite of the good average and the good mode in the opinion, there are 3 mentors that declare their mentor experience in the Semester of Code was the worst possible (rate of 1/5). In those cases this experience was the worst possible due the lack of commitment of the student, which did not respond the emails, and drop out the placement in an early stage without clear reasons. Regarding to the opinion about if the placement can help the students in their future professional career, also there is a common good opinion with an average mark of 3.31/5. Again, another 4 mentors (31%) think this placement would not help them in the future professional career. Taking into account that 6 mentors declared that the placements they mentorize were not finished, there are 4 students (possible of these 6 uncompleted) that will not allow, but there could be another 2 placements that despite uncompleted will help the students to achieve a better future professional career. The other 2 mode values are 4/5 and 5/5 both also chosen by 31% of mentors.



Some metrics that can demonstrate the value of the work done during the placement are those related to the questions “Would you recommend that your company/project/foundation should give paid employment to the student that was involved in the placement?”, “Would you recommend that your company/project/foundation should offer a voluntary role to the student that was involved in the placement?” and “Would you recommend that another company/project/foundation should give paid employment to the student involved in the placement?”. Regarding the first of these questions, 69% of mentors think that their company should contract the student that performed the practice, this is a good indicator about the satisfaction with the results of the placement and can be observed as an initial evidence of the knowledge alliance built among the academic institutions and business stakeholders. Regarding to the second question, 62% of mentors think that their project/foundation/company should offer a voluntary role to the student; this is another good indicator, mentors would like to continue working with the students at least in a voluntary mode. Should be taken into account the 7% of mentors that think that company should contract the student but not offer a voluntary role, could it means that there are at least one student that exclusively deserve to be hired and not to participate as volunteer. Regarding to the third question, 85% of mentors would recommend the student to be hired in other company/project or foundation, that is they think student would be valuable also for other business stakeholders (this indicate also the satisfaction and personal opinion of the mentor about the student). About if the Semester of Code placement helps students to develop positive attitude to be engaged in FLOSS projects and FLOSS Community, 69% of mentors declare that yes, the placement will help students to be engaged in the future in FLOSS community and projects. This can be another good outcome of the project, in terms of creating future collaboration relationships between business and academia through Open Source projects/philosophy. The following questions in the questionnaire are related to review the students’ skills and competencies. To assess them, the mentors rate each one in a scale of 1-5 points. Following are the average results:

• Students’ technical ability: 3.54 • Students’ ability to learn: 3.62 • Students’ tasks management skills: 3.23 • Oral communication skills: 4.08 • Written communication skills: 4.08 • Responsibility: 3.46 • Adaptability: 3.38 • Creativity and initiative: 3.31 • Personal involvement: 3.08 • Motivation: 3.31 • Receptiveness to criticism: 3.62 • Punctuality: 3.38 • Relationship with the work environment: 3.31 • Capacity for teamwork: 3.08

Average of all the results: 3.46



In general the mentors rate well the students in all evaluated skills, in all of them they have a good rating (more than the medium value, 3). Also the average of the skills is pretty good, with a value of 3.46/5. Regarding the distribution of the skills rating, the students have better skills in oral communication and written communication. The worst skills are the personal involvement and capacity for teamwork. Analysing the open questions about the problems encountered by the mentors during the Semester of Code, can be highlighted the following issues:

• Students’ lack of motivation • Students’ difficulties on asking for help • Difficulties in the companies timing and Semester of Code organization • Unclear deadlines, requirements and other organizational issues with the

Semester of Code. • Issues in coordinating the placement with universities

One key point of the questionnaire is about how mentors would improve the Semester of Code. In this question, the main answers were:

• Setting individual deadlines for each project (not common deadlines for all projects in each season of the Semester of Code)

• More advertisement about the Semester of Code • More universities participating to increase the number of participant students • In presence kick-off meeting and closing meeting with all the mentors to

gather experiences and share knowledge. • Give economic (or material) rewards to the students • Make the participation process clearer (better-defined requirements for the

projects, better coordination with Semester of Code administrators, etc.). • Improvements on the virtual placement system (the platform).

Also, about the mentoring actions they took, the mentors featured: • Maintain skype meetings • Set up goals, reviewing goals and results (code reviews, etc.) • Help with programming languages or specific technologies used in the

projects Regarding to the satisfaction participating in the Semester of Code, 92% of mentors affirm they enjoy taking part. Also 69% would like to participate again in next rounds. These answers provide a clear measure of mentors’ engagement and satisfaction; both are is quite high. Apart of the students, this programme can lead mentors and academic supervisors to learn and acquire new skills. In the case of the mentors, they feature learning about new concepts related to Computer Sciences (in the case of a mentor related to Physics more than Computer Science), new ways of working in distance, about specific issues related to the projects, etc. Other kind of learning is about what the mentors would do differently if they participate again. In this case, the mentors outline that they would communicate more with students and academic supervisors to set clear deadlines for the project, proposing specific tasks, take two students for the same project, prepare better the beginning of the placement to help student configure the development environment, evaluate better the student’s abilities at the beginning, or provide a more detailed description of the job to be done.



3.3 Academic supervisors’ placements evaluation First of all is presented the questionnaire completed by the academic supervisors involved in the Semester of Code Virtual Placements. The questionnaire is available in the URL http://forms.semesterofcode.com/supervisors and for download in http://doi.org/10.6084/m9.figshare.2955931


a. Male b. Female

4. Your University* a. Faculty of organizational sciences, University of Belgrade – Serbia b. Frederick University – Cyprus c. Open University of Cyprus – Cyprus d. Universitat Politècnica de Catalunya - BarcelonaTech (UPC) – Spain e. University of Bolton - United Kindgdom f. University of Cyprus – Cyprus g. University of León – Spain h. University of Nicosia – Cyprus i. University of Oxford - United Kingdom j. University of Salamanca – Spain k. University of Udine – Italy l. Università Ca’ Foscari Venezia – Italy m. Université Paris 8 – France

5. The name of the project you supervised* The name of the project as it appears on the Semester of Code Virtual Placement System (VPS)

6. The company/foundation whose project you supervised* 7. The student's name*

The name of the student that worked in the project 8. Did the student completed the placement?*

Did the student completed your work on the assigned project (and its tasks)? a. Yes b. No

9. If the student did not complete the assigned project, what do you think the problem was?

10. Starting date of the placement* 11. End date of the placement* 12. Please summarise the issues that the project raised for the student, and the

tasks that they needed to carry out* 13. Please summarise the learning goals achieved by the student in the

placement* 14. What problems did the student encounter during their placement? How did

you help the student overcome them?* 15. Please rate your overall experience of the Semester of Code mentoring

process* Regarding your overall opinion about the whole experience (Scale 1 to 5)

16. Please rate your experience with the student during the Semester of Code* How well / bad was your work with the student? (Scale 1 to 5)



17. Please rate the relationship between tasks the student carried out during their project and the knowledge and competencies they are acquiring in their University studies* (scale 1 to 5)

18. Please rate the way that the student carried out the required tasks* 19. In your opinion, will this placement experience help the student in their

future professional career?* 20. Do you think that the Semester of Code placement helped the student to

develop a positive attitude to engaging in FLOSS projects and the FLOSS Community?*

a. Yes b. No

21. Please rate the student's technical ability* (scale 1 to 5) 22. Please rate the student's ability to learn* (scale 1 to 5) 23. Please rate the student's task management skills* (scale 1 to 5) 24. Oral communication skills* (scale 1 to 5) 25. Written communication skills* (scale 1 to 5) 26. Responsibility* (scale 1 to 5) 27. Adaptability* (scale 1 to 5) 28. Creativity and initiative* (scale 1 to 5) 29. Personal involvement* (scale 1 to 5) 30. Motivation* (scale 1 to 5) 31. Receptiveness to criticism* (scale 1 to 5) 32. Punctuality* (scale 1 to 5) 33. Relationship with the work environment* (scale 1 to 5) 34. Capacity for teamwork* (scale 1 to 5) 35. Was getting involved in the Semester of Code programme a smooth

process?* (scale 1 to 5) 36. What problems did you encounter in the Semester of code process?* 37. What problems did you encounter?* 38. Did you enjoy taking part?*

a. Yes b. No

39. Would you take part again?* a. Yes b. No

40. Did you learn anything from being part of this programme?* 41. If you took part again what would you do differently?*

3.3.1 Academic supervisors’ placements evaluation results There is available a spreadsheet with the results of this questionnaire available in https://docs.google.com/spreadsheets/d/1wtKMtqoaRVAa4dp0H0b-xI9FgeG8BblcchlLZSzoIMw/edit?usp=sharing (all data available except names and other personal information). Some of the results are available also in a visual way in https://juancb.typeform.com/report/CQfOsh/W4XV In this case, 14 academic supervisors out of 17 answered to the questionnaire. This is a brief abstract of their responses:



The average age of the academic supervisors is (approximately) 40 years old.

If the student did not complete the assigned project, what do you think the problem was? (as in the other cases, all the comments by academic supervisors are reproduced literally, sic):

• “The student expected more contact with the company. They have little quick answer and told him that was far from finished. As its aim was to validate them and would not get to the deadline, he abandoned the practice.”

• “She had no time to finish the project because she find a job.” • “He was very slow with his task and decided to drop out the project. We

propose him to do other project in another moment.” • “Material required for the implementation was not available from the mentor

(on time).” • “The student had not the proper knowledge to solve the project from the

beginning and was losing the motivation with the project along the time. Also its location (outside Europe) was a problem to arrange meetings and enhance the project tracking.”

• “The problem was the time gap between the task submitted by the student and receiving the feedback from mentor”

• “Personal problems”



Please summarise the issues that the project raised for the student, and the tasks that they needed to carry out:

• “Create an online editor that allows adding subtitles to recordings.” • “Apache Corinthia http://corinthia.incubator.apache.org/ is a project that

concentrates on generating a data loss free documenter converter as well as a responsive design document editor for mobile devices (touch enabled devices). Office Documents of today usually use either the ODF or the OOXML standards. These standards are very well documented and standardized. However both standards include a larger number of items that are "optional", meaning a given implementation may not support and may even delete it; "implementation defined", meaning nobody really knows how it is handled. When organisations tender for new office programs, they trust that asking for ODF/OOXML secures interoperability, sadly enough that is not case. In this project, we aim at providing a compliance sheet divided in 2 parts: Optional features and implementation defined features. And made as a list of checkmarks with comments, so the implementor can fill it out, and the organisations can use it to compare vendors. The project is more research than actual programming. The standards are quite big, and not easy to read (for non-programmers), so the challenge is to convert it into something easy understandable. If time permits it would be an added feature to make the sheet available online, e.g. as part of the corinthia web.”

• “This project's aim is to add high level audio functionalities to the engine that can be commonly used by games. These functionalities include for example: Playback manager: A class/interface that can handle multiple sounds being played back with different categories assigned (e.g. voice, music, background noise, notifications, etc.) where each category can be controlled. Useful for the game engine for example to easily pause/stop all sounds and adjust volume levels of different categories. Environmental audio: different sound modifications (filters, reverb, etc.) which are spatially dependant on sound source and listener position. For example a gunshot in a tiled bathroom sounds different than in a room with sound absorbing walls. Dynamic music: a music playback interface which can change the music based on some user-defined flags. For example: a game has some random background music while the player runs around and then he starts fighting against a bunch of enemies and the music gets more exciting. This interface should then use predefined loopable music samples and transition nicely between them changing the mood of the background music according to the action that is currently going on. Random sounds: repeating sounds all over again sounds boring and is easily noticed by the player (for example footsteps) so this functionality should add a list of sounds to the sound actuator instead of a single one, with the ability to choose sounds randomly or sequential when it is triggered.”

• “Implement a relevant subser of missing/incomplete DOM interfaces defined by the WebGL specification. Design and implement offscreen, hardware-accelerated rendering for the 3d canvas rendering subsystem in Servo. Investigate automated testing strategies for WebGL canvases.”

• “Inkscape is a vector graphics drawing program that uses the W3C SVG (Scaler Vector Graphics) standard as its native file format. Version 2 of the SVG specification is under development, adding many interesting and useful



new things. Interest in SVG has grown dramatically the past couple of years due to its inclusion in HTML5 and support in all major browsers. This project is to add one or two of the new features to Inkscape. The student will be expected to browse the SVG 2 specification and the CSS specifications it references, find one or two related features that interests them (see link for suggestions), create a plan for implementing those features in Inkscape (with help from the mentor), and then do the implementation. Depending on the complexity of the chosen features and the skills of the student, a project might include: rendering the new feature, creating a GUI for the new feature, and/or creating an SVG 1.1 fallback for the new feature. The student should have good communications skills and a strong understanding of C++. An alternative project would be to implement 'pollyfills' to render new SVG/CSS features in browsers that don't yet support them. This would allow Inkscape to begin to use these new features without having to worry about browser support. A student pursuing this path must have strong JavaScript skills (as well as good communication skills).”

• “Develop a magento reward points extension. The module should have the following features: Enable/Disable module Configure which products can be bought with points. There are products that can be bought only with points or only with money, but not with a mix of them. The rest of products will only can be bought with money. Create rules similar to the magento cart rules and catalo rules in order to: Configure how the user can gain points by buying some specific products. Be able to modify the users points from the administration panel. Enable/disable points earn by commenting products. Also, configure how many points the user will earn.”

• “The student will be trained to work as part of team of individuals who are maintaining and improving the Linux distribution "OpenMandrivaLx" . His main role will be to monitor incoming bug reports and act on their content as directed by their current supervisor. The student will be initially responsible for establishing the validity of reported bugs and for gathering additional information as directed by the team member currently managing him. Due to the nature of the enterprise the student will work with people of differing departmental responsibilities and skill sets and will need to learn to communicate his needs at different levels of technical ability. He will need to develop these skills to a high level in order to gather the maximum information from bug reporters who are often very new to the Linux operating system. When necessary he will refer to his Mentor for additional training and support. The student will be expected to be available for 8 hours each day and should be present on the #openmandriva-cooker irc channel for this period. He will be expected to respond promptly to email communications and undertake to do all the tasks that his mentor directs. During his period of internship the student will be trained in the use of Bugzilla (Bug management software), he will learn how to build RPM packages on our "ABF build farm" and he will learn some basics about RPM spec files (the files that direct how the package is to be built). He will be taught the fundamental approaches to debugging the type of issues that occur during the production of a Linux distribution.”

• “Develop an extension for Magento to add three extra fields for registered customers. This field will be optional at registration time. The three fields will



be: customer's DNI and the front and back image of the DNI. The images should be saved in an Amazon S3 bucket. Some products of the magento shop requires the customer to send a copy of the DNI, so the extension should have a configuration page at the administration panel. In such page, the user should be able to configure for which products and/or categories the three extra fields are mandatory. When the customers buy any product in which the DNI is required, if the customer hasn't yet filled this fields, the checkout page should ask the customer to send this information.”

• “Develop a magento extension to simplify the default product image bulk import of the platform. The extension should allow the user to select the folder in which the product images are. Then it should upload to the server all the images and bind each image with its related product. Some desired features are: Product images should be stored in a Amazon S3 bucket. The extension should notify the user if for a product there already exist imported images and ask to the user to: import images (the existing images will not be removed and the new ones will be uploaded and imported) import images and delete old ones (the existing images will be removed) cancel import (cancel import only for this product) Make a report of the imported images, products, errors... Be able to list all the products without images. User's manual integrated into the magento admin area.”

• “The student had to improve the use of HTML5 in HTMLCleaner. He had to extend an existing tool and integrate his changes.”

• “Programming in new languages (web languages). Study a new system and think how to improve it.”

• “The student was committed to create some clients to connect to various servers related to the Sunpy Now Module. Sunpy had part of the basis for implementing the clients, but need a lot of work by the student to complete the task (this work would be easier if the student have the proper knowledge in Software Engineering).”

• “Learning about web interface development technologies Choose adequate technology for the project Cope with JavaScript and frameworks Design an interface Realize web interface for a social network”

About the learning goals achieved by the student in the placement:

• “Learn the use of RESTful services. Learn the use of web technologies (HTML, CSS), learn new additional technologies (e.g., subtitle files).”

• “Technical capacity; Learning capacity; Work administration; Oral and written communication skills; Sense of responsibility; Adaptability; Creativity and initiative; Personal involvement; Motivation; Receptiveness to criticism; Punctuality; Relationships within the working environment; Team spirit; Diplomacy”


• “Familiarity with the HTML canvas APIs, and WebGL/OpenGL APIs in particular. Identifying and prioritizing potential technical challenges in a brand new implementation. Writing automated HTML/JavaScript test to verify



correctness Translating English specifications into Rust code. Writing Rust code that uses multiple threads for optimal performance. Identifying barriers to reusing existing conformance tests. Reading existing C++ code from other open source browser engines.”






• “Learn HTML5, work with version control tools, perform remote communication and upload the results.”

• “Learning new technologies, Remote communication, Communication in English”

• “Learn Python in an initial way. · Learn about Software Engineering and Software Patterns · Learn about Git environments · Learn about time management”

• “Dived into ReactJS Upgrade knowledge about UI and UX” What problems did the student encounter during the placement?

• “Not good communication with the mentors. Had to learn a lot of new technologies.”

• “He didn't give time to finish the practices and we seek a temporary solution with the university.”

• “Her practice coincided with a month of mentor's holidays. She kept answering but less frequently.”

• “He dropped out the project and he want to do another project in next rounds.”

• “Necessary material was not available by the mentor. It was also not easy to contact the mentor to acquire this information.”



• “The student had not very extensive knowledge in Software Engineering · The student had not previous knowledge in programming with Python · The student had not previous knowledge in distributed development environments like Git · The student lost the motivation along the project To help her with their problems, I recommend several books and resources to learn Python, Software Engineering and Git. Also the mentor and me tried to motivate her in each meeting we had along the project development.”

• “The biggest problem was student's inexperience with JavaScript generally. I managed to help student overcome those issues by holding several crash courses focused on JavaScript basics”



How they you improve the Semester of Code process:

• “Align it with the semester of the universities. Make the communication with the mentors more formal.”

• “Opening the periods for the eligibility of students, making able the stakeholders to fix all the deadlines and periods to perform the placements, reinforcing the dissemination of the VALS project and extending the scope of the program to more countries and more types of students”







• “Opening the periods for the eligibility of students, making able the stakeholders to fix all the deadlines and periods to perform the placements,



reinforcing the dissemination of the VALS project and extending the scope of the program to more countries and more types of students”

• “Align with the university semester.” • “Alignment with the semester of the university. Also probably perform the

placement in the framework of a specific course.” • “Opening the periods for solving and proposing the projects, improve the

communication between the involved stakeholders” • “I think more details about projects should be available to students. Also, it

would be very helpful if supervisors had an opportunity to directly contact project mentor in order to grasp some more information. Overall SoC process is pretty good. These are just minor issues”

What problems did supervisors encounter:

• “Not aligned with the semester. Unavailability of mentors.” • “It's complicated do the agreements with the institutions.” • “It's complicated do the agreements with the institutions.” • “It's complicated do the agreements with the institutions.” • “It's complicated do the agreements with the institutions.” • “It's complicated do the agreements with the institutions.” • “It's complicated do the agreements with the institutions.” • “It's complicated do the agreements with the institutions.” • “It's complicated do the agreements with the institutions.” • “Unavailability of the mentors in some cases.” • “Unavailability of mentor. No access to required information (code and

data).” • “Marking the projects as solved and the process to assign the project to a

student” • “Unresponsive project mentor”


• “Remote placement is an interesting concept.” • “Administrative processes in the universities are very strict, especially

regarding schedules, and are not shared by all the universities all over



Europe. Schedules and workload in companies are not always compatible with universities ones and students’ planning”

• “Administrative processes in the universities are very strict, especially regarding schedules, and are not shared by all the universities all over Europe. Schedules and workload in companies are not always compatible with universities ones and students’ planning”







• “Collaboration with different people.” • “Communication with different people. Nice experience for the students.” • “I continued learning to work in international teams”

If you took part again what would you do differently?

• “Nothing” • “Try to be more independent in the workplace” • “Nothing” • “Nothing” • “Nothing” • “Nothing” • “Nothing” • “Nothing” • “Nothing” • “Nothing” • “Have some rules for the communication with the mentors.” • “More formal communication with the mentor.” • “I will track better and more continuously the projects where I am involved”



• “Be more engaged in concrete project”

3.3.2 Comments on academic supervisors’ placements evaluation About the evaluation of the placements made by the academic supervisors, can be featured the following results: Regarding to the students that complete the placement, in this case out of the 14 academic supervisors that answered the questionnaire, the 50% (7) of the placement were completed, and (obviously) other 7 placements were not completed. The reasons provided by them regarding the non-completion were:

• Students needed more contact with the company on time. At least two students did not get enough feedback from the company and finally drop out the placement.

• Some students find a job during or at the beginning of the placement and abandoned it.

• Some students advanced slowly in the tasks related to the placement and lost motivation in such way that led to abandon.

• Insufficient knowledge to complete the tasks of the placement. • Personal problems (in one case).

Other problems that student found during the placements (and not necessarily those that led to drop out the placements) were:

• Bad communication with mentors • Learn a lot about some technologies unknown previously • Coincidence among the dates of the placement and mentor’s holidays

(mentor respond, but less frequently) • Some needed material of the project was not released by the mentor in time • Lack of knowledge in several technologies, methodologies and theories

needed for the projects • Inexperience in professional contexts.

About the overall experience of the Semester of Code mentoring process, the academic supervisors rate it in average with 4.14/5. The 50% of academic supervisors rated their experience with the maximum rating 5/5, 29% with 4/5, 14% with 2/5 -only 14% rated below the medium value the experience (rate as bad experience)-, and 7% with a medium rating 3/5. The experience with the students during the Semester of Code was also quite good. In average, the academic supervisors rate it with a 3.79/5 score. 3 of them (36%) rated the experience 3/5 points, while the 5/5 and 4/5 points were rated by 29% and 29% (58% of the mentors rate the experience with students with good and very good scores). In terms of relationship among the tasks carried out by the student in the placement and the knowledge and competencies, in general, the rate provided by academic supervisors is 3.93/5, which present a good relationship among both. Regarding to the way students carried out the required tasks, academic supervisors think that they did it in a good way, providing an average mark of 3.57/5.



The point of view of academic supervisors about the usefulness of the placements in the future professional career, they think that it will help with an average rate of 4.07/5. This opinion represents a strong consideration about the utility of the Semester of Code in the future development of the students. Also the 93% of the academic supervisors think that the Semester of Code helped the students to develop a positive attitude to engaging in FLOSS projects and FLOSS Community. Only one of them thinks it would not. The following questions in the questionnaire are related to the students’ skills and competencies. They are rated in a scale of 1-5 points. Following are presented the average results:

• Students’ technical ability: 4.0 • Students’ ability to learn: 4.21 • Students’ tasks management skills: 3.71 • Oral communication skills: 3.64 • Written communication skills: 3.79 • Responsibility: 3.86 • Adaptability: 3.57 • Creativity and initiative: 3.79 • Personal involvement: 3.71 • Motivation: 3.57 • Receptiveness to criticism: 4.0 • Punctuality: 3.64 • Relationship with the work environment: 3.79 • Capacity for teamwork: 3.64

Average of all the results: 3.78 In general, the academic supervisors provide a good evaluation of all the students’ skills, especially in the cases of the ability to learn, technical ability and receptiveness to criticism. The worst evaluations are those that concern the adaptability and motivation. The academic supervisors considered that getting involved in the Semester of Code is a smooth process; they rate it with 4.36/5. Despite of the good opinion, they provide some considerations to improve the Semester of Code:

• Align it with the semester of the universities. Make the communication with the mentors more formal.

• Opening the periods for the eligibility of students, making able the stakeholders to fix all the deadlines and periods to perform the placements, reinforcing the dissemination of the VALS project and extending the scope of the program to more countries and more types of students

• Align with the university semester. • Alignment with the semester of the university. Also probably perform the

placement in the framework of a specific course. • Opening the periods for solving and proposing the projects, improve the

communication between the involved stakeholders



• I think more details about projects should be available to students. Also, it would be very helpful if supervisors had an opportunity to directly contact project mentor in order to grasp some more information. Overall SoC process is pretty good. These are just minor issues

In general, can be observed two main trends regarding the timeline in the improvements proposed, one is about align the Semester of Code with the semester of the university and the other is about open the periods for students eligibility and application for the projects. The Semester of Code during the previous rounds and the future round is trying to test both hypotheses to research what approach is more successful. Also the academic supervisors feature the need of improving communication with mentors to coordinate better the projects. About the problems encountered by the academic supervisor, the most common issues were:

• Not aligned with the semester. Unavailability of mentors. • It's complicated do the agreements with the institutions. • No access to required information (code and data). • Marking the projects as solved and the process to assign the project to a

student Again, in this case, the academic supervisors expose the problems contacting to the mentors and the timeline issues with the course. Also, they outline issues signing the agreements to recognize the placements within their institutions, and finally issues accessing code and marking the projects as solved in the Virtual Placement System. About if the academic supervisors enjoyed the experience, all of them (100%) opined that they enjoyed the experience, and they would like to take part again in the Semester of Code. As in the case of mentors, academic supervisors were asked about what they learnt participating in the Semester of Code. In this case, they (briefly) feature:

• Remote placement is an interesting concept. • Administrative processes in universities are too strict and change among

different universities across Europe, which make harder the placement recognizing in all of the participant institutions

• Communication with different people. Nice experience for the students. • Learning to work in international teams

And finally, the mentors would like to do in a different way in their future participation in this program: plan communication rules with the mentors, track better the mentorized project and be more engaged in the placement.



4. Conclusions To finalize this deliverable, some conclusions are presented in order to comment the main results gathered during the evaluations pre-placement and post-placement. In the case of the pre-placement evaluation, 50 students participated. In general terms, their academic institution informed them, but only 60% of students visited the websites and 43% reviewed the projects list. This lead to that only 14% asked for access to the platform and finally 5% of them composed a proposal for projects. Of this 5% of student, 29% got accepted the proposal and 40% of them started to work in the project. Regarding to the communication with the mentor, 67% of students declared that the communication was excellent, while other 33% stated a poor communication. In the scope of the reasons provided by students to not participate in the Semester of Code, the 5 featured reasons were “too little time for applying”, “inconvenient timeframe for project execution”, “insufficient details on Semester of Code”, “lack of monetary compensation” and “concerns about communicating in English”. Regarding to the three main reasons, it is clear that must be solved improving the communication and dissemination processes in the initial stage of the Semester of Code as well as enhancing the process to sign up and the timeline to participate. As stated previously, the Semester of Code partners are trying to mitigate these issues, at least in the case of the timeline, modifying in part the Semester of Code workflow removing the constraints in the dates and project timelines. Also in the case of communication, the partners are encouraging universities to carry out more presentations and dissemination of the project to the student, but finally it depends in each institution, the professors and staff involved in them. About the other two reasons, they escape out of the focus of this project. In the case of the lack of monetary compensation, VALS project and Semester of Code was designed using the idea of establishing alliances through win-win strategies, and providing specifically academic rewards to the student, so this reason cannot be mitigated in the current scope of the project. Regarding the concerns about communicating in English, the issue is also out of the scope of the project, it has been observed as a local problem in some countries (Spain for example), and should be solved by the academic curricula of each country. In the case of the post-placement evaluation, in general terms the completion rate of placements was among 50-58% (depending on the number of students, mentors and academic supervisors that respond the questionnaire). This completion rate percentage is much lower than the success rate of other similar initiatives like the Google’s Summer of Code (80-89.7%). Despite of the difference the in success rate, is observed that in this kind of programmes the success rate is increased over the time, and in the case of the Semester of Code the second round of placements was more successful than the first one. Nevertheless the success rate, all of stakeholders involved (students, mentors, academic supervisors) express a good satisfaction with the overall experience in the placement, the students rate it with a 3.58/5, mentors with 3.38/5 and academic supervisors with 4.14/5. In general terms, any of them rate the experience below the mid value -3-. Among the relationship and experience among the stakeholders, the opinion of each one to the others is also good. In the case of the students, they rate the experience



with the mentors with a 4.0/5, and with 3.92/5 in the case of academic supervisors. From the point of view of mentors and academic supervisors, they rate only their experience with the student, not with other mentors or academic supervisors. Regarding the experience with the student, mentors rate it with a 3.69/5 and academic supervisors with a 3.79/5. Again, the averages of the experiences are located in the positive part of the scale (1-5, neutral value= 3), and dodging some bad ratings –bad experiences among stakeholders- due all the placements were not completed and other negative contexts, in general these assessments of the experience can be considered quite good. Other main issue reviewed in the questionnaires is the relationship among the tasks carried out in the project and the knowledge and competencies students acquire during the studies. In this case only students and academic supervisors answered this question. The students rate it with 2.92/5 value and academic supervisors with 3.93/5. It is intriguing how the students feel that almost the work done in the placement is not aligned with the knowledge and competencies they are acquiring in their studies (the value is below the mid mark in the scale), while the academic supervisors think they really have a clear relationship. The explanation for that difference, from the point of view of VALS partners, is that students are rating the relationship regarding to technologies, concepts and so on which are often different to the knowledge they acquire in the University; while the academic supervisors rate it more focused in the competencies (professional or not) they are acquiring as well as the knowledge in a more theoretically (rather than practically) way. About the Semester of Code and the future career of students, there were presented several questions related to. The students think moderately that these placements help them to plan the future professional career (3.42/5) and are not truly confident that it will help to get a job in a near future (3.17/5). In the same way, the mentors think that the placement could help the student in the future professional career (rate of 3.31/5). Finally, the academic supervisors are the most optimistic stakeholders regarding this question; they rated this question with a 4.07/5 mark. In their opinion it surely will help in the students’ future professional career. Related to Semester of Code and work, only 1 student got a job offer related to the placement carried out (1 student declared it). Despite of that, 69% of mentors declared that they would recommend to their company/project/foundation to hire the student involved in the placement, or even that, 85% would recommend the student to be hired by other company/project/foundation. In general terms, it is possible that the Semester of Code, despite of the questionnaire responses regarding to what stakeholders think about if the placement will help in the future professional career, will help really them to find a job (regarding for example the mentors’ opinion). Other relevant aspect in the VALS project is the Open Source philosophy/principles, in this case, 75% of students, 69% of mentors, 93% of academic supervisors think that the students participation in the Semester of Code helped them to develop a positive attitude regarding to FLOSS community and projects. So can be assumed that this initiative help in general to enhance the feelings and relation of students and Open Source. About the students’ competencies demonstrated during the placements, mentors and academic supervisors rate them quite well (3.46/5 mentors, 3.78/5 academic supervisors). Among the 14 competencies and skills reviewed by the mentors and academic supervisors, the 3 best valuated by mentors were oral communication skills



(4.08/5), written communication skills (4.08/5) and with equal values students’ ability to learn (3.62/5) and receptiveness to criticism (3.62/5); while the three best valuated by academic supervisors were students’ ability to learn (4.21/5), students’ technical ability (4.0/5) and responsibility (3.86/5). In the case of the worst valuated, in the case of mentors the three worst were personal involvement (3.08/5), capacity for team work (3.08/5) and students’ tasks management skills (3.23/5); in the case of academic supervisors those were the adaptability (3.57/5), motivation (3.57/5) and with equal evaluation (3.64/5) the oral communication skills, punctuality and capacity for teamwork. About how easy students, mentors and academic supervisors find it to get involved in the Semester of Code, students rated the easiness with a average of 4.0/5, mentors with an average of 3.08/5 and academic supervisors with 4.36/5. In this case, the mentors are the most critical stakeholder, possibly due most of them participated in other similar initiatives like the Google’s Summer of Code, which is the most representative and clear: it exists for more than 10 years, in comparison to the first year of the Semester of Code and its different changes and approaches tested during the pilots. Regarding this issue, all the stakeholders propose different approaches and improvements that can be incorporated to the Semester of Code. Regarding to if the stakeholders enjoyed their participation, all of them agreed in responding yes. 75% of students enjoyed to participate, 92% of mentors also enjoyed, and finally the 100% of academic supervisors also opted for it. About if they would like to participate again, the “yes” option was the most common, with 58% of students, 59% of mentors, and 100% of academic supervisors. To finalize the questionnaires, the stakeholders involved commented also what they learnt participating and what would they do differently if they participate again. These comments, as well, as the other gathered will be very valuable to improve the Semester of Code in future editions.



5. Acknowledgements This publication only reflects the views of the authors, and the Commission cannot be held responsible for any use that may be made of the information contained therein.

6. References [1] F. J. García-Peñalvo, I. Álvarez Navia, J. R. García-Bermejo, M. Conde- González, A. García-

Holgado, V. Zangrando, et al., "VALS: Virtual Alliances for Learning Society," in Proceedings of the

TEEM’13 Track on Knowledge Society Related Projects, F. J. García-Peñalvo, A. García- Holgado, and J.

Cruz-Benito, Eds., ed Salamanca, Spain: Grupo GRIAL, 2013, pp. 19-26.

[2] F. J. García-Peñalvo, J. Cruz-Benito, M. Á. Conde, and D. Griffiths, "Virtual placements for

informatics students in open source business across Europe," in 2014 IEEE Frontiers in Education

Conference Proceedings (October 22-25, 2014 Madrid, Spain), ed USA: IEEE, 2014, pp. 2551-2555.

[3] F. J. García-Peñalvo, J. Cruz-Benito, D. Griffiths, P. Sharples, S. Willson, M. Johnson, et al.,

"Developing Win-Win Solutions for Virtual Placements in Informatics: The VALS Case," in

Proceedings of the Second International Conference on Technological Ecosystems for Enhancing

Multiculturality (TEEM’14), F. J. García-Peñalvo, Ed., ed New York, USA: ACM, 2014, pp. 733-738.

[4] García-Peñalvo, F. J., & Cruz-Benito, J. (2016). Proyecto Europeo VALS y Semester of Code:

Prácticas Virtuales en Empresas y Fundaciones relacionadas con el Software Libre a nivel Europeo.

In M. Martín-González (Ed.), Buenas Prácticas sobre la Universidad Digital (pp. 60-67). Madrid:

Cátedra UNESCO de Gestión y Política Universitaria.

[5] F. J. García-Peñalvo and J. Cruz-Benito, "Proyecto Europeo VALS y Semester of Code: Prácticas

Virtuales en Empresas y Fundaciones relacionadas con el Software Libre a nivel Europeo (Versión

póster)," presented at the Seminario Bienal "La Universidad Digital". Taller de Buenas Prácticas:

Presentación de Experiencias, Madrid, 2015.

[6] F. J. García-Peñalvo, J. Cruz-Benito, M. Á. Conde, and D. Griffiths, "Semester of Code: Piloting

Virtual Placements for Informatics across Europe," in Proceedings of Global Engineering Education

Conference, EDUCON 2015. Tallinn, Estonia, 18-20 Mach 20015, ed USA: IEEE, 2015, pp. 567-576.

[7] F. J. García-Peñalvo, J. Cruz-Benito, D. Griffiths, and A. P. Achilleos, "Tecnología al servicio de un

proceso de gestión de prácticas virtuales en empresas: Propuesta y primeros resultados del

Semester of Code," IEEE VAEP-RITA, vol. 3, pp. 52-59, 2015.

[8] VALS European Project. (2015, 10/1/2015). Semester of Code User Guide. Available:

http://vps2.semesterofcode.com/sites/all/modules/vals_soc/help/index.php



[9] F. J. García-Peñalvo, "Entrepreneurial and problem solving skills in software engineers," Journal of

Information Technology Research, vol. 8, pp. iv-vi, 2015.

[10] F. J. García-Peñalvo, J. Cruz-Benito, D. Griffiths, and A. P. Achilleos, "Virtual placements

management process supported by technology: Proposal and firsts results of the Semester of

Code," IEEE Revista Iberoamericana de Tecnologías del Aprendizaje (IEEE RITA), vol. 11, 2016. doi:

10.1109/RITA.2016.2518461.