FINODEX 1st open call - statistics and questions
Opened in October 2014 and closed the 19th of December with the reception of 196 proposals from all over Europe. The following information is published as an overview of the process of the call as well as clarifying some of the questions we have received afterwards. The numbers and statistics of the call are made public for the audience to know how we work and select any proposal. Please use this information in your own good for the next second open call or for whatever reason you may have. The overall selection of proposals has been divided in a very clear step-by-step process:
Submission of proposals
It was done through F6S Platform which basically controlled that everyone uploaded the three required documents and fulfilled a basic questionnaire for statistical purposes. Time limit, which was 15PM (CET) was also controlled by the system, making impossible to anyone to upload a proposal after the time passed. The graph shows the number of proposals opened and submitted during the time the call was opened. 356 applications were opened and 196 finally submitted which makes a rate of 55.06%. Our first conclussion, though we already believed it, was that the vast majority of proposals were received within the last 48 hours.
Selection of evaluators
This was part of the call as well. In case you did not know. The proposals have not been evaluated by the FINODEX partners at any point. An open call was opened and published in our typical channels (website, newsletter and twitter account), in order to get a good number of experts for the evaluation. The selection of those experts was done among the partners within FINODEX consortium following a very easy approach.
- The evaluators could not be employees from FINODEX partners, nor EC workers, nor proposers or having a conflict of interest.
- Every potential reviewer was marked in four criteria by every partner: experience as a reviewer, open data, business consulting and FIWARE knowledge. The final mark was the result of the average of all the partners' to avoid anyone imposing the rest a reviewer.
- Two different rankings were done: one sorted out by the FIWARE knowledge criterion and the other one with the other three. Reason for populating two different lists was because it is compulsory for the FIWARE Accelerators to have specific experts on FIWARE reviewing each proposal and we decided to assure it this way.
In principle 10 evaluators (5 from each list), were pre-selected and informed. The final number, 8, took into consideration the final number of received proposals and the internal rules for evaluation: every proposal at least by 2 evaluators (one from FIWARE ranking, another from the other ranking) and no more than 50 proposals per reviewer.
The evaluators will evaluate all the phases of the acceleration and their details will be kept anonimous during the whole process to avoid any kind of external inference. In fact, they have signed a contract which does not allow them publishing they act as evaluators for FINODEX accelerator. A total of 45 candidates presented their CV divided per countries as seen below. The overall list of the potential candidates as well as their marks was sent to the European Commission. The 8 selected ones come from 4 different countries.
The evaluation of proposals
The explanation of the process was indicated in the Guidelines for Applicants of the call. So have a deep look into the details related to the four criteria, the thresholds and that kind of issues. 196 proposals were submitted, with the following country distribution:
Before evaluation there was a elegibility check which arised some surprising results for us, 38 proposals were not eligible for the following reasons:
So, our second conclussion was that we need to improve in our future presentations the presentations about the "open data" requirement. This is one of the reasons that have made us launch an open data survey to our newsletter subscribers recently. We want to adjust and improve our training materials, FAQ section and communication in the infodays about this matter. It is our intention to improve and avoid having so many non-eligible proposals for this reason in the second open call.
The numbers below show a comparison between the countries of the submitted proposals, submitted & eligible proposals and submitted non eligible proposals.
After the eligibility check was carried out, the 158 eligible proposals were passed to the external evaluators. Each proposal was reviewed by, at least, two evaluators. One of the evaluators is a FIWARE expert, while the other is a General expert.
- FIWARE Evaluators marked all sections, being the weight for FIWARE section an 80% and a 20% for the others.
- General Evaluators marked all sections, being the weight for FIWARE section a 20% and an 80% for the others.
Proposals were given in groups of 25 to the evaluators. After evaluating this first set of 25, an average mark given by each evaluator was calculated and a ranking of evaluators was ellaborated. This ranking permitted passing the highest scored proposals to the evaluators with the lowest averages as a method to avoid disparity of criteria. In the case a proposal received a disparity of marks of over a 25% a third evaluator reviewed the proposal and the arithmetic average was done. The third evaluator was a FIWARE expert or a General one according to the pending evaluations per evaluator, their possitiveness and country of origin. Though it was not possible in all cases, given the high number of Spanish participants, we tried to avoid the evaluation of proposals from the same origin of the evaluators, as a way to avoid potential links or connections among proposals and evaluators. It is worth to say that no conflicts of interest were pointed out during the whole process. As a result of this evaluation, all the partners were informed about their situation:
- 50 proposals pre-selected.
- 16 proposals were over the thresholds in the reserve list.
- 2 proposals were marked as not eligible because of multi-participation in FIWARE Accelerators.
- 92 proposals were under the thresholds.
- 38 proposals were not eligible.
The negotiation of proposals
This phase consists on assuring the information provided by every proposer corresponds to reality (legal docs) as well as the signature of the contracts by the proposer. During the negotiation 3 pre-selected proposals drop-off. Two decided voluntarily to leave and another one was selected by another accelerator.
Only two new projects from the reserve list were called for negotiations, given the fact that there was a drawn by three companies in the list and there was not time, nor the legal framework, to define it, 49 projects have entered our Phase 2 for the 1st Open Call. The new call will include a procedure to sort out this kind of drawns.
Moreover, a list of the 49 is available here and also within the open dataset containing the results of the FIWARE Accelerators.
During the following weeks a new FAQ section for the second open call will answer more questions about the overall process and selection of proposals.