Ewaluacja PO IG – Priorytet 7

What is your overall impression of Priority 7:

• a) it should be abandoned
• b) it should be corrected
• c) it should be continued as it is now

if b) – what should be changed?

Our overall impression is that it should be corrected. The programme?s aims are worthwhile, but it is not clear that the initiative will be able to successfully deliver on these aims. See recommendations suggested in Question 5.

Is the goal of P7 accurate and coherent with the rest of the Programme?

• a) definitely yes
• b) rather yes
• b) average
• d) rather no
• e) definitely no

DEFINITELY NO.

The programme details several action lines for the better use of technology (computerization, e-services platforms, infrastructure, etc.), but doesn’t mention the benefits of citizen centricity, take up, etc.  Similarly, it details the technology to be used, but lacks the systems and business processes to effectively take advantage of the technology.  In many ways, the technology is the easy part, and putting in place the frameworks, structures and processes to manage collaboration within government is much more difficult.  The European Commission estimates that IT costs are only 45% of the total cost of running a successful information society initiative, and organisational change and the accompanying business processes require 55% of the investment.  But the current proposal does not talk about the actions required for organisational change at all?

The section on metrics has a similar problem – Governments are increasingly calling for robust cost/benefit measurement methodologies to justify investments in ICT, but the metrics in the current initiative do not reflect this. Several governments have issued more comprehensive multi-dimensional measuring methodologies, in particular, the European Commission eGEP study has put forward a new multidimensional eGovernment Measurement Framework (eGMF).  This multi-dimensional approach has been further refined and is used by gov3, which is based on several type of both quantitative and qualitative metrics:

• Hard cash value, measuring short term savings in terms of avoided costs
• Potential monetary value, assigned to impacts mostly deriving from increased productivity
• Volume metrics, used to measure impacts for which is not possible to assign a monetary value, but which can measured in numbers (for instance „decreases in numbers of security breaches”)
• Qualitative scores, assigned to strategic impacts (for instance „contribution of an IT investment to agency capacity to meet national policy requirements”)

At the very least, metrics should focus on the realisation of benefits rather than supply-side metrics (e.g. looking at take-up of e-services than on the number of services available online).  If you would like us to detail this further, the expert who led the European Commission eGEP study, Cristiano Codagnone, is now part of the Gov3 team, and I?m sure he would be happy to provide you with additional information on metrics.

Are the financial resources for Priority 7 adequate to the planned activities?

• a) definitely yes
• b) rather yes
• b) average
• d) rather no
• e) definitely no

It is difficult to answer.  The main problem we see is not with the budget, but rather with the design and implementation of the initiative.  There is a high risk that the initiative in its current form would not deliver on its objectives – but additional funds would not ensure success.  It is rather more important to revise the initiative, and then create a detailed roadmap that demonstrates the main action lines, workstreams, timeframes, deliverables and dependencies.  This roadmap would clarify what could be done within a specific budget – and strong programme management would ensure that deliverables are on target.

Are the groups of activities proposed in the Priority 7 justified from the point of view of its goals and Programme’s goals?

a) „Digital Poland for All”:

• a) definitely yes
• b) rather yes
• b) average
• d) rather no
• e) definitely no

Rather no.  See comments under question 5

b) computerisation of administration:

• a) definitely yes
• b) rather yes
• b) average
• d) rather no
• e) definitely no

Rather no.  See comments under question 5

WHAT WOULD BE YOUR ADVISE TO THE POLISH POLICY-MAKERS CONCERNING PRIORITY 7?

Particularly on a concept ?Digital Poland for All? and computerisation of administration.

Digital Poland for All:

Digital Inclusion is an essential ingredient in the transformation to an information society, so we agree that a Digital Poland initiative should definitely be part of the programme.  However, the programme in its current form is missing key ingredients.  Notably, in order to succeed, we believe that any digital inclusion initiative must have the following:

• A detailed understanding of the barriers to digital inclusion – what exactly is preventing citizens and businesses from using technology, and what changes would influence their take up of technology?  For example, with regard to boosting entrepreneurship, is it a demand side problem (e.g. barriers to innovation), or supply side (e.g. lack of a venture capital market)?  For citizens is it because they don’t see the benefit of using technology, because it?s too expensive for them, because they don’t have the skills they need, or because there isn’t enough local content available to them?  A successful initiative will be designed specifically to address the barriers identified in a particular market, and include all the factors of digital inclusion, including Access, Motivation, and Confidence.
• To work successfully, a digital inclusion initiative should focus on partnership and enabling the market. This includes involvement of ICT vendors, digital content and service providers, voluntary and community organisations and employers – as well as various different branches of the public sector (e.g. central and local government, industry, education, etc).

You can find more information about critical success factors for digital inclusion in our white paper on the subject, available at: http://public.gov3.net/public_pages/limited/global/resources/white_papers/white_papers/achieving_diigital_inclusion.htm

Computerisation of Administration:

As per our comment under 2 above, we believe that this priority is somewhat off course. It does not take advantage of the learnings and mistakes that have been made elsewhere.  The real benefits of e-government are not so much from the „Computerisation” of public administrations as they are from their „Transformation”.  Organisational change, structures for collaboration and partnerships, and joined-up service delivery are all key – yet are missing from the current proposal.

RECOMMENDATIONS:

Our primary recommendation is that in order to successfully take this forward the government needs to build its capacity to deliver e-government and information society initiatives.  This could be done in-house or working with a partner, but in either case it is essential that the government work with people who have a track-record of delivering similar programmes in government.  This is a „soft cost” – and to provide a reference the European Commission eGEP study estimates that 75% of total costs are on these sorts of organisational assets (compared to 10% on hardware and 15% on other ICT complements).  It’s not enough to understand technology or e-commerce, since the unique requirements of government are such that the most difficult challenges are likely to be organisational.

Additional suggestions include:

• Running an outside „health check” of the initiative to review what has been done to date and provide concrete recommendations for correcting the initiatve.
• Re-think the programme portfolio so as to ensure that the main action lines will deliver on the stated goals of the programme. This „portfolio optimisation process” would include the examination of initiatives based on strategic fit, user benefit, government benefit, and do-ability.  The process would also balance early benefits (or „quick wins”) with longer-term benefits, costs and risks.
• Develop a detailed roadmap that delineates actions in the short and medium-term, and what exactly needs to be done in order to ensure that the programme delivers on its stated goals.

DO YOU HAVE ANY ADDITIONAL COMMENTS, THOUGHTS ABOUT THE PROGRAMME OR THE PRIORITY?

We ran an initial evaluation of the risk involved in the programme using the Gov3 risk barometer, and this demonstrates very high levels of intrinsic risk (see below).  Because the risk is so high, strong governance and risk management will be absolutely essential in making this work.  Yet the governance arrangements described in the document are fraught with problems.  We suggest that getting the governance structures in place and the levers and incentives aligned is an absolute must before taking this forward.

The Government can also use the Gov3 Risk Barometer tool for a self-evaluation, it’s freely available – CLICK HERE.