Tag Archives: Metrics

… how government can build the capacity to make the most of Grand Challenges

What is the role of government in solving really big problems? Is it the government’s place to set the research agenda for the private and NGO sectors through grand challenges?

How can we facilitate open and transparent sharing of ideas and solutions between the individuals, organizations and broad sectors engaged with grand challenges?

Can we develop a simple test or set of metrics to determine if a given problem would be a strong candidate for a grand challenge?

Design and Metrics

How should government agencies decide whether to use a grand challenge model instead of a more traditional form of R&D?

What unique features differentiate grand challenges from prize-induced contests, hackathons and other forms of citizen engagement?

Grand challenges, by definition, are focused on achieving audacious goals. Could they be more effective if they gave participants more guidance on achieving gradual milestones? Is it practical for agencies to develop a type of timeline to help guide the progress of participants and stakeholders?

How do we create real-time visualizations or iterative metrics to convey the impact of the process to challenge participants (i.e., keeping a two way structure where people who draft challenges can be kept in the loop and know of their progress/ adapt models iteratively)? How can the structure of challenges be more closely aligned to an iterative policy progress, taking into account small, incremental changes?

How much pre-planning and structure should accompany grand challenge initiatives? Considering the likely involvement of government in solving multi-stakeholder challenges—unlike other open-innovation projects—would it be beneficial for partnerships to be made before the official launch—for instance, to attach a well-known, respected private-sector partner’s name to the initiative?

How should government agencies develop the internal teams charged with shepherding grand challenges from launch to completion?

Should grand challenge projects incorporate any best practices from other non-competitive collaborative innovation programs like hackathons and idea “boot camps?”

How do government grand challenge initiatives compare—in terms of success and participant engagement—to similar public-innovation programs in the private and NGO sectors?

What kinds of hybrid activities could be developed combining grand challenges with other types of citizen engagement initiatives? For instance, would it be effective to combine the framing and rhetoric of a grand challenge with the activity of a solver network?

Participation

Despite grand challenges’ focus on engaging many organizational stakeholders, are individuals working toward solutions in these areas, either as part of a network or individually?

Prize-induced contests are often associated with small start-ups. Are the businesses engaging with grand challenges similarly small and/or newly created, or are more established businesses and NGOs more likely to be working on the challenges?

Are colleges and universities working to solve grand challenges? If so, are the efforts largely faculty-based or student-based? Have any research centers or courses been launched to work toward solving grand challenges?

Impact

What effects have grand challenges had on workforce development? Are individuals learning valuable new job skills through engaging with the challenges? Are new businesses gaining traction following their participation? Are businesses—new or established—gaining consumer goodwill through their participation in programs that benefit the public good?

Are grand challenges being used as a means of government procurement? Do businesses engage with them as a type of audition for government contract work?

Funding

Considering the scale of grand challenges, most initiatives are unlikely to yield immediate results. Would the government be better served devoting a small amount of money to traditional R&D instead of largely relying on the private sector to deliver results, without much in the way of gradual, observable progress?

Incentives

In general, grand challenges do not offer the same type of direct monetary award available in prize-induced contests. Is there enough incentive to draw in private actors considering the scope of the challenges and the financial and time commitments necessary to work on them, let alone complete them? Are audacious goals to benefit the public good enough?

Does the incredible scope of grand challenges’ goals overwhelm potential participants and act as a barrier to participation? That is, are grand challenges too grand?

Can a relatively informal Web-based call to action be effective in inspiring multi-stakeholder collaboration to address grand challenges in the same way that it can be effective in inspiring individuals or small groups to create new solutions to more tractable problems?

Does the multi-stakeholder nature of grand challenges lessen the competitive spirit of those participating or considering participating?

Considering the reputation-building potential of altruistic public-sector innovation efforts for private businesses, can governments spur private engagement with grand challenges through a focus on the potential for positive media attention and consumer goodwill?

Gavin Newsom and Code for America partnered to launch the Citizenville Challenge, “a call for local government leaders to commit to working toward a government for the 21st century.” Can grand challenges—with no concrete prizes or rewards—be effective when exclusively focused on increasing the innovative capacity of government itself?

Evaluation

How should grand challenge submissions be judged? Is the type of relatively straightforward judging panel that is typically used in government contests a practical solution for evaluating submissions and proposals for less-concrete, more audacious grand challenges?

How can the evaluation process be developed to ensure that innovation—rather than ease of deployment, cost and other practical concerns—is the central metric upon which submissions are judged?

If a judging panel is the ideal means of evaluation, who should sit on the panel? Members of relevant government agencies? Influential members of the private sector? NGO workers with a vested interest in the challenge area?

Considering the lack of prizes being awarded for grand challenges, and in the interest of not alienating participants, is there a way to ensure that all relevant stakeholders and collaborators receive recognition and benefits from a successful grand challenge submission?

How can participants whose submissions are not accepted be inspired to remain engaged with the challenge? Can the evaluation process act as an opportunity for feedback, rather than as the final judgment as to whether a proposal is satisfactory or not?

… how government can develop the capacity to make the most of Prize-Induced Contests

Can we develop a simple test or set of metrics to determine if a given problem would be a strong candidate for a prize-induced contest?

Design Best Practices

How can we facilitate open and transparent sharing of ideas and solutions between the individuals, organizations and broad sectors engaged with grand challenges?

How should government agencies decide whether to use a prize-induced model instead of a more traditional form of R&D?

What unique features differentiate prize-induced contests from grand challenges and other forms of citizen engagement?

Do new or overlapping prizes dilute the effectiveness of others?

When is the best time to create or discontinue a prize?

What are appropriate objectives for a prize? And what is it that makes a prize effective at achieving them?

Are there certain types of prize structures that are better able to create a community of problem-solvers—both in the interest of winning the prize and beyond?

Contests often provide large-scale goals, but often lack more gradual milestones. Could they be more effective if they gave participants more guidance on how to progress through the contest?

Should prize-induced projects incorporate any best practices from other non-competitive collaborative innovation programs, like hackathons and idea “boot camps?”

Many prize-induced contests feature both public and private sponsors. How do different types of sponsorship affect results and engagement? Do contests sponsored by both the public and private sector address broader issues like the multi-stakeholder grand challenges?

What design best practices are most effective in eliciting contest submissions with value to consumers?

Participation and Incentives

The stereotypical public participant in prize-induced contests is the retired or under-employed person working in a garage. Is this actually the case? How involved are start-ups and established businesses in contests? Are universities seeking to increase student and faculty engagement in the programs?

Can we determine who does not actually participate in a contest by looking at who enters for the first round and doesn’t advance?

Do prizes create incentive for individual effort rather than teamwork within or across institutions?

How do government prize-induced initiatives compare—in terms of success and participant engagement—to similar public-innovation programs in the private and NGO sectors?

Challenge.gov features many contests that are focused on science and technology, as well as many addressing social and public policy. Are more people getting involved in the technical challenges even though many social and policy issues challenges likely have a lower barrier to entry and require less-advanced knowledge and skills? And if so, why?

Are people involved in lower levels of government engaging with policy issues challenges the way people in tech industries are likely engaging with the technical challenges?

Recently, New York Mayor Michael Bloomberg’s foundation “launched a $9 million contest inviting mayors to come up with the best ideas to tackle urban problems.” Can prize-induced innovation contests be successful when targeted exclusively to people within government?

Public and Private Impact

Is there any guarantee that a prize for new ideas will stimulate commercialization and widespread adoption critical to improving lives on a large scale?

What effects have prize-induced contests had on workforce development? Are individuals learning valuable new job skills through engaging with the contests? Are new businesses gaining traction following their participation? Are participating individuals and companies gaining government employment or contracts following the contests?

Funding

While prize-induced projects have clear ties to other forms of crowdsourcing, is there a place for crowdfunding in these projects—both in terms of supplying the prize money and stimulating private engagement on expensive projects?

What makes prize-induced projects preferable to the use of social impact bonds, which have many similarities but rely on private sector financing rather than taxpayer money?

Is the pay-for-performance system currently used by some healthcare providers another possible guide for government agencies to draw on public innovation without upfront costs? Could such a program be instituted without contracting the pool of “unusual suspects” working toward solving problems?

Evaluation

How should grand challenge submissions be judged? Assuming that a panel is used, who should sit on it? Government workers from relevant agencies? Influential members of the private sector? NGO workers with a vested interest in the contest?

How can the evaluation process be developed to ensure that innovation—rather than ease of deployment, cost and other practical concerns—is the central metric upon which submissions are judged?

Assuming a winner is not found, can the evaluation process act as an opportunity for feedback for participants, rather than as the final judgment as to whether a proposal is satisfactory or not?

Post-Contest Concerns

Should contest participants—including individuals and private businesses—or government agencies hold the intellectual property rights to innovations created as part of government contests?

What are the best practices for data sharing, both between participants and after the contest ends?

A common complaint regarding prize-induced contests is the lack of post-contest engagement with developers. How can government agencies ensure that new innovations do not become obsolete due to lack of attention, especially considering one of the central reasons for such contests is lessening the workload of government employees?

… how to respond to the basic tension between using the instruments of representation, what we think of as the instruments of self-government, and modes of participation.

By David Cohen, Senior Advisor, Civic Ventures; Senior Congressional Fellow, Council for a Livable World

Here are my questions that stem from my experience at initiating, formulating and implementing public policy which has lots to do with policy and politics.

A few words of caution. As important as elections are–after all it is one act that we all engage in as a people– they are not what I have in mind. I say that believing the equality of the vote in participating and counting results is a basic value that is currently undermined in countless ways.

The present systems lead to significant exclusion. How is that overcome? That leads to another aspect of the underlying question: how do we get beyond the current interest group models so that other mediating institutions or approaches will organize, be heard and compete.

I am also not talking about participation as the equivalent of a large audience in the high school gym which may have as its purpose a protest, a rally or even begin to lead to engagement. But the high school gym assemblage does not lead to sustained participation.

Differences on policy and politics are inevitable. How are agreements reached and accepted so people feel that they have been listened to, have had their say and perhaps have had a partial effect.

Questions:

How can representation be strengthened so it fosters participation?
Is there room for random consultation as in a citizens’ jury. Can that
happen on highly technical questions. If not, how is expertise made understandable so that people don’t withdraw from civic engagement.

How do we educate people–and not limited to high school students– in the life cycles of issues so that their participation is not delegated only to experts. That kin of education develops participation and in the process strengthens representation.

Given the 24/7 news cycle, and the viral capacity of triggering fears that frighten decision makers, what are the immunization and inoculation steps that need to be taken to allow deliberation to breathe? Think death panels in the health care debate.
How can officialdom experiment, value its efforts at learning and learn from mistakes without being hung out to dry. That is where immunization and inoculation techniques come into play.

We value robust debate. At the same time civil discourse has to reject words that wound–racial, religious and ethnic epithets and stereotypes.

Critical thinking should be valued. What are its elements? What do we know when and how people are listened to? There is more to listening than decibel levels. Are there examples? Yes, I participated in its emergence in tobacco control efforts in the earliest internet days, nearly 30 years ago, of ordinary people fighting against a behemoth, the tobacco industry with the deepest pockets. I saw it through the use of the fax on the pro-democracy world that challenged totalitarianism in the Communist countries of east and central Europe.

Public work leads to specific outcomes:

(a) in education that engages the professions but the students and their parents as well.

(b) in health care that engages the professionals as a team and people accepting their own responsibility for healthy, but not monastic, life styles.

(c) in environmental protection and neighborhood life dealing with sorting out the issues of usage (parks, recreation, shopping, size),
housing (lot size, what mixes, shopping and parking), transportation
(what flexibilities via biking, zip car, shuttle service).

How do we elevate such work so that it known as “public work,”
a naming by Harry Boyte in his writing. Public work bridges, but does not avoid the tensions between representation and participation.

What are the ways to learn what has worked, what hasn’t, where improvements lie. This surely greater than a task for evaluators, useful and necessary as they may be.

Underlying the tension between representation and participation is the tension between the experts and informed lay people. They have to learn from each other.

How do they learn? In what ways?

 

… how to incorporate evidence-based decision-making in our governance structures to change the status quo.

By Mihaela Ulieru, President, IMPACT Institute for the Digital Economy; and Director, Adaptive Risk Management Laboratory

What are the essential questions that we, as researchers and practitioners, need to pose or haven’t even considered yet?

Due to the lack of adequate policy frameworks, the obstacles in implementing innovative solutions at all levels are the limited capacity of social processes to manage rapid change in institutional design, planning and public services, rather than technological innovation. If only we knew how to incorporate evidence-based decision-making in our governance structures to change the status quo.

–          How to revolutionize our coercive and disabling governance structures and enable the transition to agile, participatory responsive social network operating systems which foster creativity and enable innovation?

–          How to design validation frameworks that reveal the impact of policies on the work ethics, culture and productivity in our organizations?

–          How to increase people’s work productivity via engaging mechanisms that stimulate rather than oblige, transitioning the current work organization processes from contract to commitment by fueling performance through visceral engaging architectures of participation which infuse a blissful, epic-like meaning to the purpose of work?

–          How to enable remote access to education via eLearning strategies facilitating personalized curriculum targeted at each student’s career goals via online course enrolments and credits?

–          How to enable eHealth and mobile health solutions, empowering patients to get engaged and be accountable for their health?

–          How to enable the personalized medicine revolution to transform our medical system? How to make room for the latest genomic discoveries in medical practices?

What do you think an interdisciplinary open government research network should look into?

How can we find ways to anticipate the effects of a policy before it is applied, using latest advances in predictive analytics and the wealth of data that modern information communication technologies provide? How can we accelerate the adoption of informed, evidence based governance using this wealth of data?

Are there broader categories of concern that have not yet received the requisite research attention?

Taken together, interconnected grids of communication, electricity, and transport integrated with the social fabric of citizen participants will undergird development of this century’s energy-efficient and sustainable living, hosting the institutions and technologies of transformed low-carbon economies. If only we knew how to understand and advance the interweaving of humans and ICT to create a world with social, physical, and cyber dimensions, enabling a kind of social network operating systems with citizens as “players,” and “inputs” whose interactions use the “invisible hand” of democratic informed choice to address complex, interdependent global-scale challenges such as sustainability and climate change.

–          How to enable the deployment of such socio-technical infrastructures while supporting the transition to a post‐manufacturing, innovation- and knowledge‐based green economy and society?

–          Which governance models and structures are appropriate for such socio-technical infrastructures? What is the role of knowledge governance and social computing in this broader context?

–          What are the principles of management and engineering that will infuse these emerging socio-technical complex large-scale systems with the ability to discover a variety of potential solutions in the collective intelligence repertoire to meet future challenges as they arise?

–          How to redefine indicators that expose the impact of the convoluted effects of interdependent socio-political-economic factors on the current global dynamics, negatively affecting the overall wellbeing and sustainability of life on Earth? How to define new indicators of wealth and social well-being in a global context?

…how innovations in science and technology change the way we work together to enhance our well-being.

By Beth Noveck, GovLab Continue reading

…what the economic, political and social impact of increased transparency is?

By the Transparency Working Group @ the London Summit on Re-Thinking Government, Re-Imagining Democracy Continue reading