Invited Talk at University of Twente’s CuriousU Summer School

CuriousUI was invited by the University of Twente to speak at their annual “CuriousU” summer school, a combination of “classic” academic summer school and summer festival (university adminstrators, take note, it seemed like a lot of fun!).

I spend half a day with a group of students discussing: “When Wind Farms Cause Cancer – Public Perception of Engineering Megaproject Risks”. Topics included:

  • Megaprojects and risk: Why big engineering projects are hard to get right
  • Engineering view of risk: Playing by the numbers
  • Cognitive Biases: The evolutionary heritage of a 200’000 year old brain
  • Social Movement and Social Movement Organization: Why humans take collective action, and how we are being manipulated into it
  • Psychometric risk assessment: Numbers don’t matter for how afraid we are of something
  • Discussion and workshop: Industry, Politics, Media, NGOs –Who cares about an informed, fact-based dialogue? Should we be worried? What is our responsibility as the educated elite?

You can download the extended description of the session here.

Design Society Risk SIG creates “Must Read” list

I am just back from chairing the annual meeting of the Design Society’s Special Interest Group on Risk Management Processes and Methods. We had a very productive meeting at the ICED15 conference in Milano. The main result was to use our collaborative online literature database on risk management in product development and design to create a “recommended reading” list. Target audience is researchers transitioning into our field, particular PhD students, as well as established academics developing a new interest. Particularly noteworthy was that we identified a number of “blank spots” in our database, regarding for example psychometric risk assessment practices, as well as Robust Design related aspects. We have now begun the process of closing those gaps, as well as crowning 1-2 papers in each of our categories to the “recommended reading” status. We expect to finalize the activity until our next annual meeting.

In the news: Whitepaper on Engineering System’s Take on Complexity Management Published

The Department just released a news-item on the Whitepaper I co-authored with Christian Thuesen, Pedro Parraguez Ruiz, and Joana Geraldi. Am copying the text below.

PMI Complexity Management Whitepaper
Thought Leadership Whitepaper discussing an Engineering Systems Perspective on Complexity Management

Engineering Systems group publishes whitepaper for ½ million readers in prestigious forum

The world’s largest project management organization publishes Thought Leadership Whitepaper by Engineering Systems Group.

Last year the world’s largest project management organization with more than 500,000 members, Project Management Institute (PMI), approached the Engineering Systems Group in PSM asking for a Thought Leadership Whitepaper on Engineering Project Management.

A team from Engineering Systems Group, consisting of Josef Oehmen, Christian Thuesen, Pedro Parraguez Ruiz and Joana Geraldi put together the whitepaper that gives an Engineering Systems perspective of managing complexity in engineering projects, programs and portfolios.
“The purpose of the whitepaper is to stimulate discussion both within the PMI leadership team as well as among members regarding better ways to manage complexity”, says Christian Thuesen.

He is complemented by Josef Oehmen who explains that by covering an angle relatively unexplored by PMI the group expects not only to start fruitful discussions but also to add new perspective and methodology to the large international forum.

“Current practices are driven by a “bottom up” approach of collecting what works in industry, consolidating it, and communicating it to members (in forms of standards and guidelines). What is missing is a complementary “top down” approach from an academic perspective to help uncover “blank spots” in the current portfolio of practices, and stimulate innovation in management practices by developing novel approaches. The whitepaper provides such a top-down perspective, encouraging exploration of novel approaches to manage complexity, and operationalizing them in the context of project management.”

The whitepaper was published in April and presented in May at the PMI Global Conference in London to a crowded room and was very well received by project management practitioners. PMI has published the whitepaper as a key resource on complexity on its homepage. It can be downloaded onPMI’s Website.

The whitepaper is written for practitioners and contains the following:

  • It provides an introduction to a system-oriented view of complexity,
  • It illustrates three key drivers of complexity (organizational and technical complexity as such, human behavior, and uncertainty),
  • It highlights a range of novel complexity management approaches that are researched and further developed by the Engineering Systems Group, such as Network Analysis, System Dynamics, Modularity, Antifragility and Mindfulness.

Would I have believed myself? On evaluating the quality of reports on topics that one does not know a whole lot about

I posted a guest column on Barry Brook’s blog Brave New Climate on March 29 that I reproduce below (visit Barry’s site for his introduction). Also see the page on my 15 minutes of fame for background.

On Sunday, March 13, my cousin in Japan posted an email I had written to him on his blog in the early morning at 3am EST. The email explained the context of nuclear physics and engineering, as well as discussed the events at the Daiichi-1 reactor until that point. It also featured my very strong opinion that they are safe. By lunchtime, it was the second most twittered site on the internet (you can read the whole story at http://bit.ly/e1It0T). At the end of the day, it had been translated into more than 9 languages (often multiple times), and after 48 hours had been read by several million people. Two weeks into my unwanted and luckily rapidly cooling off Web 2.0 stardom, I have begun working through the trauma and reflecting. Thanks for sharing, you might think. But one question in particular came up that also has some general relevance:

Would I have believed myself if I came across that blog and had no prior knowledge of nuclear physics and engineering? Or asked another way: How do you judge the quality of TV, radio, print and internet news reporting on topics that you are only superficially familiar with?

Read the answer below. And like everything I write, it is rather lengthy!
Working in an interdisciplinary field as an academic, it is often necessary for me to judge the quality of information from areas outside my core expertise and decide whether they are reliable sources worth studying. Also, when you work with students, you start to develop little antennas when you read to judge if the student really got what she or he is writing about, and ultimately the quality of the students work (although you as the supervisor of course know everything better, well, you might notalways be familiar with all the details).

So let’s take the example of my email-turned-blog, imagine I was living in Japan, had no idea about nuclear science and engineering (not too big a stretch someone just said), was looking for some info on Fukushima and came across Jason’s blog. Do I read it? All of it? What do I do then?

My approach to evaluating any sort of reports on the internet (and elsewhere) consists of 5 elements.: 2 regarding trustworthiness, 2 regarding the style (as a measure of effort put into a piece, but also a good indicator of the level of understanding of the author of the subject that he/she write about) and 1 element for content (arguably the most difficult to judge if you are not already familiar with the field). I will have to give myself credit on some of the dimensions, so I am asking you ahead of time for your forgiveness of some literary narcissism in the following.

1. Judging obvious fishiness (Trust)

When you surf the web, you come across a lot of stuff that you can safely disregard immediately. So I have two criteria for an immediate go/no-go decision at the onset:

a. Context: What is the context of the information? Blogs can be places where people put great stuff, but also incredibly stupid things (as I said, just Google my name these days). In the case of Jason’s blog, no points for great existing content, but also no minus points for tons of conspiracy theories and UFO posts. 0 points

b. Hoax potential: Would I have believed the whole story, cousin at MIT writing an email, setting up a blog to share it? Probably yes. Story looks interesting enough at first glance and setting up a blog is little enough work. Testing the opposite hypothesis: Why would anyone go through that much trouble of writing such a long text; invent such a boring cover story; and then assign the authorship to a total nobody in nuclear engineering, and not some expert in the field? So again, nothing major in favor, but also not a deal killer, 0 points.

2. Trustworthiness of the author (Trust)

Again, we have two criteria:

a. Past experience in the field. Is the author an authority in the field? Google clears that one up pretty quickly, certainly not. -1 point.

b. Bias, agenda, background: Checks out, engineering guy, MIT, probably has done his homework. 1 point.

3. Style and presentation (Style)

Is the narrative and style appealing? Again, I usually use this as an indicator of effort and level of understanding on the side of the author. Before I send the original email of to Jason, I scanned it one more time and thought to myself “Hm, this has actually turned into a nice piece of writing.” I probably would have had the same reaction scanning the text – well structured, flowing narrative, clear reasoning. 1 point.

4. Quality of the structure of the work (Style)

Does the article follow a logical structure? The article does seem well structured. It introduces the fundamentals, then progresses to describe what happened in Fukushima so far and drawing on these fundamentals. Seems to make sense. However it is not an academic treatise and strongly opinionated. Still, 1 point.

5. Content quality of the work (Content)

Here, since this is the most important category for me, I use a number of criteria:

a. Are the general fundamentals right? Are general engineering and physics fundamentals right that are used in the writing? Are the terms correctly used? Yes, 1 point.

b. Are specifics right? Are specific fundamental facts (e.g. half-life, types of elements etc.) and specific facts (sizes, amounts, temperatures, events) correct to the extent that I can verify them? Yes, 1 point.

c. Is there an uninterrupted logical flow from context and facts to interpretation? For the most part, yes. There are no logical breaks between the context, the facts being discussed in that context and the conclusions that are drawn. In its own little universe, it makes sense, no conclusions falling out of nowhere, no contradictions. However, again, the writing is not objective and strongly opinionated. But still, 1 point.

d. Are the sources given? Does the article contain sources so I could verify the claims and facts presented by the author? No, not in the narrative, not as footnotes. -1 point.

6. Possible next actions:

So, what should I do with what I just learned from reading the document? If we tally up the points for a first impression, we get 4 out of 10 points. And looking at the critical points, one of them is a biggy: No sources so I could easily verify if what the author claims is true or not. So what to do with it?

a. Disregard. This would mean thinking “oh my god, what a load of junk and a waste of time”. No, that is not what I would have done.

b. Use it to build mental model of the problem and investigate further. This means I use my newly acquired knowledge to build a mental model of the problem. What is the relevant context? What are the critical facts I need to know or monitor? That mental model is then tested (can I confirm what was said about the context, can I confirm what facts were presented?), and once that is done, run with it to grow the context (i.e. integrating understanding of spent fuel ponds) and interpret incoming facts (i.e. how dangerous is the latest venting of steam)?

c. Believe and be done with it. The information I just acquired solves my problem. I believe everything and am done with it (in this case, worrying about Fukushima).

As you can probably tell by the length of discussion of the different points above, I would have gone with b. That concludes my therapeutic reflections. And maybe you find the assessment process useful to make a more conscious choice of the news programs in TV, radio, press and internet you decide to support (I did, and that is why I love Barry and his site bravenewclimate).

Where does that leave us?

1. Help people understand the context. If you help people to understand the context, you help them to help themselves in the future. My hope is that the email made a small contribution to helping the general public, as well as some journalists, in building the context to make a better informed assessment of new facts as they come in. Do your part with your family and friends (as I had originally intended…)

2. Take a stand against mass hysteria. The email I wrote contains both an introduction to some relevant physics and engineering, as well as strong opinions about the safety of the plant you may or may not share. One part lives on on the MIT website that was created to provide some more of the same, fact-based and understandable context information; the other part has hopefully inspired a couple of people to also speak their mind in a general atmosphere of panic.

3. Demand balanced and quality reporting. Demand discussions of “possible” and “most likely” scenarios in the news. Call the newspaper editor, TV station and radio station and complain about the garbage that is still put out there. Make a conscious choice regarding your news viewing, reading and listening habits. News shows are out there to produce viewers, listeners and readers that they can sell to advertisers, not quality news. If you don’t demand it, it won’t happen.