What’s the best way to solve a wicked problem — by working in a large group sharing ideas via the web or as individuals? That’s the question George S. Davidson (1400) and his research team attempted to resolve this summer in an experiment funded by Sandia’s internal Laboratory Directed Research and Development (LDRD) program.
The research, conducted by George, Courtney Dornburg (12335), Susan Stevens (12335), Stacey Hendrickson (6761), Travis Bauer (6341), and Chris Forsythe (6341), yielded some surprising results.
“In this day and age of email and the Internet, our expectations were that computer-mediated group brainstorming, i.e., across the web with no face-to face contact, was going to have the best results,” George says. “What we found, however, was that people working as individuals were at least as effective and possibly more so than those brainstorming in a group over the web when trying to solve wicked, tangled problems, both in terms of quality and quantity.”
George assembled a team that consisted of himself, three psychologists (two of them PhD students), and two cognition researchers to investigate tools and methods for bringing large numbers of people together to solve difficult problems via the web. The team anticipated that the group brainstorming could result in a huge pool of ideas that might lead to solutions. The research team decided to pursue an electronic brainstorming experiment built around the common face-to-face technique used at Sandia where people submit ideas written on yellow sticky Post-It® notes.
As the team designed the experiment, the initial issue was coming up with a difficult question that had no right or wrong answer, Courtney says. In June Courtney, a psychologist who has worked at Sandia for two years, attended a new-hire breakfast hosted by Labs Director Tom Hunter. During the breakfast, Tom posed a challenge to the new hires, asking them to think about the implications of two popular models in management theory. In one view workers are just another natural resource to be used. In contrast, the second model sees employees as assets, which can be made more valuable by investing in their development.
Courtney shared the question with the team, and they unanimously decided to make it the wicked problem for the experiment.
They recruited 120 Sandia employees and contractors and 26 student interns through Sandia Daily News and word of mouth to participate in the experiment, which lasted four days in August. The participants were broken into two groups, those who worked alone and did not see the ideas of the other participants and those who worked in a group and were able to see and build on the ideas of the other members in the group via the Labs’ intranet. Of the 120 employees and contractors, 69 contributed ideas.
During the experiment, participants logged onto the website anonymously and saw the question displayed at the top of the screen. They were asked to input their ideas — the more the better, but no name calling or abusive language.
The research was unusual because, while studies have been conducted with large brainstorming groups, most have been performed in academic settings with college students as participants. This experiment was, to the research team’s knowledge, the first done in a laboratory/ industrial setting over an extended period of time.
In addition, most previous studies looked only at quantity — number of responses — while only a small number examined the quality of the ideas. The Sandia experiment analyzed both. Responses to Tom’s question were scored by team members according to originality, feasibility, and effectiveness. Responses were also analyzed by STANLEY, a Sandia-developed text analysis tool that looked at common noun phrases and key words.
“We were amazed at the length and quality of the responses, both from the people working as a group and those working individually,” Courtney says. “People were very engaged, often writing long, detailed responses.”
She adds that what was most interesting is that the quality of ideas from the people responding as individuals was “significantly better across all three quality ratings.”
Courtney says the finding that individuals are more successful than groups in computer-mediated brainstorming suggests a time- and cost-saving potential for companies. Generally, when electronic group brainstorming is compared to verbal brainstorming, it is touted as having the advantages of shorter meetings, increased participation by remote team members, better documentation via electronic recording, and cash savings. But the Sandia research suggests that people working to solve problems on their own might involve less time and, thus less expense, than electronic group brainstorming.
While individuals working alone fared nominally better in this study, George says, the research also indicates that group on line brainstorming can be effective when ideas are needed from large numbers of people.
“I expect that in coming years better software, including threaded discussions with moderators to focus the work and prediction markets to evaluate quality, will become a tool that large organizations will use to solve wicked problems,” he says.
Themes emerge from ‘wicked problem’ experiment
Group members and individuals working alone provided more than 200 concepts and ideas to answer Labs Director Tom Hunter’s question dealing with two models in management theory. In one view workers are a natural resource and in the other they are assets. Each concept was explored and analyzed.
Among themes that emerged from the responses were:
In ceremonies held Oct. 13 at the 19th Annual HENAAC (Hispanic Engineering National Achievement Awards Corporation) Conference in San Diego, Calif., Juan Torres (6332) received HENAAC’s Pioneer Award. He was honored for his development of US critical infrastructure assurance roadmaps, helping establish the National SCADA Test Bed, and for leading a Bureau of Reclamation program to secure dams in the western US.
Looking for a better life, four-year-old Juan, his parents, and Juan’s two-year-old brother emigrated to La Junta, Colo., in 1971, from Leon, a small city in central Mexico. But they found the streets of La Junta weren’t paved in gold. His father, Juan Sr., had attended only first grade, his mother, Clementina, third. Jobs were hard to find, but slowly life improved for the family.
Mama Tina, his grandmother, believed that many of life’s hardships could be overcome through education. “Estudia y un dia te bendicira Dios y tus manos estaran llenas de dinero,” she said. (Go to school and someday God will bless you, and your hands will overflow with money.)
“My grandmother, who passed away at 94, could barely read and write,” says Juan, “but what she lacked in formal education, she made up for with wisdom and experience.”
Juan’s parents knew that education was the key to their children’s success. Clementina enrolled Juan in a Head Start program as soon as possible. Juan was fluent in English by the time he entered kindergarten. “Their love, work ethic, and constant emphasis on education contributed to our family’s success,” he says.
Juan’s interest in science and math led him to the engineering technology program at DeVry Institute of Technology in Phoenix, then on to the University of Southern Colorado in Pueblo, Colo., and a bachelor’s degree in electronics engineering technology.
Just before graduation, his life took on new meaning and value as he received what he calls a “priceless privilege” — he became an American citizen. His parents followed suit. This privilege opened doors to challenging career opportunities, beginning with a job offer from Sandia in 1990.
At Sandia Juan supported the design of mobile command and control systems for the Air Force. He also attended the University of New Mexico and earned a master of science degree in electrical engineering. As a member of the technical staff, Juan became the engineering liaison for a command and control system at Peterson Air Force Base in Colorado Springs, Colo.
He sat on the Leadership Advisory Group of the DHS Control System Security Center and has supported congressional testimony on supervisory control and data acquisition (SCADA) systems. He is currently manager of Energy Systems Analysis Dept. 6332.
Juan says his wife Michelle has been his support in career, education, and life decisions. “Michelle is devoted to our children, Andrew and Mikaela,” says Juan. “She has made sacrifices so that we can prosper and grow as a family and as individuals.”
“My hope is to continue to encourage others, especially Hispanic youth, to be open to the possibilities this world offers,” says Juan.
“Mama Tina would have been proud,” he says. “She was right when she said life’s hardships could be overcome through education.” -- Iris Aboytes
One definition of a luminary is “a person who has attained eminence in his or her field or is an inspiration to others.” Another definition is Monica Martinez-Canales (8964), at least according to the Hispanic Engineers National Achievement Awards Corporation (HENAAC), which presented her with a 2007 Luminary Award on Oct. 12.
The Luminary Award is given to Hispanic professionals in engineering, science, and technology in recognition of significant contributions to the Hispanic technical community. HENAAC expects these individuals to continue to carry the torch at their respective organizations and inspire future generations to pursue careers in technology.
Monica, who was born in Mexico and raised in the toughest barrios of San Diego, was the first member of her extended family to go college. She earned her BS in mathematics from Stanford University and her PhD in numerical analysis from Rice University. She returned to Stanford as a postdoctoral fellow in the Department of Geological and Environmental Sciences. In 2000, she joined Sandia’s Department of Computational Sciences and Mathematics Research.
Throughout college, Monica was considering a career in medicine. She majored in pure mathematics because she enjoyed the subject, but didn’t see a career path outside of academia. Then, in 1991, she attended a conference organized by the Society for the Advancement of Chicanos and Native Americans in Science. She met Richard Tapia, a professor in the Department of Computational and Applied Mathematics at Rice University, and many of his current and former students, which included Mimi Celis, a female mathematician working at SGI, and Juan Meza, previously the manager of Sandia/California’s Computational Sciences and Mathematics Research Department.
Monica says the interaction with Tapia and his students changed her life. “I met these amazing mathematicians who were impacting society at large with their work. I saw that you could apply mathematics to hard problems in national defense, high-performance computing, and visualization,” she says.
She spent the summer of 1992 at Sandia/California as an intern, working under Meza. That fall, she attended Rice as a graduate student. For her PhD thesis, she researched fluid flow problems in an estuary base. She helped develop shallow water models of Galveston Bay that tracked the effects of tidal surges and ocean patterns.
“This was the most interesting work I’d done to that point,” she says. “There was a real environmental interest driving the work, as both shipping tankers and fisherman have a large presence in Galveston Bay.”
The underlying mathematics that Monica developed for the models got significant notice. She was awarded a National Science Foundation postdoctoral fellowship to work with the hydrogeology program at Stanford University.
In 2000, Juan Meza recruited Monica to work at Sandia in his department. From 2003 to 2005, she led her own LDRD project on an innovative new Bayesian-based approach to computational design exploration under uncertainty. In 2003 she was coprincipal investigator of a National Science Foundation grant to fund student scholarships and a doctoral consortium for the 2003 Richard Tapia Celebration of Diversity in Computing Conference.
Monica is passionate about serving as a role model to encourage more women and Hispanics to pursue careers in math and science. She says the number of women and minorities among her peers is less than it should be.
To show the world what is possible, she is involved with numerous organizations for women and minorities in science.
Monica would like to be seen as proof that perseverance and hard work pay off. She understands that as a role model, she serves different purposes to different people. “Some just want to know it’s achievable, others want help with a
specific problem, others want long-term career advice, and others want to know how I got here and how I balance my work and family life,” she says.
That work/family balance question is one Monica is highly qualified to answer — she’s the mother of two young children. As active as her career is, she has an equally full life outside of work. In her “spare” time, she cooks, cleans, bakes, launders, irons, reads novels, sews children’s clothes, makes quilts, grows plants from seed, paints, practices martial arts, and spends time tending her garden.
If she could impart just one lesson to aspiring mathematicians, scientists, or anyone for that matter, it would be that it is okay, even necessary, to ask questions.
“Between junior high school and college, I stopped asking questions,” she says. “It made that part of my education excruciating. You have to learn to find people who will help you. The hardest part is approaching someone. Even now I find myself sitting in my office, not wanting to call someone because I don’t even know what I don’t understand. But if you don’t ask, you can’t learn.” -- Patti Koning