The Limits of Expert Trial and Error

Of course I loved this comment on a recent post of mine about how to flavor stuff:

I made a vegetable soup today spiced by small amounts of vegetable stock, hoi sin sauce, angostura bitters, lea & perrins worcestershire sauce, Kikkomann soy sauce, maggi wrze, marmite, maille mustard. I can honestly say it was the best tasting soup I, or any of my guests, can remember having been served.

I routinely make soups that taste clearly better than any of the thousands of soups I had before I figured out the secret. There is no failure (I’ve done it 20-odd times), no worry about over- or under-cooking. Something else odd: There seems to be a ceiling effect. The texture could be better, the appearance could be much better, the creaminess could be better, sometimes the temperature could be better, the sourness could be better, but I can’t imagine it could be more delicious.
Why wasn’t this figured out earlier? I’ve looked at hundreds of cookbooks and thousands of recipes. I haven’t seen one that combines three or more sources of great complexity, as I do and the commenter did. There may be more trial and error surrounding cooking than anything else in human life. Billions of meals, day after day.

I think it goes back to my old comment (derived from Jane Jacobs) that farmers didn’t invent tractors. Some people claimed they did but I think we can all agree farmers didn’t invent the engine on which tractors are based. You can’t get to tractors from trial and error around pre-tractor farming methods. Even though farmers are expert at farming. I think that’s what happened here. I am not a food professional or even a skilled cook. My expertise is in psychology (especially psychology and food). Wondering why we like umami, sour, and complex flavors led me to a theory (the umami hypothesis) that led me to a new idea about how to cook.

And this goes back to what many people, including Atul Gawande, fail to understand about how to improve our healthcare system. The supposed experts, with their vast credentials, can’t fix it — just as farmers couldn’t invent tractors. Impossible. The experts (doctors, medical school professors, drug companies, alternative healers) have a serious case of gatekeeper syndrome. The really big improvements will come from outsiders. Outsiders who benefit from change. To fix our healthcare system, empower them.

20 Responses to “The Limits of Expert Trial and Error”

  1. Cliff Styles Says:

    Just getting out of their way would help: end government-sanctioned occupational licensing in medicine.

  2. q Says:

    re fermented food, you might want to do some kind of survey of what various cultures do. i had some very good hot sauce from ghana once which i think had fermented fish. i don’t remember the name but it had a name with a long ssssssss sound in it.

    also you might want to look at david chang’s momofuku cookbook.

  3. Alex Chernavsky Says:

    Seth, can you post recipes — or at least ingredient lists — for the soups you make?

  4. Tom Moertel Says:

    Tonight, I made cabbage braised in chicken stock with Mangalitsa bacon, onion, and a little raw apple-cider vinegar. I made this dish a few nights ago – my control – and it was good. Tonight, however, I also added a little miso, fish sauce, and caraway seed as seasonings. Tonight’s version was much better..

    I don’t think the caraway seed explains the difference. I’ve used it before without noticing much effect. Nor do I think the salt in the miso explains it. I adjust my salt to taste while cooking and simply used less salt this time. Something else seems to be going on.

    On this evidence, Seth, my belief in your hypothesis has increased.

    Oh, yeah. I’m also going to be adding miso and fish sauce to everything from now on. This is an experiment that rewards repetition. ;-)

    I wonder how fish sauce would taste in butterscotch sauce…

    Cheers,
    Tom

  5. Part of how to get improvements and how science progresses « Anthny Says:

    [...] Seth Roberts has two important paragraphs here: [...]

  6. Nansen Says:

    Many of the condiments in the quoted recipe have a high salt content. Does that fact ever concern you?

  7. seth Says:

    Nansen, I once varied whether or not I ate cheese (high in salt). My blood pressure didn’t noticeably change. So I don’t worry much about salt intake, so long as my blood pressure is reasonable. But it’s a good point and I would rather get my complexity from low-salt stuff.

    Alex, my soups contain fat (pork fat), protein (pork or beef, sometimes fish), vegetables (e.g., chinese greens, carrot, onion, mushrooms, tomato, cabbage, kimchi), and flavorings (ginger, sometimes garlic, mustard oil, Chinese-type aged vinegar, miso, Chinese fermented soy beans, Japanese BBQ sauce). The last three are the crucial ingredients, added at the very end, and vary according to what’s available. Right now I’ve run out of Japanese pickles. I add yogurt just before eating.

  8. Glen Raphael Says:

    I took my standard omelet recipe and added some miso, soy sauce, and vinegar. The result: the resulting omelet tasted (1) very strange, (2) too salty, (3) not particularly good.

    I took a cooking class series once and one of the big things I learned was the importance of salt. Adding salt at each cooking stage – and adding enough of it – brings out all the other flavors. The goal isn’t for a dish to taste “salty” but for all the component flavors to be distinctly perceived. When balancing a soup recipe if you can’t taste one ingredient, add more of that ingredient. If you can’t taste many ingredients, add more salt.

    People are unreasonably afraid of salt and many homecooked meals are bland due to the lack of it; the idea that *everybody* should try to avoid salt is folk wisdom, not medical science.

    Seth: a good null hypothesis here is that it’s the added saltiness rather than the added complexity that is making your soup taste good compared to how it tasted before. The good news is that unlike many of your experiments, this idea is something it would be trivial to test “scientifically”. You say the crucial ingredients are added at the end. So you could make soup right up to the point where you’d add them, then ladle the soup into a few saucepans (if it needs more cooking) or bowls (if it doesn’t). Modify each in different ways- more salt, more miso, nothing at all, etcetera – pour small amounts into numbered paper cups and do a blind taste test. You could even make it double blind!

  9. seth Says:

    Glen, I’ve never gotten results like this by varying amount of salt, which I’ve done countless times. Variations in amount of miso — which varies the amount of salt at the same time, because miso is high in salt — never produce results like this. Only when I add a second source of complexity (and reduce the amount of miso) do I get big improvements.

  10. Mike Kenny Says:

    Does the complexity need to come from a non-commercially produced source,(because of uniformity or weakness of fermentation)? I think of a cheeseburger–it has cheese, pickles and ketchup, which all fermented or made with an ingredient that are fermented. Cheeseburgers are great, but I think most pickles I have taste the same, and ditto for ketchup and American cheese. Maybe they are so uniformly produced that they don’t really introduce my pallet to the novelty homemade fermented products would.

  11. seth Says:

    Mike, the answer to your question is no. I use commercially-produced miso and Japanese pickles, for example. In the recipe I quote, all the ingredients are commerically-produced.

  12. Mark Says:

    I’m skeptical of recipes that have a lot of (flavoring) ingredients. It’s like a painter saying he mixed a great color by using 12 different base colors. Given, say, six equally spaced primary colors and black and white you can mix any color there is with four of the paints. If there are only four or five or six basic tastes, then after a while they just start to cancel themselves out and product the culinary equivalent of the muddy color you get when you mix too many paints together.

  13. Nathan Myers Says:

    This is off-theme, but my favorite new soup ingredient is sweet potato. It improves both texture and flavor.

  14. Glen Raphael Says:

    Like Mark, I’m suspicious of the lots-of-tastes hypothesis. I still haven’t tried it in soup but I did try another omelet. After last time I noticed that I’m using a new brand of miso (a red-and-white type) which has salt as an ingredient so maybe the miso I was using before was just too salty. So I made a new omelet with *no* salt as a separate ingredient and added to the eggs a teaspoon rather than a heaping tablespoon of miso. Also added: a teaspoon of yogurt, a small amount of vinegar and a small amount of natto. I left off the soy sauce since that’s almost pure saltiness – I didn’t want to risk it with the miso.

    The mushroom-and-cheese omelet that resulted was strange and kind of interesting. It didn’t taste bad but also didn’t taste unusually delicious.

    So if this phenomenon is real perhaps it needs a longer cooking cycle, or perhaps it doesn’t work in an egg omelet or wonderflavor doesn’t mix well with cheese or only works in soup. Or, lots of experimentation is required to find a collection of relative amounts that results in “pure deliciousness”; Seth is getting consistent results due to things he’s doing or amounts he’s using automatically from long force of habit. Or it only produces ultimate deliciousness in the context of some other set of ingredients Seth is using. Or it’s an acquired taste that I haven’t yet acquired.

    In summary: applying the “multiple complex taste” theory to egg omelets I have thus far: two attempts, two failures.

  15. Melissa Says:

    I have definitely made atrocious soups with this principle. I remember one that had fish sauce, miso, and soy sauce….I had to throw the whole thing away. You have to be careful, particularly with fish sauce.

  16. seth Says:

    Melissa, I’d imagine it tasted atrocious because it was too salty. I don’t use soy sauce to complexify anything because it seems to me more salty than anything else. Miso has a better balance. I don’t know much about fish sauce.

  17. Chris Burd Says:

    Seth, can you provide us a recipe? Amounts, cooking times, etc.?

  18. Glen Raphael Says:

    I, too, would like to see an exact recipe. Two failures were enough; I’ve given up on this principle as applied to omelets. I might try it on soups but I suspect the result would be the same as it was for omelets.

    Note that if the hypothesis is correct as stated, one ought to be able to pre-mix a few “sources of great complexity” to get a big tub of “super-complexity” you just keep in the fridge and add a dollop to whatever you cook. If that works, somebody could then sell “complexity” in premixed tubs at the store. miso+yogurt+wine or whatever. Then you could dehydrate the super-complexity to sell it in powder form and use like MSG. Is there a reason to think this wouldn’t work?

    Contrary theory: perhaps “crazy-spicing” for a while puts people in a different frame with regard to what they consider tasty. As might Shangri-La. Perhaps the need for “complex” flavor is enhanced after eating flavor-free calories for a while.

    Anyway, taste is subjective, but recipes don’t need to be. If this theory is correct, you should probably be writing a cookbook.

  19. seth Says:

    Glen, I’ve never compared dehydrated miso to regular miso but I imagine there’s a good reason miso is usually not sold dehydrated. “The need for complex flavor is enhanced after eating flavor-free calories for a while.” Techniques that produce complexity are found in a wide range of cuisines. As far as I can tell, almost everyone enjoys complexity. You’re right, I should be seeing how this idea can be used elsewhere than soups.

  20. Glen Raphael Says:

    “You’re right, I should be seeing how this idea can be used elsewhere than soups.”

    Sure, but at this point I’d settle for a precise soup recipe.

    Without a recipe, if I try it and it doesn’t work I don’t know whether to count my attempt as a fair test of your hypothesis. You could always say I didn’t use *enough* complexity source or I used too much of some specific complexity source or other.

    I assume miso is sold in tubs because it improves the texture. But your soup theory specifically is that the *taste* is optimized to best-soup-ever status. You make no claims for texture, appearance, creaminess. The taste effect ought to be substantially the same from complexity-bouillon cubes as it is from the tub; the other aspects would be whatever they already were. So if your theory is correct and you convince people of it, the logical outcome is that henceforth all restaurants will add complexity cubes to their soups. And the guy who sells those cubes will make a fortune.

    And then everyone will get fat due to the flavor-calorie association being stronger than ever before. (Unless complexity cubes deliberately vary the blend from one cube to the next.)

    Incidentally, the way you get better “creaminess” is by blending and then pushing soup through a fine strainer.