Mark Greville

Tag: Bounded Rationality

  • Generative AI is digital homeopathy — how I train my own model

    If it’s your first time here, you may be surprised at how few pieces I have written. (After reading for a while, you may even be glad of this fact). When friends bring up what they assume is a painful subject, they get a faraway look in their eyes. They place a gentle hand on my shoulder, gaze into the distance, and ask me if I’ve seen ChatGPT. “AI can solve your problem, Mark. It can generate thousands of posts for you. It could help the blog look less like an abandoned quarry”. They think it would solve my problem. My Problem! If that was my problem, life would be a dream. This idea misses the purpose of this blog. It misunderstands my reason for writing entirely.

    Generative AI works because it sucks in lots of data, processes it and builds statistical models. At its core, it’s a fancy autocomplete — or, as Grady Booch puts it, a bullshit generator. It acts like an annoying older brother, automatically finishing every sentence (apologies to my own brothers). GenAI probabilistically keeps predicting the next word until it produces sentences, paragraphs, and a complete piece of writing. It can do this because the statistical models have established the most probable next word. These statistics are based on text from books, academic papers and (blesses myself) the internet.

    However, there is no concept of meaning in AI. Reasoning is not programmed in anywhere. The output is remarkable, and can appear that the machine is thinking. It isn’t. This is why we sometimes get unreliable outputs — hallucinations. Any meaning we perceive is simply a vague echo from the training data of billions of people. GenAI is digital homeopathy.

    We are all lazy by default. Humans rely on heuristics to understand the world. If we didn’t, we would have to process every single thing we hear, see, smell, taste, and touch. A short walk in a city would exhaust our brain’s capacity. We would lose the ability to decide which way to go, overwhelmed by thousands of people, cars, smells, noises and the like. The great Herbert Simon coined the phrase ’bounded rationality’ to describe the cognitive limitations of the mind.

    Thinking is hard work. For me, thinking is about sucking in data, and then processing it. I process it through writing. Writing is my way of thinking.

    I first had a go at writing because my friend Gar was guest-editing a technology journal. Even though I’d never written before, I was confident that I could write about something I already knew. This confidence was quickly shattered. I was embarrassed at how muddled my thoughts were. Turns out, I knew nothing. Solid ideas fell apart the minute I wrote them down. I could barely finish a sentence without feeling the old familiar burning creep across my cheeks, embarrassed as another idea falls apart while I try to pin it down.

    Writing anything down forces me to think really hard. Because I was determined to improve my thinking, I wrote every day. I then started a blog because the potential for embarrassment at publishing poor output forced me to aim for a higher standard.

    I’m not interested in building an audience, I am trying to improve. I’m not trying to publish a lot of work. In fact, I have almost 200,000 unpublished words in my Ulysses editor. This writing habit has helped me build a model of the world. And 4 of my pieces here have reached the front page of HackerNews — this is a victory for me — a nobody from rural Ireland.

    How to Know Everything.

    Technical Debt Is Not Debt; It’s Not Even Technical

    AGI May never align with human needs

    Gladiator Style interviewing

    The dominant model on the internet is of consumption. The more we consume, the more ads we see, the more we buy, the bigger the economy. But if all we do is consume, and never take the time to process information, of even produce our own, then we learn very little. Go back 3 months and look at your internet history. What did you learn from browsing? What actions did you take? Probably close to nothing useful. Instead of spending 2 hours a day on the internet, take 15 minutes to write. Just write down some thoughts. Any thoughts. This slowly changes your understanding of the world around you for the better.

    GenAI is an information processing tool. GenAI will help people process information more effectively. But people are lazy by default. If thinking is the hardest work in the knowledge economy, people will avoid thinking where possible.

    Therefore, for those who overuse it, GenAI may well make them more stupid. Victor Del Rosal, in his incredible book Humanlike, calls this Cognitive Atrophy. I already see too many examples of people outsourcing their thinking to Generative AI tools. I see them slowly getting more stupid every day.

    Me, I’m building my own model.

  • How Technology Architects make decisions

    Or why you might spend a fortune on a red car

    Chinese translation available here

    The remarkable Herbert Simon won the Nobel Prize in Economics in 1978 and the Turing Medal in 1975. Reading about his life gives me a panic attack when I consider how little I have achieved in comparison. He published ‘Administrative Behaviour’ in 1947, and I started reading it in 2021. I started by treating it as a relic of World War II era business, a history book. It quickly filled me with horror as Simon explained business, thinking and decision making in ways which seemed obvious after reading them, but I had never even thought of. I immediately felt weak. I felt like a total imposter. How had I never read Herbert Simon before? Why had nobody told me? It panicked me for days. I dropped a reference to the book into every meeting for weeks. That practice soon calmed me down. It turns out almost no-one I know had read it either.  

    Early in the book, Simon talks about how each department in an organisation has one job. They take in information and turn it into decisions which are executed (either by them or another department). He introduces the concept of Bounded Rationality – how it is impossible to evaluate an almost infinite set of possibilities when making a decision. Instead, we must choose a smaller ‘bounded’ set of assumptions to work within. 

    Back in the actual world of architecture, I have always boiled the job down to either a) making decisions or b) provide information to help others make decisions. I’ve only ever had a vague sense of how architects make decisions, even though it’s been my job for the majority of my career.

    In a fantastic paper published in 2014, “EA Anamnesis: An Approach for Decision Making Analysis in Enterprise Architecture”, Georgios Plataniotis, Sybren de Kinderen and Henderik Proper explain the importance of capturing decisions made about architecture. They go further, arguing that capturing the reasons for a decision and alternatives considered is just as important. Documenting the rationale when a decision is made gives it context, explains the environment at the time, and helps inform future decisions. 

    The paper describes four strategies used to make Enterprise Architecture decisions. Each decision is an attempt to decide on the best alternative among competing choices. They split decision types into the following buckets :

    • Compensatory. This type of decision considers every alternative, analysing all criteria in low-level detail. Criteria with different scores can compensate for each other, hence the name. There are two types here:
      • Compensatory Equal Weight – criteria are scored and totalled for each potential option, the option with the highest total signifies the best decision. 
      • Compensatory Weighted Additive (WADD) – here a weighting is given for a criterion to reflect significance (the higher the weighting, the higher the significance). The weighting is multiplied by the score for each criterion, then each alternative is summed, the highest total winning. 
    • Non-Compensatory. This method uses fewer criteria. The two types are:
      • Non-Compensatory Conjunctive – alternatives that cannot meet a criterion are immediately dismissed, the winner is chosen among the survivors. 
      • Non-Compensatory Disjunctive – an alternative is chosen if it complies with a criterion, irrespective of other criteria. 

    Say you were buying a car, and you had the following criteria: fuel efficiency, colour, cost, and ease of parking (as scored below). 

    CarFuelColourCostParkingTotalFuel x2 Weighted Total 
    Car A9Black64191828
    Car B6White105211227
    Car C4Grey41018822
    Car D1Red1810211

    The four strategies might look like this: 

    1. Compensatory Equal Weight – in this case you pick the highest unweighted total – Car B
    2. Compensatory Weighted Additive – because you drive long distances, you apply a double weighting for fuel mileage and pick the highest weighted total – Car A 
    3. Non-Compensatory Conjunctive – because you live in a city, you discard any car that isn’t easy to park (at least 7/10). This leaves a choice between C and D you chose the highest score between them – Car C 
    4. Non-Compensatory Disjunctive – you fall in love with a red car – ignore everything else – Car D

    Compensatory decisions are suitable when time and resources are available to   

    • gather the right set of alternatives, 
    • evaluate each alternative in detail
    • score each with consistency and precision. 

    Non-Compensatory decisions are necessary when

    • there is time stress
    • the problem is not well structured  
    • the information surrounding the problem is incomplete
    • criteria cant be expressed numerically
    • there are competing goals
    • the stakes are high
    • there are multiple parties negotiating in the decision. 

    A level of pragmatism is important when choosing a decision strategy. Using Simon’s concept of bounded rationality, compensatory decisions can never be fully worked out. Some level of assumptions are necessary, otherwise the work needed to make every business decision is almost infinite. However, within a set of ‘givens’ (given we need to decide by Friday, and given budget is x, and given resources available are y, and given etc) the weighted additive method (WADD above) has proven effective in my experience. The framework forces decision makers to consider each alternative clearly, as opposed to a clump of criteria mashed together. It also forces all parties to agree a set of weights, helping the group agree on the hierarchy of importance. These processes improve communication between parties, even when they disagree on the choices of criteria and weights. 

    A strange magic happens during negotiation of the scoring, as parties try to force their choice. The mental mathematics going on inside heads is a dizzying. I have witnessed all types of behaviour, from people determined there be no change, to an exec wanting an inflight magazine article to form the basis for all future strategy, to a head of a business unit wanting us to use his nephews school project as part of the solution, all the way to one mid 40s executive, who got so annoyed with the debate, that he started jumping up and down and stamping his feet because he wanted his decision, and “that’s what I’ll get”. 

    Start now. 

    Since first posted, this article has been subsequently published by Architecture and Governance magazine here