Mark Greville

Tag: Technology

  • Generative AI is digital homeopathy — how I train my own model

    If it’s your first time here, you may be surprised at how few pieces I have written. (After reading for a while, you may even be glad of this fact). When friends bring up what they assume is a painful subject, they get a faraway look in their eyes. They place a gentle hand on my shoulder, gaze into the distance, and ask me if I’ve seen ChatGPT. “AI can solve your problem, Mark. It can generate thousands of posts for you. It could help the blog look less like an abandoned quarry”. They think it would solve my problem. My Problem! If that was my problem, life would be a dream. This idea misses the purpose of this blog. It misunderstands my reason for writing entirely.

    Generative AI works because it sucks in lots of data, processes it and builds statistical models. At its core, it’s a fancy autocomplete — or, as Grady Booch puts it, a bullshit generator. It acts like an annoying older brother, automatically finishing every sentence (apologies to my own brothers). GenAI probabilistically keeps predicting the next word until it produces sentences, paragraphs, and a complete piece of writing. It can do this because the statistical models have established the most probable next word. These statistics are based on text from books, academic papers and (blesses myself) the internet.

    However, there is no concept of meaning in AI. Reasoning is not programmed in anywhere. The output is remarkable, and can appear that the machine is thinking. It isn’t. This is why we sometimes get unreliable outputs — hallucinations. Any meaning we perceive is simply a vague echo from the training data of billions of people. GenAI is digital homeopathy.

    We are all lazy by default. Humans rely on heuristics to understand the world. If we didn’t, we would have to process every single thing we hear, see, smell, taste, and touch. A short walk in a city would exhaust our brain’s capacity. We would lose the ability to decide which way to go, overwhelmed by thousands of people, cars, smells, noises and the like. The great Herbert Simon coined the phrase ’bounded rationality’ to describe the cognitive limitations of the mind.

    Thinking is hard work. For me, thinking is about sucking in data, and then processing it. I process it through writing. Writing is my way of thinking.

    I first had a go at writing because my friend Gar was guest-editing a technology journal. Even though I’d never written before, I was confident that I could write about something I already knew. This confidence was quickly shattered. I was embarrassed at how muddled my thoughts were. Turns out, I knew nothing. Solid ideas fell apart the minute I wrote them down. I could barely finish a sentence without feeling the old familiar burning creep across my cheeks, embarrassed as another idea falls apart while I try to pin it down.

    Writing anything down forces me to think really hard. Because I was determined to improve my thinking, I wrote every day. I then started a blog because the potential for embarrassment at publishing poor output forced me to aim for a higher standard.

    I’m not interested in building an audience, I am trying to improve. I’m not trying to publish a lot of work. In fact, I have almost 200,000 unpublished words in my Ulysses editor. This writing habit has helped me build a model of the world. And 4 of my pieces here have reached the front page of HackerNews — this is a victory for me — a nobody from rural Ireland.

    How to Know Everything.

    Technical Debt Is Not Debt; It’s Not Even Technical

    AGI May never align with human needs

    Gladiator Style interviewing

    The dominant model on the internet is of consumption. The more we consume, the more ads we see, the more we buy, the bigger the economy. But if all we do is consume, and never take the time to process information, of even produce our own, then we learn very little. Go back 3 months and look at your internet history. What did you learn from browsing? What actions did you take? Probably close to nothing useful. Instead of spending 2 hours a day on the internet, take 15 minutes to write. Just write down some thoughts. Any thoughts. This slowly changes your understanding of the world around you for the better.

    GenAI is an information processing tool. GenAI will help people process information more effectively. But people are lazy by default. If thinking is the hardest work in the knowledge economy, people will avoid thinking where possible.

    Therefore, for those who overuse it, GenAI may well make them more stupid. Victor Del Rosal, in his incredible book Humanlike, calls this Cognitive Atrophy. I already see too many examples of people outsourcing their thinking to Generative AI tools. I see them slowly getting more stupid every day.

    Me, I’m building my own model.

  • Gladiator style interviewing

    Gladiator style interviewing

    After an interview, respect the Gladiator rules:

    1. Before the debrief meeting, no interviewer may speak of the candidate. Disavow body language and sounds. This includes a raised eyebrow, a jaunty walk, or sighing.
    2. Unavailable interviewers provide a thumb and a brief written summary of feedback to an independent third party beforehand. Yes, I said a thumb. Guard this until the correct moment.
    3. Everyone available gathers for the debrief
    4. Everyone puts a thumb out sideways.
    5. The independent observer counts down 3, 2, 1, whereupon everyone either points the thumb upwards or downwards.
    6. The thumbs submitted for those not present get added to the total.
    7. Then the discussion begins.

    This is the way of the gladiator.

    Why run a post interview debrief like this? Let me tell you a quick story.

    When I was a director in Bank of America, I was trying to hire around 30 developers at top speed. This happens at big banks, you have no budget, then too much, then none again. When a hiring budget comes, the very next day ten different people call up and ask “How many have you hired yet”, as if there’s a next day delivery website available. I tell them all the same thing. “0”. They offer me a few condescending nuggets. “Have you thought about agencies?” “Did you try online”? “Do you know anyone yourself who is free?” They might as well ask, “Did you ask any of the random strangers out on the street if they are a developer?” “Do you have a coder secretly living in your garage”? “Can you hire some actors and get them to fill out the seats? It looks better when senior people come to visit. You could teach them to code, right?”

    Anyway, one of those days, our group had interviewed a developer. My interview was good. We had discussed overcoming adversity and talked through some examples of system design. The candidate told a great story about how they managed through difficult personal and professional issues. All was good in the world. I needed programmers, and I felt like this was one of the 30. The interview group gathered, and I said, “Ok, let’s get some feedback from everyone. I liked the candidate, and I’m interested to hear what you have to say”.

    The air pressure in the room dropped. It felt too hot. Claustrophobic. We were like a group of antelopes, as a lion appears in the distance. All motion ceased. Total silence. Everyone held their breaths. No-one spoke, but eyes darted around. Some stared at the floor. I thought I could get things going with a quick positive comment. What harm could it do?

    Big mistake.

    I have interviewed thousands of programmers in my career. I like to get a sense of the person, and what drives them. This is difficult to figure out. Usually because candidates are so coached that I feel like I’m in a play, as the ‘actor’ recites their lines. Some candidates are so practiced that I’m sure I could leave the room, and the interview would go on without me. They would sit forward, anticipating their next right answer, like a school kid in class. “Sir sir sir. My biggest weakness is I work too hard, I care too much. Let me give you three carefully selected examples to demonstrate.”

    There are no right answers in an interview. Or wrong answers. The world is too nuanced and complex.

    To break the fourth wall, I like to talk about their lives outside of work, their hobbies, or interests. This can loosen people up a bit. Then we get into how they see themselves in relation to others? Do they understand how their work links to company outcomes? How do they like to collaborate? What unusual ideas are knocking about in their heads? How much do they like to challenge their teammates? I’m looking for a conversation, not an inquisition. We will use the seesaw rhythm of conversation, the back and forth, the give and take to work together every day.

    I never think about how good they are at programming. Primarily because it’s a colossal waste of time. I haven’t programmed regularly in so long that my opinion is not relevant. Other programmers figure this out. For collaboration — someone who they will work with, maybe a customer or a partner. I like the candidate to meet a few potential teammates. They will work closely together, so they should meet each other.

    The hiring manager gets the final say. Hiring someone is a big choice, and the mangers future prospects will rely in part on whom they decide to hire. I will back the manager in almost every case. Almost, because managers can get so biased towards a candidate that they ignore obvious red flags.

    I have seen quite a few red flags in my time. You will do once you get into the thousands of interviews.

    — candidate lesson 1 — only apply for a job you actually want. I once had a candidate tell me they didn’t want this job as soon as they sat down. We all stood up, shook each other’s hands, and the interview was over. I was mystified by someone getting dressed up and travelling all the way across Dublin to interview for a job they didn’t want. It might have been for a bet. They may have been a secret agent trying to plant a bug in my office. Maybe they didn’t like my shoes. I still think about this one occasionally, ten years later.

    — candidate lesson 2 – a little diplomacy, please. I had one guy tell me he would rather not discuss the previous jobs on his cv. He was here to talk about the future, not the past. He would only accept questions related to the current job. I asked him how we’d build a strong relationship if he put a limit on our first ever conversation? He refused again and told me that his past was none of my business. He wanted a theoretical interview, things he might do in the future. I explained this wouldn’t give him the best opportunity compared to other candidates. Again, he demurred. After a bit of back and forth, he lost his temper and told me to ‘fuck off’.

    — candidate lesson 3 — the customer is always right. I asked one person what he would do if he designed a solution and the business person who was paying for it didn’t like it. He told me he would explain why they were wrong. And if they disagreed with his explanation? He would walk out. He didn’t like to deal with morons. He chose the word morons to describe the people who would pay his salary. I spoke to someone in the office who shared a lift with him after the interview. Being friendly, they asked him what he was here for. He told them it was an interview, and he nailed it. As he said this, he balled one hand into a fist, and slapped it into his open hand. Smack. “Nailed it”.

    There are no wrong answers, but there are bad ones.

    Getting different views helps root out obvious mismatches. It can also do the opposite and discover strengths that are unseen by others. It helps reduce one person bias, and though not perfect, is better than one person deciding on their own.

    Unless.

    Unless the most senior person in the debrief says they want to hire the candidate before anyone else has spoken. If that happens, people’s internal monologues whirr at top speed. “What happens to me if I disagree with the boss? Will it affect my job? Wow, I really need this job. A friend got laid off months ago and still can’t find anything. God, I have to pay rent. What will my partner say when I lose this job? I can’t go back to living with my parents again. Just go with the flow. He has been interviewing forever. He must know what he is talking about. Where did he get those shoes? They are disgusting. Well, he certainly thinks he knows better than me. He thinks he knows better than all of us. The bastard.”

    “Let’s just agree with him, say yes, don’t mention the shoes, and nothing bad can happen.”

    That is more or less what happened in the room. There were a couple of newer programmers who I didn’t know well. After I spoke, we started going around the room. I started with a new programmer first. “What did you think?” He replied, “yes, well, yes, it’s actually a yes for me. Yes. I thought they were good. Yes”. He said yes so many times in 10 seconds, it sounded like he was trying to convince himself, or maybe he had OCD. I asked him if he was sure. His mouth said he was, his whole body told a different story. Everyone said yes that day.

    I went home that night, struggling to forget about our flawed interview setup. By speaking first, I had influenced everybody. Then, as the agreement grew in the room, everyone started influencing everyone else. We are social creatures, and saying yes became more pleasurable. I wasn’t asking if you were saying yes to a candidate. The question had morphed into something else. The meaning had changed. I was now asking, are you with us or against us? Are you one of us or one of them? In group or out group? Right or Left? So the word yes gathered momentum. It sang out across the office like a church choir. “Yes”. By the time it came to the last interviewer, they said yes, and it felt like the last note in a chord. It shimmered in the air. Endorphins filled the room. The idea of interviewing a candidate was long gone, we had transcended to somewhere else. We were bonding together, creating a sacred communion between the group, strengthening our ties to each other, a glorious chorus of yes bringing us closer together.

    I wanted to forget about the day. I switched on the tv. The movie Gladiator was on. I sat back, relaxed, and forgot about the interview.

    That night, about 3 AM, I woke up from a dream, and I said “Yes”.

    The next day, I brought my team into my office. I told them about the new rule that came to me in a dream. After we interview someone, we may not share feedback with anyone else until the debrief. And the debrief itself has changed. At the start of the meeting, everyone points a thumb to the side. Then we count 3, 2, 1, and you put your thumb either up or down, just like Emperor Commodus would do to decide the fate of a gladiator. Only then would we gather feedback.

    Gladiator style interviewing.

    Once we tried this out, the pressure to conform to the group immediately vanished. We discovered what people really thought about candidates in a second. Hidden talents came into view, we disagreed, and debate flowed. We all learned one of the most important lessons that there is in life. We learned to listen. Our individual picture of the candidate was incomplete. We could only fill it in by considering more perspectives. This picture wasn’t perfect, but it was fuller and clearer.

    I fixed a broken interview setup.

    With a thumb.