Alooba Objective Hiring

By Alooba

Episode 63
Ed Sant'Anna on The Future of Hiring and Evaluating Adaptability in the Age of AI

Published on 1/16/2025
Host
Tim Freestone
Guest
Ed Sant'Anna

In this episode of the Alooba Objective Hiring podcast, Tim interviews Ed Sant'Anna, Data, AI and Digital Transformation Leader

In this episode of Alooba’s Objective Hiring Show, Tim interviews Ed, a technology expert with a passion for bridging business and technology. Ed shares his extensive experience in multiple roles, including engineering, product management, and AI. The discussion delves into the challenges of using AI in hiring processes, the intricacies of defining senior roles in data and analytics, and the importance of adaptability and problem-solving skills in candidates. Ed also introduces his 'chief tech translator' movement, advocating for effective communication and stakeholder management in complex technological domains. The episode closes with insightful thoughts on redesigning interview processes to better assess candidates' capabilities and a light-hearted mention of Jerry Seinfeld’s famous joke on public speaking.

Transcript

TIM: We are live with Ed on the Objective Hiring Show. Ed, thank you so much for joining us.

ED: It's my pleasure.

TIM: It is our pleasure to have you here with us, and I'm really excited to talk to you today or this evening, and it'd be great just to get a bit of an introduction about yourself. What have you been doing recently? And yeah, tell our audience a little bit about yourself.

ED: Thanks, Tim, and it's a really good pleasure to be here. Yeah, my background is everything technology and technologists by heart, having an engineering degree, but very early in my career I realized that I love technology and business, so I tend to operate in that between business requirements and technology delivery. I tend to call myself a tech translator, which is where things operate, so I've advised large enterprises, startups, etc., in everything from data to AI to digital transformation, and that's been my role really. I've done product management, and I have had loads of hiring opportunities. We will discuss that later and anything in between. I've worked many different hats throughout my career, and again, I am releasing this movement now that I call the chief tech translator, which is anyone who has that ability or need to take something very complex within their domain of knowledge and translate that to somebody else. and don't think that this is just a solution architect or someone like me. You could be a product manager, and you need to translate your tech to somebody. Capabilities of your product to your business: you might even be a lawyer that needs to take legal requirements to your clients, right? Science, medicine, etc. So I'm creating this little community so we can share tips and help each other progress and just progress the world. We just think the world can be very complex, and we need people to translate that as well, so that is another thing I've been doing.

TIM: That's a great introduction and maybe a good place to start and just hear a little bit more about why you have started this movement and what you think the real value is.

ED: Yeah, again, there is this requirement, right, this need, and we can see that from job adverts everywhere, in most senior to lead-type roles, there is this requirement of stakeholder management, the requirement of great communication skills. What does it actually mean, and are there any frameworks or tips or methodologies that we can all use to get to that point? And it can be simple things, like how to present yourself, how to provide something very concise, how to put yourself in the shoes of the other person to explain something that might sound simple to you but is complex to the wider world. Yeah, so that is a main reason I created this, because there is a requirement here, and it's been there forever. for these sorts of roles in the industry

TIM: And is it a skill that's in shortage, do you think? Is there just a lack of it, and we need more of it?

ED: I think so, yeah, it's really challenging. There is that very common, famous joke by Jerry Seinfeld, the comedian, and he talks about how from a statistic that he saw of what worries people, number one is public speaking, and number two is death, so basically that means that in any sort of funeral, people would prefer to be inside the coffin than actually doing any sort of eulogy. and I think that really reflects what I'm talking about, and public speaking is very important in these sorts of roles, and sometimes public speaking is just to 10 people in your organization, but a lot of people will struggle, so I think this is a skill that is required; it is really missing out there.

TIM: You already take the record as the first person to mention Jerry Seinfeld on this podcast, and I, for one, am very happy with that, so congratulations, and thank you for sharing that, Jake, which you delivered perfectly, by the way.

ED: It's my pleasure. I'm a big fan of cycling. I've always been. Still, the best sitcom of all time, I think,

TIM: and what about you? You mentioned a couple of little letters there in your introduction around AI—the two probably most said letters in combination now in the history of the world—and it feels like AI is changing everything as we know it: every industry, every role. It seems as though it is in some way changing, being impacted, or will be impacted by AI. What about hiring? I'd love to have a discussion there around the potential for AI and hiring. Have you had a chance to start dabbling with AI at any part of the hiring process? Have you seen candidates start to use AI as part of their application process?

ED: Yeah, that's a great question. So, seeing candidates, yes, many, and you can most often recognize the patterns of a cover letter or a CV that has been written by AI. Some of them may forget to remove the final little sentence from some generative AI tools that say, Did you enjoy it? Did you like this? What would you like me to expand so it becomes very obvious in terms of using it myself for hiring? We have actually used automation for video interviews, so that was really interesting. It was very specific, as it was a high volume of vacancies and a high volume of candidates, and it was also outsourced. so abroad again I'm based in the UK, and in that particular market, we would have 100 applicants within a single day, and it would just multiply over the next few days. The thing that we realized is that it's very difficult for our recruitment team to do the first screening, which is in a different time zone, right? So they need to wake up very early to talk to them, and for whatever reason, some candidates don't show up; there are laws, or it might just be a cultural thing. So we started trying to fix that problem, and what we did is we used a recording system so the candidate would record themselves for two minutes. just introducing themselves but also answering a couple of simple technical questions that would then be assessed by AI the transcript summarizes it as well, so it makes it much easier for the team to identify it; it also filters out those who are not fully committed to the interview process, right? So the ones that actually show up on video are the ones we put forward. Yeah, that was really useful. I don't know if that's been your experience as well in terms of AI in the process.

TIM: If I were to summarize the conversations I've had with people over the past two months, I would say maybe five percent have started to dabble with AI in any kind of evaluation like you've just described there. Like it's pretty rare so far where they have used it a bit more has been Oh, I want to create a job description. Great ChatGPT, give me a job description for a data analyst or whatever, so there are kind of content creation pieces. Give me interview questions for this role. Bye They start to use it that way, which I think makes sense because they're things that are almost like outside of a system where you can just go and do it and go, Now I'm going to add this to my job portal or ATS. So I'm going to use these as my interview notes. I think for the actual evaluation of the candidates, you need a proper product, a proper application layer that's got the AI embedded, like this AI video tool you mentioned, to actually get any value out of it, and then I suspect part of the challenge will be maybe why we haven't seen these tools adopted much yet is because that hits AI laws and then people laws as well, and so depending on the market that you're in, that might be very complex to navigate around both for the vendor to create the product and then for the company to start using it, so I feel like that's why we haven't seen much yet, but it's inevitable that it'll change. I think what do you reckon?

ED: Yeah, I agree. What is really interesting, though, is that you've described the hiring team creating the questions, and then I've seen many candidates just use AI to prepare themselves, so they go, Which questions would I be asked for this role based on the job description? So now we have this almost AI-to-AI conversation that is It's bridged by the humans really, but effectively the baseline has been created by AI, which is quite interesting, but look again, I'm an AI expert, and that's exactly the point, right? It's to simplify and keep things more efficient for us. One thing I would even recommend is What's the best way to describe myself? You can actually upload your CV to the AI prompted in this sort of manner and maybe even say, Look, I'm trying to apply for jobs that look like this. What is the best way to describe it? How would you adjust my CV or adjust my delivery to hit the points that I want from this particular job? I think that's really useful. What I wouldn't recommend is writing the entire cover letter on my behalf because then it's not you, right? Writing it and then asking you to adjust it and condense it gives you tips; that's different, right? That's using it as a correction mechanism rather than talking on your behalf. Yeah, that'd be my view.

TIM: yeah and I think that's a really helpful tip for candidates in this hiring market because any candidate in the world probably at the moment is going oh my God there are how many applicants for this role already there's 702 weeks for this data analyst role in XYZ company and so they're probably thinking geez I'm going to have to apply to a lot of roles to get an interview which must be confronting especially if you don't have a role at the moment that's just not a nice place to be and so they're probably then using AI to write the CV to maybe even apply to the roles but the problem is for the candidates maybe just saying Hey ChatGPT write the CV for me and make it perfect quote unquote for this job is going to make them sound the same as each other, and it's going to create the exact opposite effect of what they want, which is to stand out from the crowd. They're going to have this generic-sounding chachapiti in the realm of blah blah blah, using all those common words, and then that's not going to help them at all. As you say, it's going to be that they're going to lose that authenticity that their genuine voice

ED: Exactly, and you look at the volume of CVs you mentioned; that is really interesting, and it's very different now compared to maybe a year later. Yes, I think part of it is because there is a big number of professionals that are available in the market, that being redundancies in the tech world, but also I think a lot of it is because it's becoming easier to apply the easy apply button on LinkedIn. Yeah, your AI is writing things for you, but it doesn't mean that there are loads of good candidates for a particular job. I'll give you an example. I spoke with a recruiter that had 1400 applicants to a particular role, and it was a head of role, right? And I said, Okay, so tell me, how are you doing this? Is there AI involved? She said, No, we don't do any AI. Okay, interesting, but how do you filter down? Half of them are eliminated immediately from us asking the questions in the actual application on the basis of, Do you even have the visa to be in this place? Do you need sponsorship? That eliminates half of them. Okay, that's still 700. what next then she will have a look and eliminate those who have absolutely no experience at all related to that and that filters down to less than a hundred then she started looking at the ones that actually match call some of them and then she realized that half of them would be a lot of them they lied in that question so they do need sponsorship so long story short she ends up with four or five actual candidates that she can put forward to the the hiring team and it's just I think this hopefully this is comforting for those who are seeing all these different numbers and then to your point Tim I completely agree. You don't want to be just another person, so if you just use AI to create the whole thing, it will look like everybody else's. You have to put your own effort in; you have to put your own thoughts into the process.

TIM: We've got a thousand applicants because, as you say, it might be that more than 90 percent of them are completely irrelevant. So it's just a case of making sure that your CV can stand out from this sea of noise, really, which is still a challenge because, yeah, this person's done a filtering in a certain way. I spoke to someone else last week who said they'd had 1500 applications; they said we read the first 350, and we got four candidates to interview. and then that we stopped because why should we read the next thousand? It's still going to be a challenge either way. One thing that struck me is I feel like there's going to be this interim period where these job boards, or at least these kinds of inbound candidate flows, are going to be a bit broken because the candidates have adopted AI en masse. the companies There's going to be this delay, as we've spoken about, because companies take longer to make decisions and need to be more holistic; they need to be more considered, etc. There's going to be this interim period where they're dealing with all these high volumes, and they've got this screening issue. Does that mean then they're going to start to go You know what? We should go back to the referrals or the networks or the kind of shadow jobs, which I feel like we'll have some issues with making it fair because now suddenly it's not an open market, but maybe that's the right way to go for companies. Maybe that's the right way to go for candidates; if they have networks at the moment, should they be leveraging them?

ED: Yeah, it's a good point, and yes, I think they should because at the end of the day, it's a warm introduction for the hiring team. What that normally means is they will not stop advertising the roles, so they will still have the hundreds, maybe thousands, of applicants, but the referrals will be at the top of the list, right? So if I'm giving any tips to candidates out there, absolutely use your network and use those referrals to make sure you do get at least that initial conversation because if you are the right person for the role, Having that first conversation is the most difficult part, and if you can use your network for that, then you know most of the hard work is now done. You just need to talk to people, which is always a good thing. Yeah

TIM: That's as much as you could ask for from a referral. There's potentially, from what I've heard, for let's say some more senior roles have a more relaxed hiring process One of our guests on our podcast, Matt Roberts, was describing the fact that he found it almost ironic that the more senior the roles are on average, the simpler the hiring process is, which is counterintuitive if the amount the company is spending is much higher; the risk of a bad hire is much higher if you hire a director who bombs. It's not as if it's much worse than if you hire a junior data analyst who bombs, and yet in his experience he basically said, I haven't really gone through a hiring process for 10 years. like the last two jobs he'd gotten were close network referrals who'd gotten in my coffee with someone at McDonald's, and then they basically hired him. That's his example. His exact words Has that been your experience as well, that as you get more senior, the processes get shorter and sometimes less formal?

ED: That's very interesting. I have maybe mixed feelings about this. I think that up until maybe a year or two ago, so let's say 2022, 21, 22, that was 100 percent the case. I think that most very senior roles are still working this way, and look, they may not even be advertised right, but those that are being advertised, I might actually see, and by the way, this might be a bit of a geographic market thing, so at least talking about the UK I'm actually seeing a lot more risk aversion when hiring; the more senior it is, the more I'm seeing that the opposite of what you just said is true: there are more interview steps. They tend to have a panel, and if it's truly senior, then you have to speak with the C-level, and often that conversation isn't just the one panel. It's speaking with those people individually, so I have spoken to many of my colleagues who have 5, 6, or 7 interview steps for your attention, and don't forget that those job descriptions are very specific, and sometimes they're looking for a superhero person, right? So it's a head of that is strategic, has stakeholder management skills, provides top leadership, and goes on podcasts, right? or but also needs to do some business development, so that sounds like a sales lead, but also possibly needs to project to manage and build for their time if they're working for a consultancy that sounds like a delivery lead to me. By the way, don't forget to create some propositions and to find some go-to markets, or that's now a product person, but don't forget you have to be very technically skilled; maybe reuse some code. So it's, and by the way, come to the office every day, so those are the worst options, I think. But it's not uncommon, and I'm seeing this more and more, and even at that senior level, hiring teams are struggling to find the right candidate, but the question is, are they struggling because the candidates are not there? or is it because they're expecting too much from that role and combining two or three roles together? It might be a mix of both. I don't know if you've seen or heard anyone talking in that sort of

TIM: What someone in the UK had mentioned to me recently was like, Oh, this is a really weird phenomenon at the moment where in her network of other senior analytics leaders, lots of really good people are looking for jobs, and on the other side, she's speaking to all these companies that are like, We can't find any good senior analytics leaders. and so we're like, How is that possible? That doesn't make sense. My thought was, and she mentioned this, maybe economically, that the salary is too low; the salaries have dropped, and so the people are in the market a year ago for 20k more, and now that's below what I'm willing to accept. The companies are offering too low, and it's just the salary needs to come up. It's as simple as that, do you think?

ED: I think this is one aspect, again, in product management. You have your product-market fit; I think this is a candidate job role fit problem. Yes, it might be actual salary, but I think it is the actual role as well, and in this industry of data and AI in particular, especially the more senior it becomes, the fuzzier it becomes in terms of job roles and job titles, right? So maybe a job title that feels a bit junior actually pays more than one that feels extremely if you have a lawyer is a lawyer, right? Medical surgeons are medical surgeons, whereas a lead data person—what does that mean? It can be a technical lead, for example, so a principal data scientist might be someone who is supporting other data scientists and has to have the ability to open the bonnet. Understand the code, understand which algorithms are being used, etc. but it may also have a lead data scientist position or head of data, where the whole role for them is to manage the business, understand the use cases, and work with the data science team, but he or she doesn't have to open that bonnet anymore, but I'm seeing loads of roles where these things are conflated now. I think some companies are losing that perspective of the trade-offs, and if you think of it naturally, some operate more strategically. It is likely that for the last 5, 10, 15, or 20 years, they have never looked at coding, and I think that is the mismatch, so the people who are now in the market are those who have done a fantastic job by being strategic, and by definition, they can't really code anymore. but the new hiring teams for some reason or another, I expect, in that coding knowledge, plus everything else, this is particularly the case where it's a new team or a new role, and that is especially for AI, so if you are, many companies have started building AI bottom up, so they've hired the AI engineers, the specialists, etc. and now they're feeling like there's a need for someone to lead the team. They have no reference of what that leadership looks like, so they may be asking that team of hands-on people to define the job description of their manager, right? Which may come bottom-up rather than top-down, that is what I've been seeing a little bit in my just observation, but it feels like that is the mismatch that we're seeing in the market.

TIM: Yeah, you reminded me of another conversation I had with someone also in the UK just a week ago, and they were talking about part of the gap in defining or companies hiring the senior analytics roles as well. Let's say you're hiring, I don't know, a director of analytics or something, and they are already—they would be the highest person in analytics or data in the company. then who's deciding what that role looks like? Obviously not someone in talent acquisition, because how would they know the CEO is the CEO? Who's the one who's actually putting together that role spec, and it's probably a little bit different compared to I'm sure companies run into the same issue when they're, I don't know, hiring a marketer, the highest person in marketing in the business, but marketing is less technical than data, and it's evolving less quickly. So maybe it's just because it's evolving so quickly that you end up getting these hodgepodge roles where it's just a Frankenstein of different things because they've done a Google search and looked at a few different JDs and munched them together.

ED: Yeah.

TIM: And then, yeah, what's the way out of this? One option is, yeah, get the juniors to start interviewing their future boss, which seems pretty perverse. Hire a consultant to then advise and come in and at least do the interviewing and help craft out that role. How should companies navigate through this? Who should be the one to decide what that role actually looks like in the first

ED: Yeah, this is interesting. Maybe this is now looking back to the chief tech translator aspect, so you need someone to translate that. It doesn't have to be someone external, but it tends to help. I think breaking down what it is that we need as a company is extremely important, right? So companies AI, specifically companies that have never done any AI, have no idea what to do. Maybe what they need is they need to find the person who has that strategic mindset who set up the function, and that is their head of AI, step one, which may be there for two years, and maybe it is a temporary contract; maybe it is an external person to then think about a delivery person who is more someone who's been doing it day in and day out, and they know how to manage and even to code sometimes. So breaking it down is the key to your point. I think maybe they're using AI to create that job description as well; just conflating everything together means that you have this mismatch again, and you're missing the candidate job market fit, right? But candidates need to help hiring teams as well. I have had these conversations, particularly with external recruiters, and they know that external recruiters are specialists in recruitment. They tend to be very good at being close to clients and actually pushing back on them. You mentioned a salary for this; for what you're asking for, you're not going to find someone like this. So let's prioritize what you need the most. Is it technical skills? Okay, so let's find a technical person. Right, it is very important to define these things. I think the biggest sign is when you've spent six to nine months trying to find somebody; you've been through hundreds of interviews. and you could not find anybody. That is an obvious sign that something is very wrong, and it's usually as a hiring team, as a hiring manager, I will assess what my expectation for the role is. It's probably wrong, right? Because it's impossible that there's nobody available, right? Yeah, that would be my view.

TIM: You're right that, yeah, an experienced expert recruiter will give that feedback and that pushback, which is really helpful, and part of their motivation for doing that is normally they get paid a contingency fee, so if they can't place the role, they don't get any money, so they're highly incentivized to make sure you're not running a delusional search process looking for a Frankenstein candidate. you're going to underpay I wonder whether maybe the difficulty is if you go through an internal talent team, they don't necessarily have the same ruthlessly aligned motivation. They might not necessarily know if I don't place this role, it's not like they're missing out on a 50,000-pound bonus. whereas for a recruiter, they have that always in the back of their head: if I can't place this, I'm wasting my time, and so they're laser-focused on any sense of this role is going nowhere

ED: Yeah, I agree, and I'm a big fan, especially when it's a specialized role, right? And particularly if your organization is now moving to a new area, so absolutely, you've never done data in AI; it's fair; it's asking a lot from your internal recruitment team to find that person, find a specialist. and I have spoken to some specialists that will say no to clients because, to your point, they know that they will only be paid if they find somebody. If the job description makes no sense, they will not find anybody; they will only be spinning their wheels, so they qualify them out, and I think that is really powerful, and it's a good sign; it's an interesting sign as well. If you're hiring a team and the recruiter is saying no, I can't work with this, you know that you need to go back to the drawing board and figure out something different.

TIM: Yeah, I've got an interesting example of this, not for a senior role but for a more junior role, where there was a company that came in and chatted with us, and they had no data in that company at all, no BI, nothing. This would be their first data person in the entire company, and so they had apparently been trying to hire this person for six months and hadn't really gotten anywhere. They'd gotten a referral to us just for some feedback on, like, how they're doing the process, and so they said, Oh, we feel like there's something wrong with the testing step. That was their sense based on where people were falling down in the process. Oh yeah, some candidates, they don't seem to like it, or they're pushing back, and so they shared their skills test with me that they were sending to candidates, and it looks reasonable. It's all kind of fine, and we got down to one question where it said, Oh, using VBA, do this particular data analysis task, so they require candidates to specifically use VBA. now I don't know about you but in what 2023 or whenever this was if you're a data candidate you're like I have to use VBA in this role hello Excel hell get me the hell out of here that would be the reddest of red flags it would be like a waving oh my God I don't know what for any candidate with any experience that would be just I don't want to be living in this hell and so something as subtle and simple as that can derail the entire process so imagine for a senior role with really experienced people who could smell a sham from a mile away if they see this these Frankenstein jobs that you've described that's going to obviously deter them from going on with the process if they've got a bit of leverage

ED: Exactly, that's very interesting because, yeah, the other thing I would say is be very clear with what you want and what the role entails, so I'm just looking here because I have done a little bit of research and comparing just data scientists roles; they just say data scientists, so job description one, for example, simply says data scientists with a background in data analytics and visualization. Comfortable beauty reports and visualization, and then as a data scientist, you think what's going on, so it makes no sense. others may actually describe yes, you'll be doing predictive analytics and machine learning, etc., but describe some use cases maybe or Describe the industry I'll be working on because that will, again, if you want to find the correct talent, you want to excite them about the job. And if you've ever spoken to a data scientist, the last thing they want is just to use most of that; they spend most of their time cleaning data, so that's a data engineer's job, but they will love it, by the way, so again, you just need to define those; those are some basic definitions, even though I just said earlier that it's a little bit fuzzy, but some data engineers, data engineers, data analysts, and scientists, please don't home play. them right

TIM: And yes, I think these differences to you and me are very obvious, yeah, like the black-and-white, night-and-day kind of differences; however, I could easily see how for talent teams and HR professionals these are subtle variations, like, yeah, whatever, data wrangling versus predictive modeling. Yeah, okay, calm down, speccy. Get back in your corner. I don't want to hear about these complicated details, and to me, I feel like we're almost setting up talent teams and HR teams to fail, expecting that they could possibly understand these, what to them must be subtle differences. and that's just in data. You can imagine a similar thing like a backend versus front-end engineer. Yeah, whatever, you're all coders. That kind of perception, and so the fact that they are the ones controlling at least those early stages of the hiring process—the screening, the filtering, the CV screen, the first phone screen—to me is a worry and a concern because unless they're really specialized or you've gone through a lot of effort to help coach them on what you need to look for, it must be so hard for them to do that. What do you think?

ED: Yeah, absolutely. So we need to be very specific about what we want and also have the right people with specialism having a look at it, and then there's the aspect of how many stages of interview do you have. I have seen I've spoken to some candidates that companies have lost them because they had too many steps. and I, it's not uncommon to see screening plus four or five steps, right? It's very difficult for me to understand why you would have anything more than two steps, right? So the screening you already should be able to understand a bit of cultural fit, certainly communication skills; some of them match the skill to the actual job description, but to your point, it might be a keyword. type of match, but then the next stage you should be able to assess them technically properly, right, either through tests, standardized, specific, etc., and/or interview and conversation, right, and that tends to be with a peer, and then finally you have a more A A wider conversation about prospects, about growth, about how you collaborate with your hiring manager or hiring teams. instead what I'm seeing is you have you break down several of these stages into three or four, so one is just the exercise, then you have one conversation with the peer and then a second conversation with the peer, and then you have a conversation with the hiring manager, the manager's manager, the CEO, and then Oh, by this point that candidate is really exhausted. And especially talking to those who have been made redundant, they are now applying for several jobs, so they have this happening multiple times during the week with multiple different companies. We need to recognize this aspect as well, right? It's very important to not overdo it either.

TIM: And it's so amazing how quickly things change because this is really driven by the market conditions because if we went back two years, it was the polar opposite. It was like we cannot test candidate skills because we don't want to even give them a chance to drop out of the process; just do a quick interview and get a hire and offer in front of them. Sorry, as soon as possible. Now we're at the opposite end, but it feels like the companies have forgotten a little bit about the trade-off and the upside of having a shorter process and not dragging candidates through the ringer. That's one thing, and then I'd also say once the process gets that long, it's not even a trade-off question. It's not even that we'll have a longer process, but it'll be more accurate even if it's slower. I don't think so. After four or five interviews, there's no difference. You're not going to learn anything new, so it's just worse; there's no trade-off.

ED: Yes, I completely agree, and I would even say that you may say no to great candidates purely through exhaustion, them being exhausted of the process, but also the more you speak to people, the more you're going to try and pick the problems right, so the other trade-off I've noticed is the expectation that the candidate has to be an absolute 100 percent fit. Whereas if you think about it, the main reason people change jobs is to stretch themselves, so by definition, in my next job, I want something that is only 80 percent there because the other 20 percent I will develop, but this is interesting. I don't know what your experience has been, but I've seen a lot of these It needs to be the perfect fit, or they don't know enough about this particular technology, but I personally prioritize the capacity to adapt and to be agile, especially in hard skills. Before ChatGPT was released to the public, most people had no idea what a large language model was. So everyone had to learn in the last two or three years, so expecting now everybody to know everything about the latest technology is a little bit unreasonable. I would actually prefer to get someone who has the capability to put some problems to them to see if they can solve them; their capacity to assimilate and develop it is a lot more important than whatever skills they've acquired previously. But anyway, I'd love to hear otherwise what's your experience on that.

TIM: Yeah, I agree completely, and if I could summarize people's thoughts in the last couple of weeks when we've been talking about the future of these roles, like what is a data analyst even going to be doing in a few years? What skills do we hire for? Now everyone's saying, Yeah, adaptability has got to be pretty high on that list. ability and willingness to learn new things because, as you say, if the technology is changing so quickly, it doesn't make sense to fixate on whether they know X, Y, or Z school skills. Sorry, because they're changing so fast, one thing I'd like to share is then how to evaluate that in the hiring process because I think it's difficult. a lot of the time it will come down to behavioral interview questions where you're trying to get an understanding of whether they have a pattern in the past of learning lots of things quickly and whether they seem excited to learn new technologies. Normally, that's what it comes down to, but I feel like that's a This is a general problem with interviews: we rely on taking the candidate's word for it at a certain level. like it's all about talking, they feel like talk is cheap for our business. We identified that our single most wanted trait in a candidate was that ability to learn the same kind of thing because your startup has the same problem: things are changing all the time, and you just have to get on with it. And so we tried to think, could we change our hiring process to somehow select for that trait? What we came up with was interesting, not really that scalable, but it's still interesting for our software engineers. We gave them a problem in R, the statistical language, knowing that no software engineers use that; it's just a stats and data language. So we knew that none of them would know that language; pretty much we gave them a relatively simple algorithm to solve, but they would have to go and learn a brand new language from scratch, and so we were thinking, like, Oh, okay, how are they going to cope with that? Some might say, Go to hell. Why would I want to learn this stupid language? I'm a software engineer. Okay, probably a red flag, I would say. Others would just not be able to figure it out. They'd be like, Oh, how do I install R on my computer? I don't know how to library or whatever. Okay, also not a good sign because it's not that complicated. If you're already a software engineer, to learn a new language, and so through this process we managed to get candidates who, from scratch, could learn a whole new language or at least enough to do this problem and want to do it and successfully do it to solve a problem, wow, amazing, that's an end-to-end demonstration of exactly what we want. So I felt like that was an interesting thing, but it's just not really that scalable. I don't know how to repeat that across a marketing role in accounting or this role or that role. domain-specific Yeah, have you thought about that adaptability that learning is? Is there some way we can coach that out in the hiring?

ED: Yes, so I do like the idea of posing a problem. Yeah, encoding this is a great example for a non-coder. It might be just an existing problem in the market, right? So it might even be a problem that you need to solve in that new role, so for marketing, plus it might be how do we describe our new AI capability and take it to market. and I do give these problems ahead of time as well because it becomes real-world, right? That's the other thing. I've seen interviews where people want to catch somebody out, so they ask the question immediately, so you need to have the answer right now, but that's not real life, so pose that question a week before the interview and ask them to present or even whiteboard it for you; that really gives you an idea. And then you pose questions, as in, tell me how you learned this, especially if it's a marketing role where a person has no AI background, for example, just that thought process on how to actually acquire that knowledge, how to break down the problem, and I really like when they are humble and they say, I had no idea. I had to find out, and by the way, I loved finding out more about it. To me, those are all green flags, and to your point, the red flag is when the person says, Look, I'm not an expert; I don't want to know. Oh, that's bad. Next! Yeah, so it's an interesting question.

TIM: And for companies who At the moment I'm a little bit concerned about take-home projects and thinking maybe they're null and void now because the candidate could just do that with ChatGPT, and so am I really getting an understanding of their skills? I think a good tip there is, yeah, that's just the first bit, though. The second bit is then presenting it to you and going through it in real detail live, which they cannot fake, so if they've just superficially whacked something through Chachapiti, that could be a coding challenge; that could be a project; that could be anything. Yeah, they're going to come unstuck pretty quickly, but if they've actually understood it and used AI to enhance their answer, get feedback, or generate ideas, then they'll be able to stand up to that. It's going to say interrogation members"—not the right word. That deep dive that you might do into their thought process in the live interview, you can't really fake that.

ED: Exactly, and it's all about results at the end of the day, right? So you can come to me and say, I'm an expert in AI, and I've done all these things, okay, but what results did you actually achieve? And in this case, for an interview, we pose that problem: they come and present to you how concise that presentation is. Do I care if they've used AI to create the presentation? I don't think I would actually applaud that. That's not a problem at all if it looks right. The actual delivery is good if they feel comfortable in front of an audience that they haven't seen before and if they can really hit the numbers right. So did they actually understand the problem and address it, or are they just throwing a bunch of technical jargon on slides? Those things are very important to assess, and even though I'm an AI person, I don't think we can today use AI to assess that. This is the point where you need a person to evaluate.

TIM: Yeah, I agree we're not quite there yet with AI. I think it could be in the not-too-distant future. I feel like we're on our way. I personally feel like humans make a lot of hiring errors and are a cause of the bottleneck in a lot of the process that AI wouldn't be, so I feel like there are probably some pros and cons at the moment. My guess is that there'd be more pros than cons in adding AI to most of the hiring process personally, and I didn't feel that way six months ago, but I'm certainly an AI optimist at the moment, and I also feel like a lot of the I don't know what you think about this, but I feel like sometimes we put technology on too high We expect too much of it, almost like with driverless cars. I think the rate of accidents is way below humans, yet if a driverless car has an accident, it's on the front page of every newspaper, so we're expecting perfection from them even when we're so terrible, and I feel like hiring similar is so flawed at the moment. like it's so unfair in some ways that I struggled to see how AI could make it worse than what it is, you know what I mean.

ED: Yeah, it's a good point. Yeah, there are so many accents already to your analogy, right? So how can I make it worse, or, on the contrary, it makes it better? Yeah, I think that one of the key points here with AI in general is that there's still a lot more potential in the technology itself, available potential, not thinking about developments coming. The capabilities right now are much bigger than what we are using it for, so the analogy is we are taking a formula one car and using it just to go down the shops for milk and bread, but there is an analysis paralysis problem because this is moving so much and there's so much development happening that people are not doing enough with what we have today. So that is part of what we need to help everyone achieve; that's the key challenge.

TIM: Yeah, and I feel like it's just a matter of time before this catches up because maybe this is just the sort of application gap—like the technology is there; it's now just the very specific bespoke business applications that need to be built on top of it, which takes a bit of time, and then the adoption of a product—any product—takes a bit of time, a bit of risk aversion, a bit of AI fear mongering, and a bit of this is on people data. So once you work out those, maybe 18 months down the track, then we'll have probably some widespread adoption, I would have thought, of kind of HR AI tools.

ED: Exactly, and the key thing is we need to start from asking the right questions, and the question should never be, Where can I use AI in this area? It should always be, What is the problem? So you mentioned there are already many challenges in recruitment, so when you want to fix these problems, maybe the answer is AI; maybe it's not; maybe it's a combination of AI, other tools, processes, etc. and that's where I'm very passionate about the whole cheap tech translator aspect; it's about starting from the business challenge, or if you're not in business, your patient challenge. If you are a doctor, write your client's challenges. If you are a lawyer, it's really not about the usual expression. I have a hammer; everything is a nail. I have AI; everything needs to be worked on. No, that's not how things operate, but you need to have that tool in your background knowing how it works because that is one of the things that you have in your arsenal to address problems. okay

TIM: I have said the same thing. I'd like to just throw a slightly different perspective on that at the moment, which is I can think of a few companies I've spoken to recently, maybe three or so, who basically told their entire organization, Stop working. Just think about how you're going to implement Claude or Chachapiti in your day-to-day job, just like from first principle Sit down and go, What am I doing? How can I use this groundbreaking technology, which is like a solution searching for a problem? This is not normally the way I think of going about it, but maybe because the technology is so transformative… If you don't sit down there and just think, Hold on, is 90 percent of what I'm doing now pointless or automated or whatever? and maybe we could achieve some higher level, maybe then there's something to be said for, like, just blunt instruments whacking everything with AI because it's probably so amazing now it could solve a lot of our problems. what do you think

ED: That's very interesting. I think the key is to find the balance, and if you have someone who can facilitate that conversation, then it's very different. Okay, down tools. Let's use AI. We may get people a bit confused, but down tools, and by the way, let's have a think about stuff that we do that's very repetitive because the person who knows AI knows that repetitive tasks are the first thing that you want to automate anything automation really Ah, okay, so I have to click on this copy that email to this form 100 times a day. Can I fix this so I think that is better? I agree this is similar to having a car; right? Someone presents to you a car, and you go, I don't quite know what to do with it. Oh, I can go faster to a destination. Okay, so what was my problem? My problem is I was always late, or my problem was I had to wake up yesterday to then walk or use a bike to get to the destination, and now I can get there in half an hour. It's important to find the combination of both right and bridge these two areas.

TIM: Is there also maybe even a higher-level paradigm rethink? Sorry, that's using a bunch of bullshit words. Imagine the old horse-to-car problem: we can't be fixated on a faster horse, or we need to find a better way to clean up the horse shit, or more stables, or no, just forget about the horses. The horse is dead. The horse is done. this is a

ED: Okay, yeah, I think it is, and that's why you can't really think just technology; you mentioned BS. Now let's use some consulting thing, which is people process technology. You've heard of that people process technology, so if you look at just technology, it's a bit of a challenge, so a good example is if I look at a pharmacy in the UK, some of them will still have loads of prescriptions that are on paper, and if you start saying, Let's use AI here, you can say, Yeah, we could, but why are we used to using paper? That's the process, right? So instead of paper, it's a system. Okay, so now we can use AI, or we can say the problem of the paper is that it comes from the doctor as a paper. Okay, in which case can we scan it, make that digital, and then use AI on top? And by the way, I do have a case study that was exactly that, so it's a pharmacy group that needed about, I think, 50 people in general in total to do these manual prescriptions, and the solution was exactly that. scan it Convert that to text that is readable digitally. Use AI to summarize it and connect to the system as it should, which is what they were doing, typing it in, copying, pasting, etc. You know what happenED: now you don't have 50 people; you only need 5 to do it now, and now you're going to say AI is taking people's jobs. Oh, that's not true because there's such a shortage of pharmacists that the vast majority of these 45 people are now being retrained to be pharmacists, so to your point, a complete paradigm shift to your point on how this company operates, so I agree this is the way we should think, and that basically just going back to the Formula 1 car to go for milk That is the problem. I think most people are thinking, Can I use AI to summarize my text or to check my wording? That is a small beer where you really wanted to find the transformation aspect, and for that, you need to actually top down conversation from the company to think about the entire people process and technology stack.

TIM: Ed, this has been a fascinating conversation. I have one final question for you, and that is if you could ask our next guest one question, what question would that be?

ED: So I think it will be exactly in this area, which is how can we redesign the interview process to truly assess a candidate's adaptability and problem-solving skills? This is a long-waiting question to be answered, I think, but I would ask that question to the next guest.

TIM: I will ask that question to the next guest. I will also ask Chachapiti and see what it thinks; it might have some ideas for us. Ed, it's been a great, relaxing, and engaging conversation, and I'm sure our audience has really enjoyed it. Thank you so much for sharing all of your thoughts, insights, and wisdom.

ED: Thank you so much, Tim. Thanks for having me, and yeah, it's been a pleasure.