Interested in this topic? Visit the links below for more resources:
Visit The IIA's website or YouTube channel for related topics and more.
00:00:02 Speaker 1
The Institute of Internal Auditors presents all things internal audit tech in this episode, Charles King talks with Emraan Mula about the transformative potential of AI and internal auditing. They discussed AI's evolution from basic automation to advanced applications, the challenges of adoption.
00:00:21 Speaker 1
And the future of internal auditing. The conversation also covers continuous monitoring, risk assessment and the role of human judgment in an AI driven world.
00:00:32 Speaker 3
Imran, I am so excited to have this conversation with you. I feel like you always bring so many great ideas and sort of forward thinking. And what what you do at CAP 1 is is really.
00:00:42 Speaker 2
Really impressive. It's good to be here and talking to you. You always bring out the best conversation. So excited for this.
00:00:48 Speaker 3
One excellent. All right. Well, when we were.
00:00:51 Speaker 3
Back at GAM, we had a session on AI with Anton Dam from Audit board and you know, one of the things that was really eye opening for me in that session is every time I do a session on AI, I asked the audience how many of you are using AI at work.
00:01:08 Speaker 3
In historically, over the past year, let's say it's been roughly half maybe, maybe a slight majority, 60% of people raise their hands. When we were having that session, it was 3/4. At least maybe more.
00:01:21 Speaker 2
I do think like a lot of more ordered shops have gone ahead and got their auditors, Jenny I, to sort of the white label.
00:01:28 Speaker 2
And just out-of-the-box, Jen. Yeah. So I think definitely the conversation has changed just from few months ago when we did this. You know, it was maybe 20 percent, 30% to 6070%. Yeah, we're seeing more people with Jenny I in their day-to-day what they do with it is another.
00:01:44 Speaker 3
Question. Right. Well, I think that's that's exactly the right question to ask. How is it being used?
00:01:48 Speaker 3
Because when we asked the question about how many of you are using this for assurance related activities, the number was a lot small.
00:01:55 Speaker 3
And that's one of the things that we see in our work with clients as well is that people buy AI tools, but really widespread adoption, the kind of adoption that's leading to really transformational outcomes. We're not seeing that yet and I know you were an early adopter and definitely I consider you a leader in the space.
00:02:16 Speaker 3
How are you thinking about driving adoption in your team and making sure that you're getting some of the ROI from these AI invest?
00:02:26 Speaker 2
I think like the sort of common pitfall in adopting AI is going to auditors and asking them what we how are you gonna use it, right? It's similar to the conversations that we've had on analytics, right? Do you have a auditor asking an analytical person like how do you want to use analytics? It often comes down to.
00:02:46 Speaker 2
People want to, I always say AI is the new outsourcing, right? You've been in the era where automation was the new outsourcing. So now AI is in new outsourcing, a lot of auditors.
00:02:56 Speaker 2
Want to outsource to AI what they find tedious? You know, hard to do, but that is not where true transformation I think where true transformation is is reimagining order and thinking. If AI can do all the things that auditors could do, we're doing manually what should an audit?
00:03:17 Speaker 2
Job do what is highest value. What are the biggest insights we can bring to our execs, boards, stakeholders, and how do we reimagine how we.
00:03:27 Speaker 2
Work. So in terms of thinking, how do I sort of encourage teams to think about it as start with the end and work backwards? Don't start with, OK. I have Jenny. I what could I use it up for? And don't throw everything at Jenni. And what ends up happening, which we I think I see it. You you probably see it is most people use it to.
00:03:47 Speaker 2
Wordsmith reports issues and things that they spend a lot of time doing. True, there is some value in that, but it does not fundamentally change the value of the audit function.
00:03:57 Speaker 3
Yeah, I sometimes think of it as the the paradox of quick wins. You can go say ohh we did all these things and they, you know had some incremental value, but quick wins rarely transform the function and get these kind of radical improvements and efficiency and effectiveness that we're looking for from these big technology investments.
00:04:17 Speaker 2
100% I think the it's the. It's quick, easy, convenient for auditors, but in that you missed the asking. The bigger questions of how do we truly transform the value that audit brings to the enterprise? Yeah.
00:04:31 Speaker 3
So you and I both love technology. We both think that internal audit is you know, right in the middle of being disrupted. I'd love to look out, you know, let's look out 10 years and let's talk about what is an internal audit shop look like 10 years from now when when all of this agentic, AI and AI and all of this is.
00:04:52 Speaker 3
You know more fully embedded into what we're doing, how is internal audit changed in your opinion?
00:04:58 Speaker 2
How I think of it is in 10 years from now, internal audit functions and everything that how an internal audit function operates will fundamentally change.
00:05:08 Speaker 2
Let me explain a little bit, give it a bit of color so everybody talks about agentic AI. So you in my view you will probably not have one autonomous AI doing it all, but instead of that, what you will have is a core AI that knows your enterprise policies, knows your processes.
00:05:29 Speaker 2
Knows your wrist, so you think of it as as generalist expert on all things related to your enterprise.
00:05:36 Speaker 2
And what that will combine with is sort of small language models that are specialists at certain things. So there's a small, small language model that is a specialist at evaluating third party contracts. There is another small language model that's an expert at evaluating BCP business continuity programs and essentially.
00:05:57 Speaker 2
Mythology will become will include audit mythology will include a library of prompts and sort of AI tool calling that.
00:06:06 Speaker 2
When you are in a certain part of the, either the planning process, risk assessment, certain agents come back and give you sort of, here's the key outliers. Here's the key risks here are the key themes. Here are the key trends that we want to go and evaluate and auditors job then becomes instead of going and collecting all these data and doing stakeholder interviews, it becomes about.
00:06:27 Speaker 2
Investigating and opining on based on these data points, where do we think risks are? Where do we think our controls are in terms of mitigating?
00:06:38 Speaker 3
So that's a pretty big change from where we are today. So I wonder you know that that's going to take some investment that's going to take some change management. You know if if I I came to you and said you've got an unlimited budget for innovation right dollars don't matter, we just want really innovative next generation internal audit talk to me a little bit about how do you.
00:06:59 Speaker 3
Prioritize those investments. Where do you start? What should you? What should other internal audit leaders be focused on in terms of bringing that future to the present?
00:07:09 Speaker 2
I think like the mistake most people make is they pick the tool and then they decide what to do with it. So you picked Kenya then like what can we throw at Kenya? I think that's the wrong way to think about it in order. You should always start with the risk. What are your highest risks? Then you find the tool or technology or AI that is appropriate.
00:07:29 Speaker 2
For that. So in lot of cases you might be better off not using 10.
00:07:34 Speaker 2
Right. Because the technology is not mature enough to give you what you need. It might be machine learning or NLP that gives you more predictable, reliable outcomes that can sort of fundamentally transform your process. So in terms of where to start, you want to think about it. Will this be better into my process?
00:07:54 Speaker 2
And does this now become the default process by which we do this? We cover this risk if it is something that sits on the side that is a nice to have that you could still do it without and there's nobody would know. I think you've not realized the value.
00:08:10 Speaker 3
You know, one of the things that we've been doing at KPMG is really kind of breaking down the internal audit process start to finish, right. We have a whole process decomposition and we're going through every one of those tasks and we're saying.
00:08:24 Speaker 3
Which of these can be technology enabled, right, not just Gen. AI, but really looking at the the full stack of, you know, Gen. AI is is a piece, but there's, you know I'll say traditional AIML kind of solutions.
00:08:37 Speaker 3
Basic analytics that a lot of internal audit teams already have. Automation you know, whether it's RPA or workflow, looking at your audit management tools that you have and thinking about how do you really optimize the use of those and and get value out of them and then going through and saying, well, there are some things that we can 100% we can automate this or we can use.
00:08:59 Speaker 3
An AI to perform this task.
00:09:01 Speaker 3
And there's other tasks where we can use some kind of technology to partially automate it, and there may still be a human piece and there's pieces like stakeholder engagement that we may not ever want to automate because those are the parts of our job that we think are really important and where we still need a lot of human interaction. And as we think about.
00:09:21 Speaker 3
Moving through that process, we try to layer in a view of, well, what is going to be the biggest return on that investment and return. Maybe that may be dollars return may be a little bit more intangible.
00:09:34 Speaker 3
Like you know what is going to create more job satisfaction for our people? What is going to take something away that they maybe aren't as interested in doing or make the doing of that a more interesting, more rewarding activity or what is going to add value to our other stakeholders, right? Not people on our team, but what are other executives going to see?
00:09:55 Speaker 3
As valuable from the internal audit.
00:09:58 Speaker 3
In terms of the value we deliver to them, maybe in the form of reports or insights, but also in terms of the interactions with audit stakeholders and reducing the amount of time they need to spend on an audit reducing the the non value add time, they need to spend gathering data for us or answering questions for us rather than having a really meaningful.
00:10:18 Speaker 3
Dialogue and that's sort of been my approach is if you really kind of flow chart out on audit and really pinpoint those areas and just move through the life cycle of an audit, I think that can be a really valuable exercise.
00:10:31 Speaker 2
I think that is the right approach. The only thing I would add is as you go through this step like I think it's important to go back to what is the true value of performing this process right instead of getting caught in the mechanics of it. I think the mistakes like people do hear that, but then they got get caught up in the mechanics of like step one, Step 2, Step 3.
00:10:51 Speaker 2
Can we? Which of these steps can we automate? But I think that can distract the conversation away, right? And it gives you something very quick and easy to latch on to and solve for, but doesn't truly, you know.
00:11:05 Speaker 2
Up the value, so if you're doing risk assessment, what you're trying to opine on is what is the highest risk? How? What evidence do I have that we should be more concerned about these risks than we were six months ago and sort of having a data-driven view of risks across the entire life cycle?
00:11:25 Speaker 2
I think that is where true value of AI is, because ultimately what AI is able to do better.
00:11:31 Speaker 2
These take patterns, identify patterns. Take these patterns and extrapolate them and look across the enterprise to identify where else could this be happening. These are not easy questions without the data. Without the technology to answer, and those that is where the highest.
00:11:47 Speaker 3
Value. Yeah, I think you're exactly right. The like the risk that you run into when you take this.
00:11:52 Speaker 3
Very mechanistic approaches that you sort of do in back in the RPA days, we used to say don't pave the cow path, right. And I think the same the same risk is true. If you have a very traditional internal audit approach and you go through step by step, you may actually end up automating things that don't need to be.
00:12:08 Speaker 3
In the first place, in an AI driven or data-driven world.
00:12:12 Speaker 2
Exactly. I think like The thing is like if you're automating something that doesn't even need to be done, you can you can provide a higher value by sort of reimagining your work. I think automation makes it easier to be stuck in the past.
00:12:29 Speaker 2
Like I think we think of automation as the way to the future, but sometimes that's what's keeping you in the past because you're not rethinking, OK?
00:12:36 Speaker 2
If I can do something else altogether, why am I still, you know, because now you have said these are the steps and this is the automation you have to use and now you're stuck in this process and that's all you.
00:12:47 Speaker 3
Will ever do. That's such a great point. Can you think of a process at as you if you thought about Capital One, you thought about your internal audit process?
00:12:56 Speaker 3
Is there something where you can remember technology fundamentally changing?
00:13:01 Speaker 3
Your approach where you you either you know to the point we're making stop doing something that you used to do or do things in a completely different way because of the technology that your teams have available.
00:13:12 Speaker 2
I think like in general looking out across my audit experience, I'll give you a few examples for continuous monitoring, for example, there was a time.
00:13:21 Speaker 2
Where ordered and I know you have a hot take on this that if you're doing continuous monitoring, you're managing somebody else's process, but uh.
00:13:29 Speaker 2
Definitely like that was the thought process for a while on continuous monitoring that you do something, you do the same thing every quarter and you're buying on the same thing, but ultimately what ends up happening is your even your stakeholders know every quarter. These are same questions. They're going to ask me and it is now no longer. You've no made no changes to your plan.
00:13:50 Speaker 2
Scope and you're just running this process.
00:13:53 Speaker 2
And ultimately, going from there to data-driven and letting the data tell you what are the outliers, what are the emerging risks that we've not seen? We don't know how to opine on these or document these, but we're going to go and investigate and do targeted audits where we think there is a risk that we have never seen before.
00:14:14 Speaker 2
We think that could end up materializing. That's one example. Another is, I think going back to sort of.
00:14:22 Speaker 2
A lot of people talk about the audit processes, but thinking about assurance itself ultimately doing audits is the audit department's job. But I don't think people think enough about assurance itself. So if you think about something like going from a sample of 5000 to doing a full population test over billions of transactions, what?
00:14:42 Speaker 2
Ends up happening even if you find something that is outlier of 1% of that population. It helps you investigate the root causes and get to the true deficiencies in your control environment instead of, you know, a check the box. We did look at it we didn't find.
00:15:01 Speaker 3
I'd love to to double click on the continuous monitoring idea so one of the things that auditors often ask me when they start getting into the world of continuous monitoring or continuous auditing is when you have, I'll say, anomalies, something that requires follow up may or may not result in a finding.
00:15:21 Speaker 3
You know, they're disconnected from the traditional episodic audit.
00:15:26 Speaker 3
Report. So traditional internal audit. I have an audit. I do the audit, I do the report and I may not touch that area. Maybe never again or maybe not for a few years. And so that report is the state of the Union, or at least our point.
00:15:38 Speaker 3
Of.
00:15:38 Speaker 3
View when you have new information coming about a process every month or every quarter, what's your approach to?
00:15:47 Speaker 3
Following up on that, but then reporting on what you find, how do you you know that that doesn't seem like an audit report?
00:15:54 Speaker 3
But you also need to tell people that you're looking at whether it's management or the board. How do you think through that and what's your continuous monitoring follow up and reporting look like? And I think that this is a really good question because this is a challenge. All audit shops are going through and again how I did it previously, I can't today.
00:16:10 Speaker 2
Yeah.
00:16:14 Speaker 2
Because, you know, methodologies are different across ordered shops, but what I've.
00:16:18 Speaker 2
Sound so to be the best practice is essentially you have to let the data tell you, right? You cannot go in with the view that this is how we look at continuous monitoring and we document them and that's it. That is boxing yourself in. So it could lead to one of multiple routes and you have to be open to those.
00:16:39 Speaker 2
And be also cognizant of the fact you need to define some guardrails on which paths.
00:16:46 Speaker 2
Can be taken under which circumstances and what is the checks and controls and reviews in place for those to happen? So you could go down there out of. I looked at it. It confirms our view of the risk and last audit opinion is still valid. You documented what has been looked at in a way that you can leverage it if.
00:17:05 Speaker 2
You are doing in related audit or something that it needs to be referenceable so the evidencing is really important of what was looked at, how did you reach the conclusion, even if it's not an audit report, it's very important that you always every time audit gives an opinion, it is grounded in you know, rationale and documentation that backs a third party can read and get.
00:17:26 Speaker 2
To the same conclusion. The other routes are basically doing a targeted audit. You're like, OK, we don't necessarily think.
00:17:33 Speaker 2
Think we know enough about this to opine? We need to go and investigate more. So you go down.
00:17:38 Speaker 3
The target almost becomes a sort of quasi risk assessment. You identify something so now.
00:17:42 Speaker 2
We're going to go exactly and or it might be you've had some conversations, your view of the risk has changed, but you think you you understand it enough that you don't need to go and deeper dive. In that case, you change your risk assessment.
00:17:55 Speaker 2
And changed it from say a medium to a high and that would change your audit plan. It could go one of those routes.
00:18:02 Speaker 3
Yeah. So as you know more and more.
00:18:06 Speaker 3
I've been seeing internal audit teams refresh their internal audit strategy like the global internal audit standards now have prompted this rethinking of how internal audit is going to work and and I'm seeing a lot of continuous auditing on those those strategies. And I always think about the interplay.
00:18:26 Speaker 3
Between the strategy that we want to do because of the nature of the business, because of the needs of the stake.
00:18:32 Speaker 3
Builders, but that's informed by the technological capabilities, right? New technologies like Gen. AI, Open strategic opportunities that simply weren't there in the past. And so you have this hopefully virtuous cycle of strategy influencing decisions about process and technology, but technology capabilities influencing.
00:18:53 Speaker 3
Our strategy and I'd, I'd love to hear you kind of think about how you have approached your internal audit strategy and your innovation strategy at CAP one given some of the disruptions in technology lately.
00:19:05 Speaker 2
And I think I always say like innovation is where what's impactful meets what's possible, right? And in light of what's possible, you essentially go back and reimagine, sort of which are the processes. Ultimately, I go if the way you think about it is also it has to be rooted in what order it's value. So you have to look at it.
00:19:26 Speaker 2
Say, where are the areas where we are not able to provide a higher level of assurance because we are limited, but what was possible so you go and explore those right audit departments and auditors are drowning in data these days like it's the opposite problem of say 20 years ago where people like.
00:19:42 Speaker 2
Where I don't know how to find data now people like I've just too much data. I have no confidence that I'm opining my opinion factors in everything that I've been given and and and by the way, too much good data, right, good, high quality curated managed data is everywhere and we sort of have the opposite problem of.
00:20:02 Speaker 3
You would you would hear 10 years ago. Well, we've got all this data, but it's not really reliable. It's not really good. It's it takes.
00:20:09 Speaker 3
A lot of crap to work with that data and now so many investments have been made, we're reaping the benefits of that and we're a little bit overwhelmed, you know.
00:20:19 Speaker 2
Yeah. So I think ultimately how I think about it strategy wise is like what is the risk, how are we going to opine on it? What is the technology allow us to do. And then you come to how do I.
00:20:30 Speaker 2
Reframe my entire process. Maybe going back to a little bit about ordered cycles, right? So we've traditionally had like these are these are this is how you cycle through orders.
00:20:40 Speaker 2
But now you can move to sort of a more flexible ordered plan where you pick, go where the highest risks are. You place reliance on the work you've done. Unless there is a change in risk in instead of like doing the same order and coming up with the same opinion, you can now do flexible auditing and let it be driven by risks in the.
00:21:00 Speaker 2
Enterprise so.
00:21:02 Speaker 2
All of this has been made possible by sort of some of the emerging technologies, but what we are still not getting enough of, maybe, and I think I I hope in the future years will be is a lot of the unstructured data can now be analyzed in the same way that structured data could be analyzed, right. So a lot of places where auditor says I don't have good data.
00:21:23 Speaker 2
They actually meant I don't have tabular, right?
00:21:26 Speaker 2
And now they have, you know, governance memos, risk assessments, pages and pages of documents that can be converted into structured data that you.
00:21:36 Speaker 3
Can handle. Yeah, I think that's such a great point because the ability to not just analyze and synthesize these large unstructured data sets, but the ability to.
00:21:46 Speaker 3
Now on the fly, create knowledge assistance that will help you deal with.
00:21:50 Speaker 3
You know, regulations with frameworks with internal audit methodologies, you know the capabilities of of, of things like copilot and Jim and I and these other things that allow you to just reference your own documents really opens up a whole world of how we interact with these large unstructured documents and data sets. I think that's a really a great point.
00:22:10 Speaker 2
And I think like.
00:22:12 Speaker 2
How I also think about it right as you go through it like one of the things I would always say is the accuracy of the model is the clarity of the subject matter, right? It's the ability of your subject matter expert whether that's a regulatory expert or a risk expert to articulate.
00:22:33 Speaker 2
How do they parse information? How do they get to an opinion and then that allows?
00:22:39 Speaker 2
To to teach a high or, you know, build automation that follows the best in class approach to evaluating risk and opining on right.
00:22:49 Speaker 3
So I imagine a lot of subject matter experts would hear that and think no. Like, that's the end of my world, right? So talk to me a little bit about the.
00:22:59 Speaker 3
Human side of this how are we? You know, I I don't think either of us believe that AI or AI agents are here to take everyone's jobs. In my experience, the list of to DOS and internal audit is way longer than we've ever gotten to. So if we get to a few more, I think that's great.
00:23:16 Speaker 3
But as an innovator and someone who introduces technology, not just gonna, but you've been doing this for years, right? So you introduce new technology. RPA was like, oh, no, the automation is coming for our jobs. And then Gen. AI came out coming for our jobs. But how do you, how do you think through that and how do you talk to teams or other leaders about how this technology impacts or influences their staffing and resourcing?
00:23:37 Speaker 3
Strategies.
00:23:38 Speaker 2
I think for for the foreseeable future, AI or any of these technologies are going to be human in the middle for sure, right. There is no autonomous AI, at least for the foreseeable future. I do want to go back to how do how should people think about it? Right. I think you're right in that people.
00:23:58 Speaker 2
Think of it in a very sort of will this take my job away sometimes and or in a very narrow? How can this make my life easier? And I think those are both narrow questions, right?
00:24:12 Speaker 2
It is in fact an exciting time to be in order because you can do all like the most interesting work of.
00:24:20 Speaker 2
When you have all this technology available to give you the patterns, the risks, the themes and trends to investigate, and basically it allows to you to be more do more interesting work, but also do the highest value work which is work that requires.
00:24:40 Speaker 2
Expertise.
00:24:42 Speaker 2
Human judgment and navigating Gray space that you cannot simply train a model to do, or you cannot have this technology do it for you, and I think that's the way.
00:24:52 Speaker 2
To think about it.
00:24:54 Speaker 3
Yeah, I I love that. I mean, the thing that drew me to audit at the beginning of my career was that I feel like I'm a really inherently curious person. I always want to ask questions.
00:25:03 Speaker 3
And the ability to go into the business and ask questions about virtually any business process, any function that was really, really appealing to me, I never got into internal audit because I wanted to like test three-way match controls, right or look at change management controls or anything like that. So I think if you get down to the root of.
00:25:23 Speaker 3
Why do we love what we do? Why is this a job that I want to have? Those are not the kinds of things that, at least in in in my experience, that we are targeting technologies at right. We're targeting technology.
00:25:34 Speaker 3
That how do I do user access testing more effectively or how do I make sure that our financial controls are working the way they're supposed to work at scale maybe 100% population or whatever it is? It's not about eliminating curiosity, it's about feeding curiosity.
00:25:53 Speaker 2
And I think that's the thing that people are maybe missing in this is all this technology gives audit a better seat at the table and audit does have a unique seat at the table which is what I personally found most excited.
00:26:06 Speaker 2
Thing coming into audit in the second-half of my career because in audit is a unique place because you're looking across businesses across processes and the ability to connect the dots and sort of bring these insights that people like. I would not have known just being in my own.
00:26:26 Speaker 2
Silo in my own world that these problems are more universal than mine, and there are others in the enterprise or the wider industry who found solution to these, so that ability to find sort of deep rooted problems, common thematic problems that.
00:26:42 Speaker 2
And help people think through the solutions, I think that that is where internal audit can really, you know, bring a lot of value to all enterprises and it this technology transformation enables order to have a better seat at the table, have more sort of higher impact conversation then sort of narrow why do 50 people.
00:27:01 Speaker 2
Have access to this application when they shouldn't, which is really sort of not the kind of thing you want to.
00:27:07 Speaker 2
Talk.
00:27:07 Speaker 3
To execs about, right? Yeah, exactly. Well, I think I think it comes back to that. That fear is is rooted in not having a culture of innovation where you expect things to change and you expect to disrupt yourself.
00:27:21 Speaker 3
And as an innovation leader and you've got a pretty big team and and an extended team of resource providers and vendors and others building that culture of innovation is so core to innovating to keeping people engaged, to dealing with some of those concerns about is it coming from my job. Let's talk a little bit about things that.
00:27:41 Speaker 3
Leaders can do to drive the spirit of innovation in their team so that they really embrace this new technology rather than being concerned about it or confused by it.
00:27:51 Speaker 2
So I think as leaders there is, you need to think of innovation in multiple ways, right? A lot of leaders just think of innovation in sort of the technology like just tell me what is the technology, show me what it can do and that's how they sort of think of innovation, but that leaves a lot at on the table.
00:28:11 Speaker 2
How I think about it is like you're thinking about like what has like cutting edge innovation. Essentially for me is what's only just become possible.
00:28:20 Speaker 2
Meets impactful, right? So you need to look at, OK, what could I not have done? You know, there's that kind of top down transformative innovation. But what you also need to embed into your day-to-day is identifying innovation, where you while you're doing it, you're like we struggle with coming up with an opinion on this. We struggle to risk us as this.
00:28:40 Speaker 2
And you, you're collecting all these ideas. So that's what we I've done at every place I've been is introduced sort of a process.
00:28:48 Speaker 2
Where people doing the job while they're doing the job, they can submit in 2 minutes an idea and you look across these the power of that has been when you look across these ideas also you see a pattern and that tells you a lot about where your process is, where your value is weaker, where your personnel are struggling to.
00:29:09 Speaker 2
Perform execute what you state as a policy or as a goal and connecting that like this is where we want to go. This is where our people are struggling, then allows you to find technology that solves for those problems that truly transform the day to.
00:29:24 Speaker 3
Yeah, I I think that's that's so great. And I think you know you're in a really fortunate position of working for a company where innovation is is core to your brand. And so it's it's sort of in the in the soup at cap one, right and and I think regardless of whether that's the world you're living in.
00:29:44 Speaker 3
Or if you're in an organization that's maybe not quite as forward leaning when it comes to innovation in general and tech specifically, the advice I always give to internal audit teams is you should keep pace with the business.
00:29:57 Speaker 3
It's pretty rare in my experience that internal auditor is the first adopter of any new technology. It happens sometimes, but I think I think it's the exception rather than the rule. And what I think is far more common is that internal a lot, it lags the business by several years in terms of their adoption of technology. Somebody in operations will get some great new tool.
00:30:17 Speaker 3
And then internal audit.
00:30:18 Speaker 3
That gets around to it later. I wonder, do you agree with that idea of of keeping pace and and how should I think about or how should audit leaders think about really capitalizing on the investments that have been made at an enterprise level without putting technology first, but understanding that you've got all these tools on the work bench?
00:30:40 Speaker 3
You've got to decide which ones you want to pick up and when you want to pick them up.
00:30:43 Speaker 2
So I have a sort of a nuanced view on one of those things, but I will get to sort of your first point of keeping pace. I think as more and more controls get automated more and more technology gets introduced in this sort of managing risk and how operations work, you do need to keep pace because you cannot challenge a model with a sample testing or.
00:31:03 Speaker 2
If there is a model out there, you need to be able to build a challenger model. That challenge is sort of the efficacy of the models used. Similar to that, I think there is a need for order to be able to.
00:31:15 Speaker 2
Process the same level of information right and make their own parallel, independent opinion of those decisions in the same way that first and second line functions are doing it. But when I sort of have a now more nuanced view, having been in this industry for a long time, I didn't have the same view that coming into order I was like is ordered really the.
00:31:35 Speaker 2
Place that will be for me because I want to be at the frontier of innovation. What I've come to realize over sort of my career is audit can be actually the beacon for innovation for the entire bank because one audit has several advantages in that, right.
00:31:51 Speaker 2
One while innovation comes in OPS and revenue generating functions, I have in my careers in innovation and risk management functions is lagging and that's where I do think. In fact, audit might be ahead in some cases of risk management functions which then helps audit, inform and educate business on how they can better.
00:32:11 Speaker 2
Manage access.
00:32:13 Speaker 2
And, you know, mitigate their risk. The second thing is audit has a unique point of view in that you are able to look across the enterprise.
00:32:21 Speaker 2
So going to the board, you can say these different functions. They're struggling with the same root cause. So we are. These are the things we need to prioritize as an enterprise. So again, you do need to keep pace, but you should also, I think audit department should think of like where is it that audit can provide that unique value to the enterprise be a beacon for innovation.
00:32:42 Speaker 2
And risk management because a lot of businesses are essentially risk management is a core part of the business.
00:32:48 Speaker 3
I mean certainly in financial services, it's absolutely front and center. Yeah, I love internal audit as a beacon for innovation. We should get some bumper stickers printed up the IRA, that's fantastic. You know, I think.
00:33:01 Speaker 3
In early 2025, we can't have a conversation without looking into agents. It is the topic right now. Everyone is talking about agents. Maybe just to start off like give me your definition of agents and then let's talk a little bit about some of the places that you're either using or planning to use agents in.
00:33:21 Speaker 3
The internal audit world.
00:33:23 Speaker 2
So I think a lot.
00:33:25 Speaker 2
Of talk about Agentic AI I think autonomous AI. I do personally think some of it is overblown. Artificial general intelligence, some of it is overblown in that.
00:33:38 Speaker 2
Like there there will be a niche of the ChatGPT is the anthropics of.
00:33:44 Speaker 2
The world who have the resources, energy, billions of dollars to burn on creating artificial general intelligence where I focus as a sort of leader in innovation. Since I don't work for, Sam Holtman is sort of application right? Where can we apply these technologies and get real business value?
00:34:04 Speaker 2
I think again coming to my sort of view on this and where I see this going, I think for the foreseeable future it will be human in the middle. There will be no, you know, self driving audit shop.
00:34:18 Speaker 2
So to speak, and how I think about it is like how leaders should think about it is, how does this transform auditing? What are the things that are core principle do you want? Then if you're thinking about building and investing in this kind of a thing, you want your think of it as a as you develop a expert.
00:34:38 Speaker 2
In your audit department.
00:34:40 Speaker 2
Your expert would know the core principles of auditing.
00:34:44 Speaker 2
Right know how to take a risk based approach, fundamental sort of principles that never change. So those are things that you're training your model, right? You're investing money, teach them these things and these things are fundamental. They will never change. Then there are aspects which are like general, I don't need to teach this person.
00:35:04 Speaker 2
Specifically, this is just how things are done. Communication, you know, basic general ideas. So those are things that you can take out-of-the-box from a lot of the, you know, open source models. Then there are things where you say, OK, I need.
00:35:20 Speaker 2
Areas where I want to teach somebody how to look at this specific process in our enterprise and opine in this according to these specific methodology that we've developed. So in those cases you get sort of enter the space of rag and you know having sort of using.
00:35:41 Speaker 2
All like external developed general model build, building small specialized expertise and use your own data instead of just using general out-of-the-box information. So again I give a very I try to keep it non-technical but essentially there is difference between how much resources you need to spend.
00:36:00 Speaker 2
You don't want to be training your models on things that constantly change, and you're not really.
00:36:06 Speaker 2
Really looking for the model to know and remember all the time and there are things that are basically I needed just to do this audit step and then you can use them as in the context window and you basically start doing them out-of-the-box instead of investing trying to over engineer it.
00:36:26 Speaker 3
I love that. So one last question before we wrap it up here.
00:36:30 Speaker 3
You didn't start off as an internal auditor, and now you've been an internal audit for several years, so.
00:36:36 Speaker 3
What is the most controversial point of view you have about internal audit coming in from an outside perspective?
00:36:46 Speaker 2
So my most controversial view has been, and I think people do find it interesting also, is that audit, if you think about audit as a profession, it's sort of it's a fundamental pillar of capitalism. So audit has been done for like over hundreds of years.
00:37:05 Speaker 2
More or less in the same way there's been more data. There's been more technology, but it's more or less in the same way. My view is we are at a point in, in the order, sort of where we are in order.
00:37:16 Speaker 2
That.
00:37:18 Speaker 2
How audits are done, what the auditors job is, is at, we are at the precipice of it can be fundamentally transformed, and to do that, we need to reimagine how orders have always been done, and I think that is sort of something that leaders need to.
00:37:36 Speaker 2
Start thinking about more deeply, right? It's we've done audit cycles, we've done cycle audits, we've done, you know, risk assessment memos. We've done continuous monitoring dashboards, but.
00:37:50 Speaker 2
All of that doesn't exist in a world where you're able to now process information real time.
00:37:56 Speaker 2
Provide connect. Connect the dots across the enterprise. Have more you know bigger, deeper conversations instead of narrow specific views that maybe are lower value. So.
00:38:12 Speaker 3
This has been a great conversation. Imran Mullah, thank you so much.
00:38:15 Speaker 2
For being here today. Thank you, Charles. Pleasure.
00:38:19 Speaker 1
Small internal audit teams boost your efficiency at the 2025 audit sphere. This one day virtual conference on June 17th offers expert strategies and practical tools just for you. Register now at the dia.org.
00:38:35 Speaker 1
If you like this podcast, please subscribe and rate US. You can subscribe wherever you get your podcasts. You can also catch other episodes on YouTube or at the iaa.org that's theia.org.
