Probably the first place to look for AI-​generated hacks is in financial systems, since those rules are designed to be algorithmically tractable. High-​frequency trading algorithms are a primitive example of this, and will become much more sophisticated in the future. We can imagine equipping an AI with all the world’s financial information in real time, plus all of the world’s laws and regulations, plus newsfeeds and anything else we think might be relevant, then assigning it the goal of “maximum legal profit” or maybe “maximum profit we can get away with.” My guess is that this isn’t very far off, and that the result will be all sorts of novel and completely unexpected hacks. And there will probably be some hacks that are simply beyond human comprehension, which means we’ll never realize they’re happening.

In the short term, we’re more likely to see collaborative AI–​human hacks. An AI could identify an exploitable vulnerability that would potentially be a hack, and then an experienced accountant or tax attorney would use their experience and judgment to figure out if that vulnerability could be profitably exploited.

For almost all of history, hacking has exclusively been a human activity. Searching for new hacks requires expertise, time, creativity, and luck. When AIs start hacking, that will change. AIs won’t be constrained in the same ways or have the same limits as people. They won’t need to sleep. They’ll think like aliens. And they’ll hack systems in ways we can’t anticipate.

Computers have accelerated hacking across four dimensions: speed, scale, scope, and sophistication. AI will exacerbate these trends even more.

First, speed: The human process of hacking, which sometimes takes months or years, could become compressed to days, hours, or even seconds. What might happen when you feed an AI the entire US tax code and command it to figure out all of the ways one can minimize one’s tax liability? Or, in the case of a multinational corporation, analyze and optimize the entire planet’s tax codes? Could an AI figure out, without being prompted, that it’s smart to incorporate in Delaware and register a ship in Panama? How many vulnerabilities—​loopholes—​will it find that we don’t already know about? Dozens? Hundreds? Thousands? We have no idea, but we’ll probably find out within the next decade.

Next, scale: Once AI systems begin to discover hacks, they’ll be capable of exploiting them at a scale for which we’re simply not prepared. So when AIs begin to crunch financial systems, they will come to dominate that space. Already our credit markets, tax codes, and laws in general are biased toward the wealthy. AI will accelerate that inequity. The first AIs to hack finance in pursuit of profit won’t be developed by equality-​minded researchers; they’ll be developed by global banks and hedge funds and management consultants.

Now, scope: We have societal systems that deal with hacks, but those were developed when hackers were humans, and hacks unfolded at a human pace. We have no system of governance that could quickly and efficiently adjudicate an onslaught of hundreds—​let alone thousands—​of newly discovered tax loopholes. We simply can’t patch the tax code that quickly. We haven’t been able to prevent humans’ use of Facebook to hack democracy; it’s a challenge to imagine what could happen when an AI does it. If AIs begin to figure out unanticipated but legal hacks of financial systems, then take the world’s economy for a wild ride, recovery will be long and painful.