The Financial Industry Regulatory Authority Inc. slapped two units of Raymond James Financial Inc. with a record $17 million in fines for widespread compliance failures in the brokerage firm’s anti-money laundering programs.

Raymond James & Associates and Raymond James Financial Services failed to set up systems to properly prevent, detect and investigate suspicious activity for several years as the units saw “significant growth” from 2006 to 2014, according to a statement Wednesday from Finra.

Linda Busby, the former anti-money laundering compliance officer for Raymond James & Associates, was fined $25,000 and suspended for three months.

“This case demonstrates that when there are broad-based failures within specific areas of responsibility, we will seek individual liability where appropriate,” Brad Bennett, Finra’s chief of enforcement, said in the statement.

The broker-dealer’s $17 million fine is Finra’s largest ever related to anti-money laundering, according to the self-regulatory organization’s spokeswoman Michelle Ong. Finra was particularly concerned about the failures as Raymond James Financial Services was sanctioned in 2012 for inadequate procedures and agreed to review its program to meet compliance requirements as part of a settlement back then, according to its statement on Wednesday.

Raymond James was relying on “a patchwork of written procedures and systems across different departments to detect suspicious activity,” according to Finra. “The end result was that certain ‘red flags’ of potentially suspicious activity went undetected or inadequately investigated,” the regulator said.

Raymond James and Ms. Busby neither admitted nor denied the charges.

The settlement “relates to system and procedure deficiencies and the thoroughness” of Raymond James’ U.S. anti-money laundering program from 2011 to 2014, Steve Hollister, the firm’s director of public communications said in an emailed statement.

The program “has undergone significant resource, process and technology enhancements aligned with the firm’s measured and thoughtful growth strategy,” he said.

“We have also begun the process of exiting our U.S. third-party foreign correspondent business, excluding operations in Europe and Canada,” Mr. Hollister said. “We will continue to refine our program to address evolving regulatory expectations and ensure we are doing our part to reduce criminality in the financial system.”