The Reform Alliance, which was inspired by Meek Mill's recent stint in prison for a minor probation violation, hopes to free one million prisoners in five years.
The owners of the New England Patriots and Philadelphia 76ers, Robert Kraft and Michael Rubin, are co-founders.
Reform says it wants to help people who are "trapped in the system".
The group's "mission" is to "dramatically reduce the number of people who are unjustly under the control of the criminal justice system, starting with probation and parole".
"To win, we will leverage our considerable resources to change laws, policies, hearts and minds," it says.
More than six million people can currently count themselves as part of the "correctional population" of the USA - which includes people in prisons and local jails, but is mostly made up of the more than four million people on probation or parole, according to Bureau of Justice statistics.
Probation is often given as a sentence instead of time in prison and can include conditions like being on a curfew or going to rehab.
Parole is when an inmate is released early from prison with similar conditions to probation.
Meek Mill has experienced all three: probation, parole and jail.
The Reform Alliance says his case is an example of the "devastating and long-lasting effects" that can occur after one interaction with the criminal justice system.
The rapper was arrested in 2007 - he says wrongfully - for drug and gun charges, aged 19.
He was sentenced in 2009 to between 11 and 23 months in county prison, but was released on parole after five months and put on house arrest.
It was during this time he started to make his name nationally as a rapper, signing to Rick Ross's label and releasing a string of hugely successful mixtapes.
Before long he was a platinum-selling artist.
But a parole violation for suspected cannabis use resulted in a ban on touring, and then after failing to get his travel plans approved by the court Meek was sentenced to prison again in 2014.
Examples of parole violations that can land people back in prison range from being late to appointments with parole officers or missing a curfew, to things more specific to the crime that was committed - like failing to attend an Alcoholics Anonymous meeting.
"When you talk about these so-called technical violations, it's not technical to the kid who can never see her mum again because she showed up late for a meeting. That's not technical, that's devastating for that individual child," Reform Alliance CEO Van Jones said.
Violations over the next few years resulted in his probation period being extended - it now lasts up until 2023 - as well as the five months in prison which ended in April 2018 and birthed the #FreeMeek movement.
It's people with a similar story to Meek's, that have been "caught up on probation and parole", that Reform says it wants to focus Report
'If someone commits a crime they should go to jail'
"Being from the environment I'm from, I don't even think it's possible for you to be an angel," Meek said as the organisation was announced in New York.
"You grow up around murder on a daily basis, you grow up in drug-infested neighbourhoods.
"And every time I started to further my life with the music industry, there was always something that brought me .
Across the US, roughly a third of people on parole are black, according to Bureau of Justice statistics - something Jay-Z raised at the event.
Jay-Z has been vocal about Meek's case, writing in the New York Times while he was imprisoned.