The unemployment rate for Black men ticked down in October while it rose for most other groups, but that may be because workers are dropping out of the labor force.

The October nonfarm payrolls print showed that the U.S. economy added 261,000 jobs in the month and that the unemployment rate for all workers increased to 3.7% from 3.5%.

For Black men, unemployment fell to 5.3% from 5.8% a month earlier on a seasonally adjusted basis. White unemployment rose to 3.2% overall up from 3.1% a month earlier.  

“It went in the right direction for the wrong reasons,” said Bill Spriggs, an economics professor at Howard University and chief economist for the AFL-CIO.

The downward motion in unemployment for Black men is likely due to the labor force participation rate, which dipped slightly to 67.2% in October, just below the previous month’s reading of 68%.

In addition, the employment-to-population ratio for Black men fell to 63.6% from 64.1% in September, which could indicate that workers have stopped looking for jobs, sending unemployment lower.

Unemployment for Hispanic workers also jumped in October, outpacing the uptick for Black and white workers. It jumped to 4.2% from 3.8% in September.

“It’s showing this continued frustration that workers of color are having in the labor market,” said Spriggs. Though overall there is strength in the labor market, “this is not the tight labor market where people can just walk in and get a job no matter who they are.”

Overall Black unemployment ticked up led by Black women. In October, the unemployment rate for Black women jumped to 5.8% from 5.4% in September.

“This is concerning because throughout both the pandemic and the economic recovery from the pandemic crisis, Black women have been lagging behind,” said Kate Bahn, director of economic policy and chief economist at the Washington Center for Equitable Growth, a non-profit

On the brighter side, the employment to population ratio for Black women didn’t change, though labor market participation ticked up during the month. That could be a sign that more Black women are returning to the labor force and are looking for jobs but haven’t yet found employment, noted Valerie Wilson, director of the program on race, ethnicity and the economy at the Economic Policy Institute.

“It doesn’t suggest that there’s a huge number of people losing jobs,” she said.

Of course, one month of data does not make a trend, so it’s important to look at the longer-term picture for workers of color.

Generally, the unemployment rate for workers of color has stepped down in recent months in-line with white counterparts, and labor force participation and the employment to population ratio have mostly held steady, said Wilson.

Still, there may be cause for concern going forward depending on how the Federal Reserve reads the October report. The labor market has remained strong amid historic interest rate hikes meant to tame high inflation, and the central bank is poised to continue its path of raising rates.

If the Fed goes too far and pushed the U.S. economy into a recession, that could have the worst impact on workers of color.

“If we throw the economy into a recession, that impact at least historically is more likely to hit harder in communities of color,” said Wilson.

— CNBC’s Gabriel Cortes contributed reporting.