The IRS has begun doling out refunds to those people that paid taxes on their 2020 unemployment benefits, which now under the American Rescue Plan excludes that compensation as taxable income. But for some taxpayers, those refunds will be smaller than anticipated--if they receive a refund at all.

According to the IRS, over 10 million taxpayers who filed tax returns prior to when the 2021 American Rescue Plan was enacted by President Biden on March 11, were identified as paying taxable income on 2020 unemployment compensation.

The American Rescue Plan allowed unemployed Americans to exclude $10,200 in 2020 unemployment compensation from their income when calculating their amount of taxable income. The $10,200 included taxpayers that filed single or married filing jointly with modified adjusted income levels of less than $150,000.

The IRS said it is reviewing these tax returns to determine the correct table amount of unemployment compensation and tax, which could result in a refund, reduced balance due or no change in the balance due or owed.

But these refunds are still subject to normal offset rules that include past-due federal tax, state income tax, state unemployment compensation debts, child support, spousal support, or certain federal nontax debts, such as student loans.

A notice will be sent by the IRS if a refund is used to pay an unpaid debt through a refund offset.

However, 100% of one’s tax refund can be seized to pay off a debt owed to a federal or state agency under the federal Treasury Offset Program, Erica York, an economist at the Tax Foundation, told CNBC.

York also explained that private creditors could access tax refunds that have been deposited into a bank account, depending on state laws, which could apply to private debts such as medical bills or credit card debt, if there is a court order.

It is unclear how federal student-loan payments will be handled with the pause in place through September.

The IRS will issue refunds via direct deposit for taxpayers who provided their bank account information on their 2020 tax return. If valid bank account information is not provided, a paper check will be mailed to the address on record.

But banks are required by law to follow court-order garnishments and state garnishment laws unless they are directed otherwise by Congress, a banking industry source told CNBC.

The IRS said the corrections being made under the American Rescue Plan are being performed in a “phased approached” to ease the burden on taxpayers.

The first phase, which is currently underway, will include the simplest tax returns, with adjustments being made for single taxpayers who did not claim children or have any refundable tax credits.

The second phase will undergo more complex returns, which the IRS said it expects will take through the end of summer to review and correct. This will include couples filing as married filing jointly.

Taxpayers can expect to receive a notice explaining any adjustments 30 days from when a correction is made by the IRS. This notice should be kept with their tax records.

Tax refunds related to unemployment benefits are coming this week. Getty