Coinstar Inc. (NASDAQ:CSTR), parent of DVD-rental firm Redbox, is expected to report slightly lower fourth-quarter earnings as the company has had to grapple with a new federal law requiring higher fees for small debit-card transactions.

The Bellevue, Wash.-based company is expected to report net profit of $20.93 million, or 65 cents a share, down 7.1 percent from $22.42 million,or 68 cents a share, in the year earlier period, according to analyst estimates provided by Reuters.  Analysts expect revenue of $496.41 million, up 25.1 percent from $396.76 million in the year-earlier period.

Coinstar reports Monday after the markets close.

Unlike most retailers, Redbox, with 28,000 U.S. kiosks accounting for about 84 percent of Coinstar's revenue, was hurt by a provision of the Dodd-Frank Wall Street Reform and Consumer Protection Act, which became law in 2010. That provision raised charges on small debit card purchases, such as DVD movie rentals. The effect on Redbox was that it had to pay 27 cents on each rental made by debit card, up from nine cents before Dodd-Frank, according to D.A Davidson analyst John Kraft.   

While Dodd-Frank became law in July 2010, the provision in question did not come into effect until Oct. 1, 2011. In response, Redbox raised its rental fee to $1.20 from $1 -- but waited to do so until Oct. 31 for rentals of DVDs and Jan. 31, 2012, for online rentals. That left a big part of the fourth quarter in which movie rental margins were severely compressed.

Quarter Four will not be a good reflection of what their (Coinstar's) margins will be because of that, Kraft said in an interview.

He believes there was probably some attrition of customers due to the price hike, but not a large amount.  

Fortunately for Redbox, the move to raise prices happened while people were still upset with rival Netflix over its price hikes and attempt to separate its DVD-by-mail service from its streaming service. Wedbush Securities analyst Michael Pachter in a research note estimated that Netflix lost nine million DVD subscribers in the second half of 2011, which gave Redbox the opportunity to snag new customers.

In addition to Redbox, Coinstar provides kiosks where customers can redeem coins for a cash voucher. The service accounted for 16 percent of Coinstar's revenue in the third quarter of 2011, down from 20 percent in 2010. Pachter believes the service will account for an even smaller share of revenue in upcoming quarters.

Competitors such as Netflix are increasingly turning their focus towards online streaming services as customers shy away from DVD sales, Pacific Crest analyst Andy Hargreaves said.

But although there have been reports that Redbox will soon enter the streaming business, it hasn't done so yet. Hargreaves said Redbox kiosks should get pretty good business for the next several years, but eventually the company will need to rely on other Kiosk businesses in order to grow.

Shares of Coinstar fell 41 cents to $49.96.

(ALSO READ: Redbox Lacks Streaming Presence as Netflix, Amazon Saturate the Market)