Wondering what to do with your millions when you die? If you’re like most other millionaires out there, you’ll probably want to pass that money down to your children, but judiciously, of course.

A new survey of high-net-worth individuals -- those who have a minimum of $5 million in assets -- by Merrill Lynch’s Private Banking and Investment Group details key habits and considerations when it comes to passing money to their children. The group surveyed 206 of these individuals, all of whom had at least one child and intended to pass some of that wealth to their children.

Of the respondents, 91 percent said family was a priority when it came to handing down money. Only 4 percent said that philanthropic groups came to mind first. Commitment to family was the biggest influence on these decisions, more than personal values, estate taxes and other considerations.

“Our intention is to help wealth creators and wealth holders to put more thought and purpose toward their giving goals,” the report declared. It acknowledged that the media often painted wealthy heirs and heiresses in varying shades of narcissism, wastefulness or downright ridiculousness. It also sought to dispel any wariness about inheritance that might stem from such negative attention. “Our research on giving and sustaining wealth is aimed at moving beyond superficial assumptions,” it noted.

Wherever the money ultimately ends up, preserving as much of the original amount was a priority for most respondents in the survey. “I will do as much as possible within the rules to minimize the taxes,” 86 percent said, while only 13 percent answered that they believed it was “reasonable” to contribute their “fair share” in taxes.

“With any giving, there’s both a desire to benefit the next generation and a need to be practical,” the report noted. Translation: Minimize estate taxes, which apply after an individual dies. According to the report, 58 percent of those with at least $10 million in assets were planning on distributing wealth to their children before rather than after they died, a move that can offer potential tax benefits.

But could giving too much money leave children coddled or perhaps foster laziness? Some wealthy individuals seemed to fear so. Although their purpose in giving was mainly to allow their heirs and heiresses to pursue their dreams, some expressed concern that giving too much could prevent recipients from reaching their full potential.

For wealthy individuals, the challenge then becomes finding that sweet spot. On average, families with $100 million said giving $63 million was “too much” for one child, although $26 million was considered “too little.” However, the report also added, “When it comes to how much is too much, it appears that there may be no one-size-fits-all answer.”

The final decision may end up being a unilateral one, as roughly one-third of survey respondents said they didn’t discuss the issue of inheritance with their families due to fears “the conversation will disrupt family harmony.”

Awkward or not, more families may have to swallow their squeamishness and initiate those difficult conversations, if they don’t want their hard-earned millions to potentially be squandered (or, horrors, lost to taxes). From 2007 to 2061, about $59 trillion will be handed down in inheritances, according to a report by the Center on Wealth and Philanthropy, Bloomberg reported.