In the aftermath of Republican defeat in a New York special Congressional election, Republican House Budget Chairman Paul Ryan is now bearing the brunt of voter anger, but it won't deter him from fighting for his Medicare plan --- even if it ruins his political career, according to a report.

I don't care about that, Ryan told ABC News in an interview on the Subway Series with Jonathan Karl. Now is not that time to be worried about political careers. Sincerely, I will be fine if I lose my House seat because you know what? I will know I did what I thought was right to save this country from fiscal ruin.

Ryan's plan would phase out Medicare as a fee-for-service system and replace it with a private insurance program backed by government subsidies. People 55 and older would not be affected by the plan. And the changes would not go into effect until the year 2021.

The plan was a major source of contention in Tuesday's special election, which saw Democrat Kathy Hochu beating out Republican Jane Corwin in a solidly Republican 26th Congressional District. Corwin supported Ryan's plan, while Hochu attacked the plan as a concerted effort to destroy Medicare as American's know it.

But according to the ABC News report, Ryan doesn't feel personally responsible for the loss.

The president and his party, they basically decided to medi-scare. They decided to shamelessly demagogue and distort what we're proposing to try and scare seniors to get votes, Ryan said. It did work to scare seniors. Now I believe that in a year and a half time that we have, that facts are going to get out and people are going to understand this problem.

In April, House Republicans voted overwhelmingly in support of Ryan's plan, where all but four Republicans voted for it.

Nevertheless, in recommending and supporting a Medicare overhaul, Republican Representative Mike Simpson said, Republicans knew they were heading for some hard times.

We knew that the first person that threw something on the table was going to get the living crap beat out of them and we might be beat up in the elections in the future, Simpson said, Bloomberg reported. But you got to be willing to take that if you want to reform the system.