Tom Brady
New England Patriots quarterback Tom Brady leaves the field after defeating the Kansas City Chiefs in the AFC Divisional round playoff game at Gillette Stadium in Foxborough, Massachusetts, Jan 16, 2016. David Butler II-USA Today Sports

A U.S. appeals court on Thursday signaled it could restore the four game "Deflategate" suspension of New England Patriots star quarterback Tom Brady over allegations footballs were under inflated before a NFL playoff game last year.

A lawyer for the players' union faced tough questioning by a three-judge panel in New York hearing an appeal by the National Football League of a trial judge's reversal last September of the suspension imposed by league commissioner Roger Goodell.

Jeffrey Kessler, the union's lawyer, argued in the 2nd U.S. Circuit Court of Appeals that Brady had no notice that an equipment violation could expose him to a suspension, and that Goodell did not have "blanket authority" under the league's collective bargaining agreement to impose it.

But U.S. Circuit Judge Robert Katzmann said as commissioner, Goodell frequently faces "novel" situations like the deflated footballs, and questioned why in those instances he could impose punishment for conduct detrimental to the league.

"Isn't that why the language in the agreement gives the commissioner broad authority for dealing with conduct detrimental?" he asked.

Brady, 38, had been suspended in May 2015, four months after under-inflated footballs were used in the Patriots' 45-7 victory over Indianapolis in January 2015's AFC championship game.

That win took the Patriots to the Super Bowl, where they defeated the defending champion Seattle Seahawks, giving Brady his fourth championship title.

The NFL suspended Brady after Ted Wells, a lawyer hired by the league to investigate the incident, said Brady was "generally aware" that two Patriots employees had conspired to deflate the balls, which could make them easier to grip.

Goodell upheld the suspension on July 28, prompting the legal challenge on Brady's behalf. Brady has denied knowing about any plan to deflate footballs.

U.S. District Judge Richard Berman in Manhattan overturned Goodell's decision last Sept. 3. He said Brady "had no notice that his discipline would be the equivalent of the discipline imposed upon a player who used performance enhancing drugs."

Berman's decision allowed Brady to play the full 2015 NFL season. The Patriots made the playoffs but did not reach the Super Bowl.

It is unclear how quickly the appeals panel could rule. Paul Clement, the NFL's lawyer, urged the court to rule quickly to bring finality to the dispute before the next football season begins in September.

"It would be an awful shame if we have this issue hanging over the league for another season," Clement said.