Ms. Khan, the chair of the Federal Trade Commission, has staked an ambitious trustbusting agenda on a case that may be difficult to win.