Ian Schrager and Aby Rosen may have been willing to support Calvin Klein's plan to tear down his Southampton castle and build a more modern home on the site, but it seems their enthusiasm isn't shared by all of Calvin's neighbors on Meadow Lane. Bill Williams Jr., who recently retired as a partner at the white-shoe law firm of Sullivan & Cromwell, filed a letter of complaint with Southampton's Architectural Review Board in advance of a public hearing earlier this week. Williams' objection to Klein's proposal, however, doesn't have anything to do with any interest in preserving the gaudy castle. He just doesn't care for Calvin's taste.