It was wrong of you to borrow my book without asking me first. (have)

-> You _{……………………………………………………………………….} before you borrowed my book

You ought to have asked (me first) before you borrowed my book

         ought to have done sth đáng lẽ ra ai đó đã nên làm gì