My roommates and I have been renting this property for the past three years. We were occasionally late with the rent, but we eventually paid. The landlord mentioned something about late fees after we signed the lease, but never enforced payment. Now that we’re moving out, he’s attempting to collect late fees for the past three years, which total over $5,000, well above our security deposit that he won't return either. Do we have any legal recourse?