I would go with "We apologize for the harm [that] we caused" which seems the most natural choice while explicitly acknowledging that some harm was caused. "All the harm" does indeed come across as a bit inflated, as other commenters have mentioned.