Early uneven ripening

[duajones]

I have two Celebrity plants growing that I bought from the local nursery. One has had leaf curl since I planted it and just hasnt grown out of it but was setting fruit nonetheless. The other is twice its size and the leaves look normal and healthy. Recently they both have struggled to set fruit and the fruit on them is ripening very early and uneven as well. Any ideas? I have had lots of whiteflies recently but havent noticed them on the plants that bad lately.