The Gemara says that when a fig falls it gets disgusting, and therefore the owner will give up hope on it. However, the indication is that this would not be true in regards to olives and carob. What’s the difference between them?
So, living in Eretz Yisrael, I am fortunate, בע”ה to have an olive tree in my backyard, and carob trees running along my street, so experience shows that olives and carob are both the types of fruit that are quite hard when they fall off the tree. That being the case, it is unlikely that they will get ruined when they fall. Figs, on the other hand, will fall off the tree when they are soft and ripe, and thus become easily ruined.