Yes, food is litter; it decomposes, but not immediately. If it lays there in hot weather, it can become spoiled and make an animal sick. Yes, they will eat it anyway, if they're hungry; I've seen them do it. How do you know they don't die from your food? They may fly off or go into the woods, die, and you never know the difference.
As usual, you seem more concerned about justifying your position than doing what's right. It's only a few more steps to a trash can, or at least compost your wasted food. Then at least it would not only have a useful purpose, but it wouldn't harm any living thing.
More information about formatting options