If Kaoru was the more imaginative sort, he might have mentioned that there were fairy stories in which eating food whilst one had been whisked away to parts unknown was kind of sort of a bad idea. But he wasn't, so he was mostly just worried about what would happen if they didn't get more of it the next day.

"We should secure our own sources of food. Relying on handouts is dangerous. If there's a cold snap and we're not prepared, we'll be in trouble. We don't know what's in the upper regions of this tree, either. Even healthy trees can be home to parasites and structural weaknesses. ...We might have to worry about birds looking for a meal."

After all, they could be considered insects infesting this particular tree. Why shouldn't a passing bird try to gobble them up?