I'm guessing they singled you out because you appeared to be without a GF or bride and they saw no kids with you. that's usually what triggers that behavior.
I too do LOTS of photography and nearly all of it has people. I hate still-life unless its pretty darn dramatic on its own. I've employed child models and adults. I get written permission and agreements prior to any shooting...but what you describe here is public-display of flowers, back-drop and people. A child's face among/within floral material is a sure-shot winner photo.
I'm guessing you were not in the USA, as kids don't normally walk around in that mode any longer...(I did as a wee-one at the ocean, but its certainly frowned upon in this era). But the parent was WAY WAY out of order with her behaviour. I may have been tempted to let her see the cameral real-real up-close....and hard.
The responsibility stands with the parents. PERIOD - No exception! The protective measures they take cannot be assaulting you and your feelings. They were SO wrong.
As a rule of thumb, I personally would never do any deliberate shooting of kids at the beach, zoo, ...anywhere without a verbal agreement from the adult at least. If there are kids present at a shoot and no parents, I don't/won't shoot them. It happens at the skateboarding and surfing events my kids are involved in. It makes things rather difficult to get a skateboard jump if there are other kids in-frame...and photoshop does not always do what you wish.
Many years ago, i carried business cards with all of my information (Identification) and contact methods (like email and phone) and would hand it to adults with or without kids. I would even feel like I had to do that with home-owners if i shot their house from the street. If hot cars are at an auto-show, or flowers at a floral show...its shutters-free! Same with any public competition unless there is a sign or announcement of no-photography.
I always offer the people involved a print, email PDF, or whatever. I just knock or approach and say "hey....I never like to shoot someone's house, flowers, garden or person without permission." No one ever said "no-shoot" with that method of introduction. They become comfortable and you nearly always learn great things about them. Sometimes you can get some really cool planned and deliberate additions to a shoot, like an old-lady on the porch of her 100-yo home.
We all had to SURVIVE by different and unique means. Only God truly knows what we went through.