This woody-stemmed plant grows in stands topping 6 feet tall. It recently bloomed with tiny white clusters near the tips. It grows in full shade along my creek. I believe this is the culprit in my severe contact allergy, but no one else in my family reacts to it. The close-up picture uses a quarter for scale.
this looks most like a privet, possibly Ligustrum ovalifolium, but the leaves seem even more rounded than usual. Look at some pictures on the internet of the flowers of this shrub, which are like your description. I’m not sure what else it could be at this time other than this or Forestiera ligustrina – the flowers are very different. The opposite leaves suggest one or the other.
Steven R. Hill, Botanist, SCNPS