Does a dragon's Siege affinity matter in combat?

No. A dragon's Siege affinity has no impact in dragon combat, because you never put siege troops on a combat dragon. Siege exists to break a tile's Durability, not to fight armies.

So if you see a dragon marked "green for Siege" or "red for Siege" on its page: ignore it for your combat decisions. In this context it's cosmetic.

What actually counts is the dragon's affinity for the combat type you play (Spearmen, Archers, Cavalry or Shieldbearers). Focus on that when you fill your armies, and don't let the Siege green/red dots distract you.