Originally, the plan was for battles to be completely non-interactive. Under that premise, I came up with some really strange ideas for taming the complexity of formations - stuff like programming the AI. Crazy stuff, really. But being purely interactive won't work either, since you'd be playing a lengthy tactics mini-game every time you attack another unit. In games with even as few as three units per side, that could take HOURS to play through.
My solution, so obvious I'm ashamed, is to make it interactive at the formation level, rather than the unit one. Using just a few actions, you can create simple, quick combat that still has all that juicy depth and unit interaction I was looking for.
Combat consists of 10 turns where each army can make one of six actions:
Advance - Advance is your basic action. It moves the entire formation one cell ahead, attacking any units directly in front of it at the end of the move. Ranged units will fire up to four spaces directly in front of them after moving.
Fall Back - Like Advancing in the other direction, you fall back one cell, not attacking at all. Any units which fall back off the end of the field are considered in retreat and are no longer part of the current combat. A defending unit can retreat from battle completely by retreating his commander off the field. A defending army can not Fall Back at all for the first three turns of combat.
Attack - Units stay where they are, but attack any adjacent unit. Unlike advance, units can attack units to their side! However, enemies behind them will not be attacked. Ranged units will be able to fire forward and diagonally forward. Combat preference will be towards enemies directly in front of them, then enemies inward (toward the center square), then outward.
Defend - Units remain still, but increase their defense. Units will also turn towards whichever enemy is the biggest threat (especially towards enemies behind them). Direction is lost if advance or fall back are used, then units face forward.
Special Ability - Some units may have unique callable special abilities. For instance, a mage may have an AOE firewall spell that he can cast. A single ability from a single unit uses up an entire turn. Also, abilities may be on a timer. For instance, firewall must recharge for 14 turns before it can be used again. Commanders usually come with at least one special ability.
End Combat - The attacking play may end combat at any time without going for the full 10 turns. Defending players may only end combat by retreating their commander off the field.
When an army has lost either it's commander or 70% of it's point value total, it will enter low morale mode. Basically what this means is that the formation will condense down as possible to be 3x3. Each row or column that has no units in it will be filled in by each outer row or column. This has a 20% chance of happening on every turn. This is to prevent an end game where two armies with minimal units can not hit each other due to incompatible formations.
Finally, the real stars of the game are the units themselves. Much like a collectible card game, each unit has the unique ability to change the rules. Here are some quick ideas:
Opportunity Attack - Free attack on forward enemy falling back.
Side Step - When advancing, move to left or right of forward enemy.
Counter - Immediately attack enemy that just attacked you.
Backstab - If enemy is hit from behind, x2 damage.
Combo Master - Successive Attack actions increase damage.
Heal - On Defense action, heal adjacent allies.
Unead - 50% chance of returning from dead after 4 turns.
Knockback - Can hit forward enemy back one space.
Cannonball - On Advance action, throws behind ally up to 4 squares forward.
Architect - On Defense action, build/fortify wall in forward cell.
Wide Slice - Attacks 3 cells in front of unit.
Bamf - Teleports to random locations on the field.
Device - Requires adjacent Gadgeteer to operate (ie catapult).
And so on. The point would be to create special abilities which modify or circumvent basic attack rules. For instance, the ability to knock back could be combo'd with two melee units with opportunity attacks to hit the unit three times. Of sending a thief unit with side step and backstab into the heart of the enemy formation to take out the commander quickly. Or attack a catapult unit from behind and take out their gadgeteers so the catapults become inoperable.