Bulk pickaxe reassignment API, plus primary/backup/tertiary pickaxe failover

Pickaxes are a critical service, and there are two use cases for this:

1. Server patching and updates. Every month, the servers that run the Foreman pickaxe services need OS patches and updates. Today someone has to bulk move the miners off a pickaxe by hand, or accept an outage for as long as the patching takes. An API endpoint for bulk reassignment fixes this, because the moves can then be part of an automated patch routine: move the miners off, patch and update the server, then move them back.

2. High availability. Designate a primary and a backup pickaxe per miner: dual assignment with a heartbeat and automated failover. In our experience, most pickaxe outages aren't hardware failures. They come from memory or other process overruns in the pickaxe service itself. When that happens during off-hours, it often isn't noticed right away, and the miners on that pickaxe go unmanaged until someone catches it. HA would handle these overnight and weekend outages automatically, instead of depending on someone noticing.

Our suggestion:

- Short term: build the bulk reassignment endpoint regardless. It covers patching, and it lets customers work around the missing HA features for the cases that matter most.

- Longer term: let us designate a primary, backup and tertiary pickaxe for each miner's assignment. If a miner's primary pickaxe is down for a set number of polling intervals, Foreman moves the miner to the backup (or tertiary). Once the primary has been back for a set number of polling intervals, Foreman moves the miner back. Please include the tertiary level. This is a critical service, and it takes no more time to include it while building it.

Thanks,

Chad