Provide Option to Automatically Pull and Prune Containers from Exec Hosts

Currently LSF allows defining of an application profile to pull images selectively based upon configuration. However there is now way to provide pre-staging of images, and pruning of under utilized images.

The customer would like support similar to what is available in other solutions.

I would suggest, either overlay configuration options into lsf.conf, or come up with another LSF configuration file such as: lsf.containermgmt.

The format of the file would include:

1) One to many repositories to pull containers from
2) Maximum target size to maintain for all images maintained locally on each compute node (could be a trigger point). Repos over this size would trigger an immediate prune/sync to reclaim space.
3) TTL for all containers in seconds, minutes, hours, or days. So that container version that have not been used in that time period are automatically pruned. This pruning would take place as required via the Maximum target size as noted above, or the scheduled time of day to prune/sync.
4) A list of containers and tag names (for example ':latest', that will be specifically synced to all Docker compute nodes. An option could be '*:latest' for example. Wildcards or regular expressions should be honored.
5) A time of day and day or days of week to prune/sync.
6) The maximum concurrent hosts that can be pruning/syncing concurrently

Additionally, provide a set of cli commands that would provide statistics on which hosts were pruned, the current size of all containers on a per host basis, the current status of pruning/syncing, and a force command to force pruning or syncing immediately for a host or host group.