Transferring ownership of Annotation Sets

Problem

When annotation sets are created by WKS and assigned to the selected (by the Project Manager) human annotators (HA), the ownership of the annotation sets cannot be changed. This means that if a HA is temporarily or permanently unavailable to finish the assigned (via annotation set) annotation work, the work cannot be transfered to another HA and it cannot be done.

Consequences:

• When the HA is temporarily unavailable to finish the work, the entire Annotation Task (a group of one or more annotation sets, whose progress is tracked as a whole) is blocked and the entire annotation team is slowed down.
• When the HA becomes permanently unavailable, the entire Annotation Task is in limbo. The workaround is to redo a lot of work in a new Annotation Task - not an acceptable solution. 

Several options (ideas)

Allow transferring of Annotation Set ownership.

Several ideas:

1. Allow Project Manager to re-assign annotation sets - relatively safe (controlled) way to transfer ownership, but still may create issues with the fidelity of the annotation overlaps, as these overlaps were created by WKS with the original HAs in "mind."
   
   • PROS: Relatively easy and safe to implement
   • CONS: Side effects are expected; addresses a specific issue, but does not improve much the overall process.
2. Dynamically adjusted hidden annotation sets - this would be radical change of the annotation workflow and UX, but it may be a promising one:
   • Annotation sets can be maintained internally by the system, only for the purpose of ensuring the overlapping document annotations, but no need to be explicitly managed by Proj Mgr and assigned to specific HAs, etc. In this case, the workflow could be something like:
     1. Project Manager uploads the training documents and organizes them in Document Sets (Btw, the management of Document Sets also needs improvement.)
     2. The Project Manager creates Annotation Tasks and adds Document Sets to them
        
     3. The HAs signup for (pull, not push approach) one of more available (unfinished) Annotation Tasks
     4. For each HA in an Annotation Task, WKS automatically creates a pool of documents, making sure that the Project Manager's setting for annotation overlap are met
     5. Each HA is presented with their own pool of documents for annotation (that is the automatically generated hidden Annotation Set)
     6. As HAs work at different speed, and join and leave the Annotation Task (the main unit of work progress that is tracked by the Project Manager), and as Project Manager adds/removes documents from the Annotation Tasks (yes, in real life this happens and should be supported by WKS), WKS dynamically adjusts the pool of documents for each HA, to ensure proper annotation overlap and to maximize the annotation speed based on the different speed of the HAs.
     7. Ta-da!
   • PROS: Dynamic optimization of the annotation process by WKS should yield best overall results (hypothesis to be validated yet)
   • CONS: Radical change - longer and more costly to execute; higher risk (compared to 1) as this is real innovation.