3.1.23. Nbor list renewal

Nbor lists are used for these purposes, in logical order

  1. list_t%check_nbors() uses the nbor list to determine which nbor tasks to run list_t%check_ready() on. In list_t%check_ready() the nbor list is used to figure out if a task is ready to be updated, by comparing the states of tasks with the function task_t%is_ahead_of().
  2. After the task gets picked from the ready queue, e.g. by dispatcher0_t%update, the nbor list is used by task_t%dnload to pick up the information from its nbors that it needs, in order to update.
  3. After the task has updated, and if it is a boundary task, the nbor list is used in list_t%send_to_vnbors(), which has as the main purpose to use MPI transparent / “invisble”. To accomplish this, first of all the virtual copy of the active task must be updated, and then the check_ready() must be used on the nbor task, by a check_nbors() being executed on the virtual copy of the active task.

At which point can one abandone an existing nbor list, and adopt a new one, without creating a change of deadlock? Clearly, not after step 1), since the nbor lists used in %dnload() must be the same as the one used to check if the task was ready to update. Also, clearly not after step 2, since the nbor list at that point still carries information about which tasks were blocked by the active task not yet having been updated.

If a new task has been added by AMR, which happens between step 2 and 3, the requirements for being “ready” will change, for all nbors of the active task that will have the new task as a new nbor. All the previous nbors of the active task, and the active task itself, need to have new nbor lists generated.

Likewise, if a task is removed by AMR, all tasks in the nbor list of the active task need to have new nbor lists generated. The best point to do this is just after the task has updated, since if it is done before, there is no longer consistency between the nbor list and the task update.

With task addition, there is no problem doing the next check_nbors() with the old nbor list, since the new task by definition carries the same information as the parent task, and hence does not add any new constraint on “readiness”, nor would adding the new task in the nbor list before the dnload() add any new information.

Both with task addition and with task removal, we require that the nbor list is generated by the thread assigned to update the task. By the reasoning above, this must be done at the very end of the update.

In the case of task removal, which is decided before the task update, there is no reason to actually remove the task until after it has updated; the update can by definition take place with the existing nbor list, and some nbors of the removed task have at the time already been added to the ready queue, and need access to the task as it was at the time.

Whether one removes the task before it is updated or after it has updated, one needs to trigger new nbor lists to be generated for all of the nbor tasks of the removed task. Those new nbor lists will not be generated until after those tasks have updated