HMLP: High-performance Machine Learning Primitives
Public Member Functions | Static Public Member Functions | Public Attributes | List of all members
hmlp::Scheduler Class Reference

#include <runtime.hpp>

Inheritance diagram for hmlp::Scheduler:
hmlp::mpi::MPIObject

Public Member Functions

 Scheduler (mpi::Comm comm)
 (Default) Scheduler constructor. More...
 
void Init (int n_worker)
 
void Finalize ()
 
void ReportRemainingTime ()
 
void MessageDependencyAnalysis (int key, int p, ReadWriteType type, Task *task)
 
void NewTask (Task *task)
 This function is called by RunTime::Submit() to record a new task. More...
 
void NewMessageTask (MessageTask *task)
 
void NewListenerTask (ListenerTask *task)
 
void ExecuteNestedTasksWhileWaiting (Worker *me, Task *waiting_task)
 
void Summary ()
 
- Public Member Functions inherited from hmlp::mpi::MPIObject
 MPIObject (mpi::Comm comm)
 
void AssignCommunicator (mpi::Comm &comm)
 
mpi::Comm GetComm ()
 
mpi::Comm GetPrivateComm ()
 
int GetCommSize ()
 
int GetCommRank ()
 
int Comm_size ()
 
int Comm_rank ()
 
int Barrier ()
 
int PrivateBarrier ()
 

Static Public Member Functions

static void DependencyAdd (Task *source, Task *target)
 Add an direct edge (dependency) from source to target. More...
 

Public Attributes

int n_worker = 0
 
size_t timeline_tag
 
double timeline_beg
 
deque< Task * > ready_queue [MAX_WORKER]
 
Lock ready_queue_lock [MAX_WORKER]
 
deque< Task * > nested_queue [MAX_WORKER]
 
Lock nested_queue_lock [MAX_WORKER]
 
vector< unordered_map< int, ListenerTask * > > listener_tasklist
 
Lock listener_queue_lock
 
float time_remaining [MAX_WORKER]
 

Detailed Description

end class MatrixReadWrite class Scheduler

Constructor & Destructor Documentation

hmlp::Scheduler::Scheduler ( mpi::Comm  user_comm)

(Default) Scheduler constructor.

end MatrixReadWrite::DependencyCleanUp() class Scheduler

Set now as the begining of the time table.

Member Function Documentation

void hmlp::Scheduler::DependencyAdd ( Task *  source,
Task *  target 
)
static

Add an direct edge (dependency) from source to target.

Manually describe the dependencies

end Scheduler::ReportRemainingTime()

Avoid self-loop.

Update the source out-going edges.

Update the target incoming edges.

Only increase the dependency count for incompleted tasks.

void hmlp::Scheduler::ExecuteNestedTasksWhileWaiting ( Worker *  me,
Task *  waiting_task 
)

end Scheduler::ConsumeTasks()

Try to get a nested task.

void hmlp::Scheduler::Finalize ( )

ebd Scheduler::NewListenerTask()

Print out statistics of this epoch

Reset remaining time.

Free all normal tasks and reset tasklist.

Free all nested tasks and reset nested_tasklist.

Reset listener_tasklist

Clean up all message dependencies.

void hmlp::Scheduler::Init ( int  n_worker)

Adjust the number of active works.

Reset normal and nested task counter.

Reset async distributed consensus variables.

end pragma omp parallel for

void hmlp::Scheduler::MessageDependencyAnalysis ( int  key,
int  p,
ReadWriteType  type,
Task *  task 
)

end Scheduler::Init()

void hmlp::Scheduler::NewListenerTask ( ListenerTask *  task)

We use a duplicated (private) communicator to handle message tasks.

Counted toward termination criteria.

void hmlp::Scheduler::NewMessageTask ( MessageTask *  task)

end Scheduler::NewTask()

We use a duplicated (private) communicator to handle message tasks.

Counted toward termination criteria.

void hmlp::Scheduler::NewTask ( Task *  task)

This function is called by RunTime::Submit() to record a new task.

end Scheduler::MessageDependencyAnalysis()

Acquire the exclusive right to access the tasklist.

void hmlp::Scheduler::ReportRemainingTime ( )

end Scheduler::Finalize()

void hmlp::Scheduler::Summary ( )

end Scheduler::Listen()

Member Data Documentation

vector<unordered_map<int, ListenerTask*> > hmlp::Scheduler::listener_tasklist

The hashmap for asynchronous MPI tasks.

deque<Task*> hmlp::Scheduler::nested_queue[MAX_WORKER]

The ready queue for nested tasks.

Lock hmlp::Scheduler::nested_queue_lock[MAX_WORKER]

The tasklist records all nested tasks created in this epoch. Accessing nested_ready_queue requires exclusive right to avoid race condition.

deque<Task*> hmlp::Scheduler::ready_queue[MAX_WORKER]

Ready queues for normal tasks.

Lock hmlp::Scheduler::ready_queue_lock[MAX_WORKER]

The tasklist records all tasks created in this epoch. Accessing ready_queue requires exclusive right to avoid race condition.

The documentation for this class was generated from the following files: