Distributed file system module
Class diagram
TaskExecutor
- is the most important class in VeloxMR. It dispatches Tasks to other remote TaskExecutors or local TaskHandlers.
TaskHandler
- has two sub-classes for C++ and Python. It effectively executes the map and reduces functions.
Ireader and Iwriter
- manages storing and reading idata. They store idata in a tree form which is then saved to disk
Network module
Class diagram
It can be viewed as an Ad-hoc RPC library designed on the top of ASIO tuned to big data applications which provides streams, class serialization, and RPC to BOOST ASIO.