How does the Reader read the data of the Writer (top-level logic)
Generally, the reader is created with a Node. For example, in the Component class (the Component can process up to four message types at the same time. Here we will choose one data type to explain):
reader = node_->CreateReader<M0>(reader_cfg)
Therefore, we can analyze how Node::CreateReader creates a Reader:
template <typename MessageT>
auto Node::CreateReader(const ReaderConfig& config,
const CallbackFunc<MessageT>& reader_func)
-> std::shared_ptr<cyber::Reader<MessageT>> {
std::lock_guard<std::mutex> lg(readers_mutex_);
if (readers_.find(config.channel_name) != readers_.end()) {
AWARN << "Failed to create reader: reader with the same channel already "
"exists.";
return nullptr;
}
auto reader =
node_channel_impl_->template CreateReader<MessageT>(config, reader_func);
if (reader != nullptr) {
readers_.emplace(std::make_pair(config.channel_name, reader));
}
return reader;
}
You can see that Node::CreateReader is created by calling NodeChannelImpl::CreateReader. Node::CreateReader has multiple function overloads, and NodeChannelImpl::CreateReader also has its corresponding overloaded functions for use:
template <typename MessageT>
auto NodeChannelImpl::CreateReader(const ReaderConfig& config,
const CallbackFunc<MessageT>& reader_func)
-> std::shared_ptr<Reader<MessageT>> {
//Parameter configuration, which will be passed down hierarchically. You can see that only channel name is configured here
proto::RoleAttributes role_attr;
role_attr.set_channel_name(config.channel_name);
role_attr.mutable_qos_profile()->CopyFrom(config.qos_profile);
return this->template CreateReader<MessageT>(role_attr, reader_func,
config.pending_queue_size);
}
template <typename MessageT>
auto NodeChannelImpl::CreateReader(const proto::RoleAttributes& role_attr,
const CallbackFunc<MessageT>& reader_func,
uint32_t pending_queue_size)
-> std::shared_ptr<Reader<MessageT>> {
if (!role_attr.has_channel_name() || role_attr.channel_name().empty()) {
AERROR << "Can't create a reader with empty channel name!";
return nullptr;
}
proto::RoleAttributes new_attr(role_attr);
//Populate attr
FillInAttr<MessageT>(&new_attr);
std::shared_ptr<Reader<MessageT>> reader_ptr = nullptr;
if (!is_reality_mode_) {
reader_ptr =
std::make_shared<blocker::IntraReader<MessageT>>(new_attr, reader_func);
} else {
reader_ptr = std::make_shared<Reader<MessageT>>(new_attr, reader_func,
pending_queue_size);
}
RETURN_VAL_IF_NULL(reader_ptr, nullptr);
//Call the created reader and execute its Init function
RETURN_VAL_IF(!reader_ptr->Init(), nullptr);
return reader_ptr;
}
NodeChannelImpl::CreateReader other function overloads will eventually call the following NodeChannelImpl::CreateReader to create a reader. For is_reality_mode_， We default to the real environment, so is_reality_mode_ Is true. Therefore, we continue to analyze Reader:
template <typename MessageT>
Reader<MessageT>::Reader(const proto::RoleAttributes& role_attr,
const CallbackFunc<MessageT>& reader_func,
uint32_t pending_queue_size)
: ReaderBase(role_attr),
pending_queue_size_(pending_queue_size),
reader_func_(reader_func) {
blocker_.reset(new blocker::Blocker<MessageT>(blocker::BlockerAttr(
role_attr.qos_profile().depth(), role_attr.channel_name())));
}
The broker here is the data manager of the Reader. There is a queue at the bottom to store the received messages. There is no analysis here. The constructor of the Reader is just a simple initialization. Back to NodeChannelImpl::CreateReader, you can see that the Reader::Init function is called:
template <typename MessageT>
bool Reader<MessageT>::Init() {
if (init_.exchange(true)) {
return true;
}
std::function<void(const std::shared_ptr<MessageT>&)> func;
//If the callback function is passed in when creating the reader, it will also be executed. Otherwise, only Reader::Enqueue will be executed, that is, it will be put into the blocker
if (reader_func_ != nullptr) {
func = [this](const std::shared_ptr<MessageT>& msg) {
this->Enqueue(msg);
this->reader_func_(msg);
};
} else {
func = [this](const std::shared_ptr<MessageT>& msg) { this->Enqueue(msg); };
}
auto sched = scheduler::Instance();
croutine_name_ = role_attr_.node_name() + "_" + role_attr_.channel_name();
//Create DataVisitor data store
auto dv = std::make_shared<data::DataVisitor<MessageT>>(
role_attr_.channel_id(), pending_queue_size_);
// Using factory to wrap templates.
croutine::RoutineFactory factory =
croutine::CreateRoutineFactory<MessageT>(std::move(func), dv);
//Create event handling process
if (!sched->CreateTask(factory, croutine_name_)) {
AERROR << "Create Task Failed!";
init_.store(false);
return false;
}
//Get data receiver
receiver_ = ReceiverManager<MessageT>::Instance()->GetReceiver(role_attr_);
this->role_attr_.set_id(receiver_->id().HashValue());
channel_manager_ =
service_discovery::TopologyManager::Instance()->channel_manager();
//Join the network topology of cyber
JoinTheTopology();
return true;
}
Let's analyze this function step by step. DataVisitor is used as data memory:
template <typename M0, typename M1>
DataVisitor(const std::vector<VisitorConfig>& configs)
: buffer_m0_(configs[0].channel_id,
new BufferType<M0>(configs[0].queue_size)),
buffer_m1_(configs[1].channel_id,
new BufferType<M1>(configs[1].queue_size)) {
DataDispatcher<M0>::Instance()->AddBuffer(buffer_m0_);
DataDispatcher<M1>::Instance()->AddBuffer(buffer_m1_);
data_notifier_->AddNotifier(buffer_m0_.channel_id(), notifier_);
data_fusion_ = new fusion::AllLatest<M0, M1>(buffer_m0_, buffer_m1_);
}
template <typename M0, typename M1>
bool TryFetch(std::shared_ptr<M0>& m0, std::shared_ptr<M1>& m1) { // NOLINT
if (data_fusion_->Fusion(&next_msg_index_, m0, m1)) {
next_msg_index_++;
return true;
}
return false;
}
template <typename M0>
DataVisitor(uint64_t channel_id, uint32_t queue_size)
: buffer_(channel_id, new BufferType<M0>(queue_size)) {
DataDispatcher<M0>::Instance()->AddBuffer(buffer_);
data_notifier_->AddNotifier(buffer_.channel_id(), notifier_);
}
template <typename M0>
bool TryFetch(std::shared_ptr<M0>& m0) { // NOLINT
if (buffer_.Fetch(&next_msg_index_, m0)) {
next_msg_index_++;
return true;
}
return false;
}
The constructors of the above two datavisitors, one is to process two message types and the other is to process one message type (that is, what we are analyzing), have created message buffers, which will store the received data. Then you can see that the only difference between the two constructors except the number of messages is that there is a member variable data for multiple message types_ fusion_， So this data_fusion_ What is it? In the multi message type DataVisitor, because it needs to receive multiple messages, it needs to fuse the data. You can see the data_notifier_ This adds a buffer_m0_ That is, the buffer of the first message. Only after the first message comes, the data fusion mechanism will be triggered. The last message of several other messages will be taken out, and then several messages will be delivered. Therefore, we need to think carefully about which message is the first message. It can be seen that in the TryFetch function, only one TryFetch directly takes data from the buffer, while TryFetch < M0, M1 > uses data_fusion_ To obtain data. When the data is not fused, false is returned.
After that, we will continue to analyze the differences between datavisitors with different message numbers. The DataDispatcher singleton is used in the constructor to add the created buffer. What is the specific purpose of this DataDispatcher singleton?
template <typename T>
void DataDispatcher<T>::AddBuffer(const ChannelBuffer<T>& channel_buffer) {
std::lock_guard<std::mutex> lock(buffers_map_mutex_);
auto buffer = channel_buffer.Buffer();
BufferVector* buffers = nullptr;
//If this channel_ If the ID has been added, it will be directly put into the BufferVector
//It can be seen from here that there can be multiple corresponding buffer s for the same channel
if (buffers_map_.Get(channel_buffer.channel_id(), &buffers)) {
buffers->emplace_back(buffer);
} else {
BufferVector new_buffers = {buffer};
buffers_map_.Set(channel_buffer.channel_id(), new_buffers);
}
}
//buffers_map_ AtomicHashMap is a lock free map implemented by cyber. Those interested can learn about its implementation
AtomicHashMap<uint64_t, BufferVector> buffers_map_
You can see that the DataDispatcher puts the message buffer created by the DataVisitor into its own buffers_map_ among. Then continue to analyze. There is another data below_ notifier_， It is defined as DataNotifier* data_notifier_ = DataNotifier::Instance() is also a singleton:
inline void DataNotifier::AddNotifier(
uint64_t channel_id, const std::shared_ptr<Notifier>& notifier) {
std::lock_guard<std::mutex> lock(notifies_map_mutex_);
NotifyVector* notifies = nullptr;
if (notifies_map_.Get(channel_id, ¬ifies)) {
notifies->emplace_back(notifier);
} else {
NotifyVector new_notify = {notifier};
notifies_map_.Set(channel_id, new_notify);
}
}
//notifies_map_ definition
AtomicHashMap<uint64_t, NotifyVector> notifies_map_;
DataNotifier::AddNotifier has the same function as DataDispatcher::AddBuffer, but AddNotifier adds a notifier to its own map. What is the specific function of this notifier?
struct Notifier {
std::function<void()> callback;
};
The Notifier function is very simple, that is, there is a callback member, which will be called when the message is received and analyzed later. The analysis of DataVisitor is finished. Its main function is to put the Buffer and Notifier of messages into the DataDispatcher and DataNotifier.
Let's go back to Reader::Init() for analysis:
croutine::RoutineFactory factory =
croutine::CreateRoutineFactory<MessageT>(std::move(func), dv);
if (!sched->CreateTask(factory, croutine_name_)) {
AERROR << "Create Task Failed!";
init_.store(false);
return false;
}
When DataVisitor is finished, it calls croutine::CreateRoutineFactory to create a croutine::RoutineFactory:
template <typename M0, typename F>
RoutineFactory CreateRoutineFactory(
F&& f, const std::shared_ptr<data::DataVisitor<M0>>& dv) {
RoutineFactory factory;
//Set DataVisitor
factory.SetDataVisitor(dv);
factory.create_routine = [=]() {
return [=]() {
std::shared_ptr<M0> msg;
for (;;) {
CRoutine::GetCurrentRoutine()->set_state(RoutineState::DATA_WAIT);
if (dv->TryFetch(msg)) {
f(msg);
CRoutine::Yield(RoutineState::READY);
} else {
CRoutine::Yield();
}
}
};
};
return factory;
}
croutine::CreateRoutineFactory mainly creates the execution function of the coroutine, polls until the message arrives, and then executes the callback function set in Reader::Init().
Then you call sched - > createtask to create a task:
bool Scheduler::CreateTask(std::function<void()>&& func,
const std::string& name,
std::shared_ptr<DataVisitorBase> visitor) {
if (cyber_unlikely(stop_.load())) {
ADEBUG << "scheduler is stoped, cannot create task!";
return false;
}
//It mainly registers the name, and then returns the Hash of name
auto task_id = GlobalData::RegisterTaskName(name);
//The cooperative process implemented by cyber is not analyzed here. It can be considered as thread
auto cr = std::make_shared<CRoutine>(func);
cr->set_id(task_id);
cr->set_name(name);
AINFO << "create croutine: " << name;
//Scheduling task
if (!DispatchTask(cr)) {
return false;
}
//visitor is the DataVisitor passed in by CreateRoutineFactory
if (visitor != nullptr) {
visitor->RegisterNotifyCallback([this, task_id]() {
if (cyber_unlikely(stop_.load())) {
return;
}
this->NotifyProcessor(task_id);
});
}
return true;
}
Where RegisterNotifyCallback is the parent class DataVisitorBase implementation of DataVisitor:
void RegisterNotifyCallback(std::function<void()>&& callback) {
notifier_->callback = callback;
}
You can see that the notifier is set_ When the callback is executed, the Scheduler::NotifyProcessor will be executed to notify the Processor to execute the process, that is, the factory in CreateRoutineFactory create_ routine.
Return to Reader::Init() to continue analysis:
receiver_ = ReceiverManager<MessageT>::Instance()->GetReceiver(role_attr_);
ReceiverManager manages all receivers as a single instance of Receiver management, which calls its GetReceiver:
template <typename MessageT>
auto ReceiverManager<MessageT>::GetReceiver(
const proto::RoleAttributes& role_attr) ->
typename std::shared_ptr<transport::Receiver<MessageT>> {
std::lock_guard<std::mutex> lock(receiver_map_mutex_);
// because multi reader for one channel will write datacache multi times,
// so reader for datacache we use map to keep one instance for per channel
const std::string& channel_name = role_attr.channel_name();
if (receiver_map_.count(channel_name) == 0) {
receiver_map_[channel_name] =
//transport is the lowest data processing class. We will not analyze it here, but call its CreateReceiver here
//Function creates a Receiver, and when data is received, the lamda passed in here will be called
transport::Transport::Instance()->CreateReceiver<MessageT>(
role_attr, [](const std::shared_ptr<MessageT>& msg,
const transport::MessageInfo& msg_info,
const proto::RoleAttributes& reader_attr) {
(void)msg_info;
(void)reader_attr;
PerfEventCache::Instance()->AddTransportEvent(
TransPerf::DISPATCH, reader_attr.channel_id(),
msg_info.seq_num());
data::DataDispatcher<MessageT>::Instance()->Dispatch(
reader_attr.channel_id(), msg);
PerfEventCache::Instance()->AddTransportEvent(
TransPerf::NOTIFY, reader_attr.channel_id(),
msg_info.seq_num());
});
}
return receiver_map_[channel_name];
}
You can see that the function of ReceiverManager::GetReceiver is to create a receiver, and then according to the channel_name is put into the map. When the underlying communication module receives the data, it will call the incoming callback function. In the callback function, we see the data::DataDispatcher mentioned above and call its Dispatch function to distribute the received data:
template <typename T>
bool DataDispatcher<T>::Dispatch(const uint64_t channel_id,
const std::shared_ptr<T>& msg) {
BufferVector* buffers = nullptr;
if (apollo::cyber::IsShutdown()) {
return false;
}
if (buffers_map_.Get(channel_id, &buffers)) {
for (auto& buffer_wptr : *buffers) {
if (auto buffer = buffer_wptr.lock()) {
std::lock_guard<std::mutex> lock(buffer->Mutex());
buffer->Fill(msg);
}
}
} else {
return false;
}
return notifier_->Notify(channel_id);
}
//notifier_ Definition of
DataNotifier* notifier_ = DataNotifier::Instance();
You can see that DataDispatcher::Dispatch fills the buffer with the message passed in by the underlying communication module, and the buffer here is passed in the constructor of DataVisitor. After the buffer is filled, the originally created collaboration can take out the data through DataVisitor::TryFetch, and the notifier - > wakes up the collaboration Notify(channel_id):
inline bool DataNotifier::Notify(const uint64_t channel_id) {
NotifyVector* notifies = nullptr;
if (notifies_map_.Get(channel_id, ¬ifies)) {
for (auto& notifier : *notifies) {
if (notifier && notifier->callback) {
notifier->callback();
}
}
return true;
}
return false;
}
DataNotifier::Notify from notifications_ map_ Take out the notifier originally passed in the constructor of DataVisitor and execute its callback function. This callback is set in Scheduler::CreateTask. Finally, Scheduler::NotifyProcessor is called to wake up the process.
So far, the analysis of the entire upper layer data flow of how the Reader reads the data from the Writer has been completed. To summarize:
Transport - > datadispatcher - > DataVisitor::buffer - > datanotifier:: instance():: notify - > callback - > scheduler:: notifyprocessor - > execute the process - > take data from DataVisitor::buffer - > put it into the blocker of reader - > execute the callback (if any) passed in by the user
In the next chapter, we will talk about how the bottom layer communicates, and there is still code in Reader::Init():
channel_manager_ =
service_discovery::TopologyManager::Instance()->channel_manager();
JoinTheTopology();
Here is the service discovery function of cyber, which will be analyzed in the following chapters.