[network simulation] NS3 gym / RL TCP

Posted by SyWill on Sun, 19 Sep 2021 07:55:35 +0200


# Terminal 1 
# ./waf --run "rl-tcp --transport_prot=TcpNewReno"
./waf --run "rl-tcp --transport_prot=TcpRl"

# Terminal 2
cd ./scratch/rl-tcp
./test_tcp.py --start=0

Basic interface

  1. For example, in a Python script, write this. (Note: gym.make('ns3-v0 ') starts the ns-3 simulation script in the current working directory)
import gym
import ns3gym
import MyAgent

env = gym.make('ns3-v0')
obs = env.reset()
agent = MyAgent.Agent()

while True:
	action = agent.get_action(obs)
	obs, reward, done, info = env.step(action)

	if done:
env = ns3env.Ns3Env(port=port, stepTime=stepTime, startSim=startSim, simSeed=seed, simArgs=simArgs, debug=debug)
# simpler:
#env = ns3env.Ns3Env()

ob_space = env.observation_space
ac_space = env.action_space
  1. Any ns-3 simulation script can be used as a Gym environment, and only need to instantiate opengym interface (PTR < opengym interface > opengym interface;); And implement ns3-gym C + + interface, which includes the following functions:
Ptr<OpenGymSpace> GetObservationSpace();
Ptr<OpenGymSpace> GetActionSpace();
Ptr<OpenGymDataContainer> GetObservation();
float GetReward();
bool GetGameOver();
std::string GetExtraInfo();
bool ExecuteActions(Ptr<OpenGymDataContainer> action);

This project is two versions of RL-TCP (i.e. time-based and event based), which can control the convergence window and Slow Start Threshold. Both versions inherit from tcpconceptionops, so they can be used in ns3::TcpL4Protocol::SocketType.

In this example, the event based interface is used to implement TCP NewReno and ns3gym is used to simulate communication with ns-3. This example can be used as the starting point of RL based TCP convergence control algorithms.

1 sim.cc

1.1 Emacs mode line

// Explain the predetermined format of the code
// Describes the C + + code specification of ns-3. The uniform style is for ease of reading.
// The specific rules are as follows: 1. The first letter of function name, method name and class name is capitalized, and other letters are lowercase; 2. Variable names are lowercase, other initials are uppercase, and local variables are expressed in g_ The member variables of the class begin with m_ The custom type starts with_ Start with t, and all constants are capitalized
; // Ensure that developers using the Emacs editor can open the file correctly
 * Copyright (c) 2018 Piotr Gawlowicz // Ensure that the following code is under the GPL copyright, that is, open source
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License version 2 as
 * published by the Free Software Foundation;
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * GNU General Public License for more details.
 * You should have received a copy of the GNU General Public License
 * along with this program; if not, write to the Free Software
 * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
 * Author: Piotr Gawlowicz <gawlowicz.p@gmail.com>
 * Based on script: ./examples/tcp/tcp-variants-comparison.cc
 * Topology:
 *   Right Leafs (Clients)                      Left Leafs (Sinks)
 *           |            \                    /        |
 *           |             \    bottleneck    /         |
 *           |              R0--------------R1          |
 *           |             /                  \         |
 *           |   access   /                    \ access |
 *           N -----------                      --------N

1.2 header file

The referenced header file is located in build/ns3

#include <iostream>
#include <fstream>
#include <string>

#include "ns3/core-module.h"
#include "ns3/network-module.h"
#include "ns3/internet-module.h"
#include "ns3/point-to-point-module.h"
#include "ns3/point-to-point-layout-module.h"
#include "ns3/applications-module.h"
#include "ns3/error-model.h"
#include "ns3/tcp-header.h"
#include "ns3/enum.h"
#include "ns3/event-id.h"
#include "ns3/flow-monitor-helper.h"
#include "ns3/ipv4-global-routing-helper.h"
#include "ns3/traffic-control-module.h"

#include "ns3/opengym-module.h"
#include "tcp-rl.h"

1.3 namespace

If you use a namespace other than ns3, add xx::yy, such as Time::NS, std::cout, and std::min().
This is a new mechanism introduced by C + +. It is mainly to solve the problem of duplicate name conflict. Names are placed in multiple namespaces to prevent name conflict.
For example, the objects provided by the standard C + + library are placed in the namespace std.

using namespace ns3;

1.4 Logging system

Define the log module of the log component, which allows the script to print auxiliary information using the macro definition in the log system.
That is, print the relevant information on the console.

NS_LOG_COMPONENT_DEFINE ("TcpVariantsComparison"); // Register a record component named "TcpVariantsComparison" with the ns-3 system.

static std::vector<uint32_t> rxPkts; 
// rePkts[i] indicates the number of packets received by the ith node
// Defines a variable, a static array of unsigned 32 byte integers with the name rxPkts.
// Static modified variable: a static variable has only one storage space in memory. It does not belong to an instance object and is shared by all objects of a class.
// const modified variable: make the variable have constant attributes, that is, its value cannot be changed in future use.

// Record and print the number of packets received by each node
static void
CountRxPkts(uint32_t sinkId, Ptr<const Packet> packet, const Address & srcAddr)

static void
  uint32_t size = rxPkts.size();
  for (uint32_t i=0; i<size; i++){
    NS_LOG_UNCOND("---SinkId: "<< i << " RxPkts: " << rxPkts.at(i));
//  NS_LOG_LOGIC ("IP Header size is: " << ip_header);

The Logging system has seven levels:

  • LOG_ERROR error, print information
  • LOG_WARN has warning, print information
  • LOG_DEBUG has debugging and printing information
  • LOG_INFO print program related information
  • LOG_FUNCTION has a function call and prints the call related information
  • LOG_LOGIC prints the overall logic description information
  • LOG_ALL print all information

There is also an unconditional log level whose log verbosity level is independent of component selection

  • LOG_UNCOND records information unconditionally

1.5 variable definition

int main (int argc, char *argv[])
  // Define various variables
  uint32_t openGymPort = 5555; // Define the open gym port number as 5555
  double tcpEnvTimeStep = 0.1;

  uint32_t nLeaf = 1;
  std::string transport_prot = "TcpRl";
  double error_p = 0.0;
  std::string bottleneck_bandwidth = "2Mbps";
  std::string bottleneck_delay = "0.01ms";
  std::string access_bandwidth = "10Mbps";
  std::string access_delay = "20ms";
  std::string prefix_file_name = "TcpVariantsComparison";
  uint64_t data_mbytes = 0;
  uint32_t mtu_bytes = 400;
  double duration = 10.0;
  uint32_t run = 0;
  bool flow_monitor = false;
  bool sack = true;
  std::string queue_disc_type = "ns3::PfifoFastQueueDisc";
  std::string recovery = "ns3::TcpClassicRecovery";

1.6 command line parameters

  CommandLine cmd;
  // required parameters for OpenGym interface
  cmd.AddValue ("openGymPort", "Port number for OpenGym env. Default: 5555", openGymPort); // The first parameter: corresponds to the command line. The second parameter: description, displayed when - printHelp. The third parameter: corresponds to the script variable.
  cmd.AddValue ("simSeed", "Seed for random generator. Default: 1", run);
  cmd.AddValue ("envTimeStep", "Time step interval for time-based TCP env [s]. Default: 0.1s", tcpEnvTimeStep);
  // other parameters
  cmd.AddValue ("nLeaf",     "Number of left and right side leaf nodes", nLeaf);
  cmd.AddValue ("transport_prot", "Transport protocol to use: TcpNewReno, "
                "TcpHybla, TcpHighSpeed, TcpHtcp, TcpVegas, TcpScalable, TcpVeno, "
                "TcpBic, TcpYeah, TcpIllinois, TcpWestwood, TcpWestwoodPlus, TcpLedbat, "
		            "TcpLp, TcpRl, TcpRlTimeBased", transport_prot);
  cmd.AddValue ("error_p", "Packet error rate", error_p);
  cmd.AddValue ("bottleneck_bandwidth", "Bottleneck bandwidth", bottleneck_bandwidth);
  cmd.AddValue ("bottleneck_delay", "Bottleneck delay", bottleneck_delay);
  cmd.AddValue ("access_bandwidth", "Access link bandwidth", access_bandwidth);
  cmd.AddValue ("access_delay", "Access link delay", access_delay);
  cmd.AddValue ("prefix_name", "Prefix of output trace file", prefix_file_name);
  cmd.AddValue ("data", "Number of Megabytes of data to transmit", data_mbytes);
  cmd.AddValue ("mtu", "Size of IP packets to send in bytes", mtu_bytes);
  cmd.AddValue ("duration", "Time to allow flows to run in seconds", duration);
  cmd.AddValue ("run", "Run index (for setting repeatable seeds)", run);
  cmd.AddValue ("flow_monitor", "Enable flow monitor", flow_monitor);
  cmd.AddValue ("queue_disc_type", "Queue disc type for gateway (e.g. ns3::CoDelQueueDisc)", queue_disc_type);
  cmd.AddValue ("sack", "Enable or disable SACK option", sack);
  cmd.AddValue ("recovery", "Recovery algorithm type to use (e.g., ns3::TcpPrrRecovery", recovery);
  cmd.Parse (argc, argv);

  transport_prot = std::string ("ns3::") + transport_prot;

  SeedManager::SetSeed (1);
  SeedManager::SetRun (run);

  NS_LOG_UNCOND("Ns3Env parameters:"); // Print ns3 environment parameters
  if (transport_prot.compare ("ns3::TcpRl") == 0 or transport_prot.compare ("ns3::TcpRlTimeBased") == 0)
    NS_LOG_UNCOND("--openGymPort: " << openGymPort);
  } else {
    NS_LOG_UNCOND("--openGymPort: No OpenGym");

  NS_LOG_UNCOND("--seed: " << run);
  NS_LOG_UNCOND("--Tcp version: " << transport_prot);
# Use the command line to display system property defaults
./waf --run "scratch/sim --PrintAttributes = ns3::PointToPointNetDevice"

# Use the command line to set the corresponding parameters
./waf --run "scratch/sim --nLeaf = 3"

Other writable commands:

--PrintHelp # print the help information
--PrintGroups # Print group list
--PrintTypeIds # Print all TypeIds
--PrintGroup = [group] # Print all typeids in the group
--PrintAttributes = [typeid] # Print all properties of this TypeId
--PrintGlobals # Print a list of globals

1.7 random variables

Function: generate random numbers to make the simulation results random.

  1. Seeding and independent repetition
  SeedManager::SetSeed (1); // When the seed is set, the parameters remain unchanged and the generated random number remains unchanged
  SeedManager::SetRun (run); // Set run ID
  1. random variable
  • ==Random number generator (RNG) = = generates random numbers by performing complex operations on the selected random seeds.
  • ns-3 provides a pseudo-random number generator (PRNG), which selects pseudo-random numbers from a set of finite numbers with the same probability.
  • Each RNG provides a long random number sequence, and the sequence length becomes the cycle period or cycle length.
  • A sequence is divided into several unrelated data streams. Each seed corresponds to a different RNG sequence, and each identification corresponds to a sub sequence segmented by the same sequence

Classes that declare random variables have a base class: RandomVariableStream
Derived classes of RandomVariableStream:

  • UniformRandomVariable: you can set the maximum and minimum value to return a random number in a uniform distribution.
  • ConstantRandomVariable
  • SequentialRandomVariable
  • ExponentialRandomVariable
  • NormalRandomVariable
  // Configure the error model
  // Here we use RateErrorModel with packet error rate
  Ptr<UniformRandomVariable> uv = CreateObject<UniformRandomVariable> (); // Default interval of random variable is [0,1] 
  // double min = 1.0; uv->SetAttribute("Min",DoubleValue(min));
  uv->SetStream (50);
  RateErrorModel error_model;
  error_model.SetRandomVariable (uv);
  error_model.SetUnit (RateErrorModel::ERROR_UNIT_PACKET);
  error_model.SetRate (error_p);

Change the run ID through the command line:

# NS_GLOBAL_VALUE = "RngRun = 5" ./waf --run scratch/sim
./waf --run "scratch/sim --RngRun=5"

1.8 creating OpenGym environment

  // OpenGym Env --- has to be created before any other thing
  Ptr<OpenGymInterface> openGymInterface;
  if (transport_prot.compare ("ns3::TcpRl") == 0)
    openGymInterface = OpenGymInterface::Get(openGymPort);
    // Use Config::SetDefault to set the default properties of an object
    Config::SetDefault ("ns3::TcpRl::Reward", DoubleValue (2.0)); // Reward when increasing congestion window
    Config::SetDefault ("ns3::TcpRl::Penalty", DoubleValue (-30.0)); // Penalty when decreasing congestion window

  if (transport_prot.compare ("ns3::TcpRlTimeBased") == 0)
    openGymInterface = OpenGymInterface::Get(openGymPort);
    Config::SetDefault ("ns3::TcpRlTimeBased::StepTime", TimeValue (Seconds(tcpEnvTimeStep))); // Time step of TCP env

1.9 other configurations

PDU: protocol data unit. PUD = PCI (protocol control information) + SDU (service data unit)
ADU: application data unit. It can be understood as the data unit actually transmitted on the network.

  // Calculate the ADU size
  Header* temp_header = new Ipv4Header ();
  uint32_t ip_header = temp_header->GetSerializedSize ();
  NS_LOG_LOGIC ("IP Header size is: " << ip_header);
  delete temp_header;
  temp_header = new TcpHeader ();
  uint32_t tcp_header = temp_header->GetSerializedSize ();
  NS_LOG_LOGIC ("TCP Header size is: " << tcp_header);
  delete temp_header;
  uint32_t tcp_adu_size = mtu_bytes - 20 - (ip_header + tcp_header);
  NS_LOG_LOGIC ("TCP ADU size is: " << tcp_adu_size);

  // Set the simulation start and stop time
  double start_time = 0.1;
  double stop_time = start_time + duration;

  // 4 MB of TCP buffer
  Config::SetDefault ("ns3::TcpSocket::RcvBufSize", UintegerValue (1 << 21));
  Config::SetDefault ("ns3::TcpSocket::SndBufSize", UintegerValue (1 << 21));
  Config::SetDefault ("ns3::TcpSocketBase::Sack", BooleanValue (sack));
  Config::SetDefault ("ns3::TcpSocket::DelAckCount", UintegerValue (2));

  Config::SetDefault ("ns3::TcpL4Protocol::RecoveryType",
                      TypeIdValue (TypeId::LookupByName (recovery)));
  // Select TCP variant
  if (transport_prot.compare ("ns3::TcpWestwoodPlus") == 0)
      // TcpWestwoodPlus is not an actual TypeId name; we need TcpWestwood here
      Config::SetDefault ("ns3::TcpL4Protocol::SocketType", TypeIdValue (TcpWestwood::GetTypeId ()));
      // the default protocol type in ns3::TcpWestwood is WESTWOOD
      Config::SetDefault ("ns3::TcpWestwood::ProtocolType", EnumValue (TcpWestwood::WESTWOODPLUS));
      TypeId tcpTid;
      NS_ABORT_MSG_UNLESS (TypeId::LookupByNameFailSafe (transport_prot, &tcpTid), "TypeId " << transport_prot << " not found");
      Config::SetDefault ("ns3::TcpL4Protocol::SocketType", TypeIdValue (TypeId::LookupByName (transport_prot)));

1.10 error module

Function: determine which data packets corresponding to the underlying distribution, rate and unit have errors, that is, random errors or packet losses.

  // Configure the error model
  // Here we use RateErrorModel with packet error rate
  Ptr<UniformRandomVariable> uv = CreateObject<UniformRandomVariable> (); // Defines a pointer uv to the uniformly distributed random variable generator object
  uv->SetStream (50); // SetStream() is a member function of ns3::RandomVariableStream class. The input is stream (the stream number of RngStream, - 1 indicates the automatic allocation of stream number), and there is no output.
  RateErrorModel error_model; // Instantiated error model
  error_model.SetRandomVariable (uv); // Generate random variable distribution of random variables. 
  error_model.SetUnit (RateErrorModel::ERROR_UNIT_PACKET); // Set the error unit used as PACET unit. 
  error_model.SetRate (error_p); // Set the error rate used by the model. The default error rate is 0.

1.11 creating a network topology

  // Create an intermediate point-to-point link
  // Create the point-to-point link helpers
  PointToPointHelper bottleNeckLink;
  bottleNeckLink.SetDeviceAttribute  ("DataRate", StringValue (bottleneck_bandwidth));
  bottleNeckLink.SetChannelAttribute ("Delay", StringValue (bottleneck_delay));
  //Bottlenecklink. Setdeviceattribute ("receiveerrormodel", pointervalue (& error_model)); / / set the error receiving attribute
  // Create links on both sides
  PointToPointHelper pointToPointLeaf;
  pointToPointLeaf.SetDeviceAttribute  ("DataRate", StringValue (access_bandwidth));
  pointToPointLeaf.SetChannelAttribute ("Delay", StringValue (access_delay));
  // Create dumbbell topology
  PointToPointDumbbellHelper d (nLeaf, pointToPointLeaf,
                                nLeaf, pointToPointLeaf,


  1. Construct dumbbell topology PointToPointDumbbellHelper
ns3::PointToPointDumbbellHelper::PointToPointDumbbellHelper	(	uint32_t 	nLeftLeaf,
PointToPointHelper 	leftHelper,
uint32_t 	nRightLeaf,
PointToPointHelper 	rightHelper,
PointToPointHelper 	bottleneckHelper 
  1. Construct star topology pointtopointstralhelper
ns3::PointToPointStarHelper::PointToPointStarHelper	(	uint32_t 	numSpokes,
PointToPointHelper 	p2pHelper 
  1. Construct network topology PointToPointGridHelper
ns3::PointToPointGridHelper::PointToPointGridHelper	(	uint32_t 	nRows,
uint32_t 	nCols,
PointToPointHelper 	pointToPoint 

1.12 install protocol stack and assign IP address

  // Install IP stack
  InternetStackHelper stack; // Create network protocol stack
  stack.InstallAll (); // Install the protocol stack on each node
  // Assign IP Addresses
  d.AssignIpv4Addresses (Ipv4AddressHelper ("", ""),
                         Ipv4AddressHelper ("", ""),
                         Ipv4AddressHelper ("", ""));

void PointToPointDumbbellHelper::AssignIpv4Addresses (
Ipv4AddressHelper leftIp,
Ipv4AddressHelper rightIp,
Ipv4AddressHelper routerIp)

1.13 flow control

  // Traffic Control
  TrafficControlHelper tchPfifo; // Create a flow control help instance to set queuing rules
  tchPfifo.SetRootQueueDisc ("ns3::PfifoFastQueueDisc"); // The default priority queue used by Linux system is pfifo_fast

  TrafficControlHelper tchCoDel;
  tchCoDel.SetRootQueueDisc ("ns3::CoDelQueueDisc"); // A packet queuing rule called CoDel

  DataRate access_b (access_bandwidth); // DataRate class in network module, unit: bps
  DataRate bottle_b (bottleneck_bandwidth);
  Time access_d (access_delay);
  Time bottle_d (bottleneck_delay);

  uint32_t size = static_cast<uint32_t>((std::min (access_b, bottle_b).GetBitRate () / 8) *
    ((access_d + bottle_d + access_d) * 2).GetSeconds ());

  Config::SetDefault ("ns3::PfifoFastQueueDisc::MaxSize",
                      QueueSizeValue (QueueSize (QueueSizeUnit::PACKETS, size / mtu_bytes)));
  Config::SetDefault ("ns3::CoDelQueueDisc::MaxSize",
                      QueueSizeValue (QueueSize (QueueSizeUnit::BYTES, size)));

  if (queue_disc_type.compare ("ns3::PfifoFastQueueDisc") == 0)
    tchPfifo.Install (d.GetLeft()->GetDevice(1));
    tchPfifo.Install (d.GetRight()->GetDevice(1));
  else if (queue_disc_type.compare ("ns3::CoDelQueueDisc") == 0)
    tchCoDel.Install (d.GetLeft()->GetDevice(1));
    tchCoDel.Install (d.GetRight()->GetDevice(1));
    NS_FATAL_ERROR ("Queue not recognized. Allowed values are ns3::CoDelQueueDisc or ns3::PfifoFastQueueDisc");

1.14 Internet module - routing

  NS_LOG_INFO ("Initialize Global Routing."); // Initialize global routing
  Ipv4GlobalRoutingHelper::PopulateRoutingTables (); // Populate routing table

Routing: the core of the network
Routing table: the most critical part of routing.
ns-3 is an open source software that can write its own routing protocol

  1. golbal centralied routing

Configure global routing for wired networks
The shortest path is used every time (it doesn't exist in reality). When creating a network protocol stack using the InternetStackHelper class, the global route will be bound to the node by default.
When the IP address is configured to the node, ipv4globalroutinghelper:: populatiroutingtables() will enable each node to have an IPv4 interface to receive the routing table, which is registered by the GlobalRouteManager.
Note: this protocol has no effect on wireless nodes (OLSR dynamic routing protocol is recommended) and can be used in Wi Fi AP nodes.

// Update routing table
Ipv4GlobalRoutingHelper::RecomputeRoutingTables (); 

// For example, update the routing table at 5s
Simulator::Schedule (Seconds (5), &Ipv4GlobalRoutingHelper::RecomputeRoutingTables); 

Properties that control routing behavior:

  • Ipv4globalrouting:: random ecmprouting whether data packets will randomly select routes with the same cost.
  • Ipv4GlobalRouting::RespondToInterfaceEvents whether the system finds events through the interface and dynamically updates the global routing.
  1. Unicast routing: unicast routing protocol
    Unicast routing protocols supported by ns-3:
  • IPv4 optimized link state routing (OLSR)
    Source code: src/olsr
    Example: examples/simple-point-to-point-olsr.cc
NodeContainer c;
// Enable OLSR
NS_LOG_INFO ("Enabling OLSR");
OlsrHelper olsr;
Ipv4StaticRoutingHelper staticRouting;
Ipv4ListRoutingHelper list;
list.Add (staticRouting, 0);
list.Add (olsr, 10); // OLSR protocol will be used before static routing protocol
InternetStackHelper internet;
internet.SetRoutingHelper (list);
internet.Install (c);
  • IPv4 Static Routing (unicast and multicast supported)
  • IPv4 ad hoc on demand distance vector (AODV)
  • IPv4 destination sequenced distance vector (DSDV)
  • IPv4 ListRouting (save the priority list of routing protocols)
    A priority list is provided to store multiple routes and support users to write their own routes. IPv4 will call these routing protocols according to this queue.
    Class IPv4ListRouting provides a pure virtual function (not implemented in the base class, but implemented in the inheritance class) void addroutingprotocol (PTR < ipv4routingprotocol > routingprotocol, int16_t priority); Allows users to add routing protocols.
    For example:
Ptr<MyRoutingProtocol> myRoutingProto = CreateObject<MyRoutingProtocol> (); // Instantiate your own routing object and save it in the pointer
listRoutingPtr->AddRoutingProtocol (myRoutingProto, -10); // Call from high to low according to the routing priority
  • IPv4 NixVectorRouting
  • IPv6 ListRouting
  • IPv6 Static Routing
  1. multicast routing: multicast routing Protocol
// Add multicast route to Node
Ipv4StaticRouting::AddMulticastRoute (Ipv4Address origin, Ipv4Address group, uint32_t inputInterface, std::vector<uint32_t> outputInterfaces);
// Parameter 1: a source address; Parameter 2: a set of destination addresses; Parameter 3: an input network interface; Parameter 4: an output network interface.

1.15 application layer module

  // Install apps in left and right nodes
  // Install receiver application
  uint16_t port = 50000;
  Address sinkLocalAddress (InetSocketAddress (Ipv4Address::GetAny (), port));
  PacketSinkHelper sinkHelper ("ns3::TcpSocketFactory", sinkLocalAddress); // Packet receiver helper class, parameter 1: protocol; Parameter 2: Address
  ApplicationContainer sinkApps;
  for (uint32_t i = 0; i < d.RightCount (); ++i)
    sinkHelper.SetAttribute ("Protocol", TypeIdValue (TcpSocketFactory::GetTypeId ()));
    sinkApps.Add (sinkHelper.Install (d.GetRight (i)));
  sinkApps.Start (Seconds (0.0));
  sinkApps.Stop  (Seconds (stop_time));
  // Install sender application
  for (uint32_t i = 0; i < d.LeftCount (); ++i)
    // Create an on/off app sending packets to the left side
    AddressValue remoteAddress (InetSocketAddress (d.GetRightIpv4Address (i), port));
    Config::SetDefault ("ns3::TcpSocket::SegmentSize", UintegerValue (tcp_adu_size));
    BulkSendHelper ftp ("ns3::TcpSocketFactory", Address ());
    ftp.SetAttribute ("Remote", remoteAddress);
    ftp.SetAttribute ("SendSize", UintegerValue (tcp_adu_size));
    ftp.SetAttribute ("MaxBytes", UintegerValue (data_mbytes * 1000000));

    ApplicationContainer clientApp = ftp.Install (d.GetLeft (i));
    clientApp.Start (Seconds (start_time * i)); // Start after sink
    clientApp.Stop (Seconds (stop_time - 3)); // Stop before the sink

1.16 Flow monitor module

Function: collect the statistical information of each stream and export it in XML format.

Statistics include:
timeFirstTxPacket: the time when the first packet in the transport stream is transmitted
timeLastTxPacket: the time when the last packet in the stream is transmitted
timeFirstRxPacket: the time when the end node receives the first packet in the stream;
timeLastRxPacket: the time when the last packet in the stream was received
delaySum: the sum of all end-to-end delays of all received stream packets;
jitterSum: the sum of all end-to-end delay jitter (delay change) values of all received stream packets. Refer to document: rfc: 3393;
txBytes, txPackets: the total number of transmitted bytes / packets of the stream;
rxBytes, rxPackets: total number of received bytes / packets of the stream;
lostPackets: total number of packets lost (not reported for more than 10 seconds);
timesForwarded: reports the number of packets forwarded;
Delayhistogram, jitterhistogram, packetizehistogram: histograms of delay, jitter, and packet size
Packets dropped, bytesdropped: the number of lost packets and bytes, divided according to the loss reason code (defined in the probe).

  bool flow_monitor = false;
  cmd.AddValue ("flow_monitor", "Enable flow monitor", flow_monitor);

  // Flow monitor
  FlowMonitorHelper flowHelper; // Instantiate the traffic monitor helper class
  if (flow_monitor)
    flowHelper.InstallAll (); // Through the help program, install the monitor in the node (you can set the monitor properties and print Statistics)
  Simulator::Stop (Seconds (stop_time));
  Simulator::Run ();
  // Note: SerializeToXmlFile must be placed in Simulator::Run(); Then run; The last two parameters of SerializeToXmlFile are used for drawing, and can generally be set to false
  std::string prefix_file_name = "TcpVariantsComparison";
  cmd.AddValue ("prefix_name", "Prefix of output trace file", prefix_file_name);
  if (flow_monitor)
      flowHelper.SerializeToXmlFile (prefix_file_name + ".flowmonitor", true, true);  // Serialize the results and write them to a file. Parameter 1: file name; Parameter 2: if true, include Histograms histogram in the output; Parameter 3: whether to activate Probes probe. If true, include probe stream pair statistics in the output.

1.17 calculating the number of packets

  static std::vector<uint32_t> rxPkts;
  // Count RX packets
  for (uint32_t i = 0; i < d.RightCount (); ++i)
    rxPkts.push_back(0); // push_back() is to add a number at the end. That is, rePkts[0]=0
    Ptr<PacketSink> pktSink = DynamicCast<PacketSink>(sinkApps.Get(i));
    pktSink->TraceConnectWithoutContext ("Rx", MakeBoundCallback (&CountRxPkts, i));
  // If the TCP protocol used contains RL, close the openGym interface
  if (transport_prot.compare ("ns3::TcpRl") == 0 or transport_prot.compare ("ns3::TcpRlTimeBased") == 0)

  Simulator::Destroy ();
  return 0;

2 tcp-rl.h/cc

2.1 #ifndef

#ifndef TCP_RL_H
#define TCP_RL_H

#endif /* TCP_RL_H */

Function: prevent the header file from being included repeatedly.

2.2 header file

#include "ns3/tcp-congestion-ops.h"
#include "ns3/opengym-module.h"
#include "ns3/tcp-socket-base.h"

2.3 forward declaration of class

namespace ns3 {

class TcpSocketBase;
class Time;
class TcpGymEnv;

Class 2.4 TcpSocketDerived

// used to get pointer to Congestion Algorithm
class TcpSocketDerived : public TcpSocketBase
  static TypeId GetTypeId (void);
  virtual TypeId GetInstanceTypeId () const;

  TcpSocketDerived (void);
  virtual ~TcpSocketDerived (void);

  Ptr<TcpCongestionOps> GetCongestionControlAlgorithm (); // Returns a pointer to the CC algorithm

Type 2.5 TcpRlBase

class TcpRlBase : public TcpCongestionOps
   * \brief Get the type ID.
   * \return the object TypeId
  static TypeId GetTypeId (void);

  TcpRlBase ();

   * \brief Copy constructor.
   * \param sock object to copy.
  TcpRlBase (const TcpRlBase& sock);

  ~TcpRlBase ();

  virtual std::string GetName () const;
  virtual uint32_t GetSsThresh (Ptr<const TcpSocketState> tcb, uint32_t bytesInFlight);
  virtual void IncreaseWindow (Ptr<TcpSocketState> tcb, uint32_t segmentsAcked);
  virtual void PktsAcked (Ptr<TcpSocketState> tcb, uint32_t segmentsAcked, const Time& rtt);
  virtual void CongestionStateSet (Ptr<TcpSocketState> tcb, const TcpSocketState::TcpCongState_t newState);
  virtual void CwndEvent (Ptr<TcpSocketState> tcb, const TcpSocketState::TcpCAEvent_t event);
  virtual Ptr<TcpCongestionOps> Fork ();

  static uint64_t GenerateUuid ();
  virtual void CreateGymEnv();
  void ConnectSocketCallbacks();

  // OpenGymEnv interface
  Ptr<TcpSocketBase> m_tcpSocket;
  Ptr<TcpGymEnv> m_tcpGymEnv;

Type 2.6 TcpRl

class TcpRl : public TcpRlBase
  static TypeId GetTypeId (void);

  TcpRl ();
  TcpRl (const TcpRl& sock);
  ~TcpRl ();

  virtual std::string GetName () const;
  virtual void CreateGymEnv();
  // OpenGymEnv env
  float m_reward {1.0};
  float m_penalty {-100.0};

Class 2.7 TcpRlTimeBased

class TcpRlTimeBased : public TcpRlBase
  static TypeId GetTypeId (void);

  TcpRlTimeBased ();
  TcpRlTimeBased (const TcpRlTimeBased& sock);
  ~TcpRlTimeBased ();

  virtual std::string GetName () const;

  virtual void CreateGymEnv();
  // OpenGymEnv env
  Time m_timeStep {MilliSeconds (100)};

} // namespace ns3

3 tcp-rl-env.h/cc

#ifndef TCP_RL_ENV_H
#define TCP_RL_ENV_H

#include "ns3/opengym-module.h"
#include "ns3/tcp-socket-base.h"
#include <vector>

namespace ns3 {

class Packet;
class TcpHeader;
class TcpSocketBase;
class Time;

class TcpGymEnv : public OpenGymEnv ...
class TcpEventGymEnv : public TcpGymEnv ...
class TcpTimeStepGymEnv : public TcpGymEnv ...
} // namespace ns3

#endif /* TCP_RL_ENV_H */

4 tcp_base.py

4.1 class Tcp(object)

class Tcp(object):
    """docstring for Tcp"""
    def __init__(self):
        super(Tcp, self).__init__()

    def set_spaces(self, obs, act):
        self.obsSpace = obs
        self.actSpace = act

    def get_action(self, obs, reward, done, info):

  1. The subclass inherits the constructor of the parent class:
    The first parameter of the constructor is self. If you inherit the constructor of the parent class, use the super keyword:
  super(Subclass, self).__init__(Parameter 1, parameter 2...)

4.2 class TcpEventBased(Tcp)

Italic style

4.3 class TcpTimeBased(Tcp)

5 tcp_newreno.py

5.1 class TcpNewReno(TcpEventBased)

6 test_tcp.py

C + + and other compiled languages need to compile the program into binary before running. They need to include a main() main class as the program entry.
As a scripting language, Python runs dynamically line by line, that is, it runs from the first line without a unified entry.
Use if__ name__ == '__ main__':, This part of the statement will be executed when the file is used as the main file; If it is a module called by other files, it will not be executed.

import argparse
from ns3gym import ns3env
from tcp_base import TcpTimeBased
from tcp_newreno import TcpNewReno

__author__ = "Piotr Gawlowicz"
__copyright__ = "Copyright (c) 2018, Technische Universität Berlin"
__version__ = "0.1.0"
__email__ = "gawlowicz@tkn.tu-berlin.de"

parser = argparse.ArgumentParser(description='Start simulation script on/off')
                    help='Start ns-3 simulation script 0/1, Default: 1')
                    help='Number of iterations, Default: 1')
args = parser.parse_args()
startSim = bool(args.start)
iterationNum = int(args.iterations)

port = 5555
simTime = 10 # seconds
stepTime = 0.5  # seconds
seed = 12
simArgs = {"--duration": simTime,}
debug = False

env = ns3env.Ns3Env(port=port, stepTime=stepTime, startSim=startSim, simSeed=seed, simArgs=simArgs, debug=debug)
# simpler:
#env = ns3env.Ns3Env()

ob_space = env.observation_space
ac_space = env.action_space
print("Observation space: ", ob_space,  ob_space.dtype)
print("Action space: ", ac_space, ac_space.dtype)

stepIdx = 0
currIt = 0

def get_agent(obs):
    socketUuid = obs[0]
    tcpEnvType = obs[1]
    tcpAgent = get_agent.tcpAgents.get(socketUuid, None)
    if tcpAgent is None:
        if tcpEnvType == 0:
            # event-based = 0
            tcpAgent = TcpNewReno()
            # time-based = 1
            tcpAgent = TcpTimeBased()
        tcpAgent.set_spaces(get_agent.ob_space, get_agent.ac_space)
        get_agent.tcpAgents[socketUuid] = tcpAgent

    return tcpAgent

# initialize variable
get_agent.tcpAgents = {}
get_agent.ob_space = ob_space
get_agent.ac_space = ac_space

    while True:
        print("Start iteration: ", currIt)
        obs = env.reset()
        reward = 0
        done = False
        info = None
        print("Step: ", stepIdx)
        print("---obs: ", obs)

        # get existing agent of create new TCP agent if needed
        tcpAgent = get_agent(obs)

        while True:
            stepIdx += 1
            action = tcpAgent.get_action(obs, reward, done, info)
            print("---action: ", action)

            print("Step: ", stepIdx)
            obs, reward, done, info = env.step(action)
            print("---obs, reward, done, info: ", obs, reward, done, info)

            # get existing agent of create new TCP agent if needed
            tcpAgent = get_agent(obs)

            if done:
                stepIdx = 0
                if currIt + 1 < iterationNum:

        currIt += 1
        if currIt == iterationNum:

except KeyboardInterrupt:
    print("Ctrl-C -> Exit")

Topics: Python