1, Introduction of ORTP
1. Review of the content of the previous two quarters
(1) Build the SDK development environment, compile the system, deploy and test the environment
(2) Explain the sample source code, compile and run, test video, and verify the hardware
2. Two ways of video network transmission
(1) Based on download: http or ftp (download and play again. Poor network speed will affect viewing. For example, when we watch movies, the network is usually poor to display buffer)
HTTP, Hyper Text Transfer Protocol * * is a simple request response protocol, which usually runs on TCP. It specifies what messages the client may send to the server and what response it will get. HTTP is an application layer protocol and communicates based on B/S architecture. The server-side implementation programs of HTTP include httpd, nginx, etc. the client-side implementation programs are mainly Web browsers, such as Firefox, Internet Explorer, Google Chrome, Safari, Opera, etc
FTP, file transfer protocol, is a set of standard protocols for file transmission on the network. It works in the seventh layer of OSI model and the fourth layer of TCP model, that is, the application layer. TCP transmission is used instead of UDP. The client needs to go through a "three-time handshake" before establishing a connection with the server The process ensures that the connection between the client and the server is reliable and connection oriented, providing a reliable guarantee for data transmission.
FTP allows users to operate files (such as file addition, deletion, modification, query, transmission, etc.) Communicate with another host. However, users do not really log in to the computer they want to access and become full users. FTP programs can be used to access remote resources to realize users' round-trip transmission of files, directory management, access to e-mail, etc., even though computers on both sides may be equipped with different operating systems and file storage methods. [1]
(2) Based on real-time: RTP/RTSP/RTCP (if the network speed is not good, the picture quality is sacrificed to realize real-time transmission, and the monitoring generally needs real-time)
RTP (real time Transport Protocol) RTP generally refers to Real-time Transport Protocol. Real time transport protocol (RTP) is a network transport protocol, which was published in RFC 1889 by the multimedia transmission working group of IETF in 1996.
RTSP (Real Time Streaming Protocol), RFC2326, real-time streaming protocol, is an application layer protocol in TCP/IP protocol system
RTCP (real time transport control protocol or RTP Control Protocol or RTCP for short) is a sister protocol of real-time transport protocol (RTP). RTCP is defined by RFC 3550 (replacing obsolete RFC 1889) . RTP uses an even UDP port; RTCP uses the next port of RTP, which is an odd port. RTCP works together with RTP to transmit actual data, and RTCP is responsible for sending control package to everyone in the phone. Its main function is to provide feedback on the quality of service that RTP is providing.
RTSP initiates / terminates streaming media, RTP transmits streaming media data, and RTCP controls and synchronizes RTP.
Detailed explanation: https://blog.csdn.net/frankiewang008/article/details/7665547/
3. Introduction to ORTP
(1)openRTP, an RTP library implemented in C (actually implemented in C + +, JAVA, etc.)
(2) In essence, it is a video server. When working, the client and server transmit video data in real time. UDP protocol is used. Because TCP is based on connection, it needs three handshakes to compare the consumption performance. It is not very good in real-time transmission.
(3) It is generally believed that RTP works in the transport layer, but in fact, RTP is one level higher than TCP/UDP
(4) The implementation of RTP (and RTCP) is stipulated in the international standard RFC3550. Anyone can write one as long as it complies with the protocol
(5) This season's article focuses on using ORTP to realize LAN video real-time transmission
2, Porting of ORTP Library
1. Prepare source code
(1) Download the ortp source code: https://github.com/dmonakhov/ortp
I will also provide you with my own good online disk link: Link: https://pan.baidu.com/s/1UML_evRZL6D8JV8aoW1UqA Extraction code: ckko --From Baidu online disk super member V5 Sharing
(2) Store it in the temporary working directory in ubuntu and unzip it, for example, under ~ / sambashare /
(3) Unzip the source code and enter the source code directory
unzip ortp-master.zip
2. Source code modification
(1) Add payload support for H.264.
stay src/avprofile.c Add in line 357: rtp_profile_set_payload(profile,96,&payload_type_h264); So as to increase the h.264 Support here payload The bound number is a standard, h.264 The corresponding is 96 payload: load
3. Configuration and compilation, installation
(1)get into ortp Directory execution./autogen.sh (2)Error 1:./autogen.sh: line 44: libtoolize: command not found solve: sudo apt-get install libtool* (2)Error 2: libtoolize: error: Please install GNU M4, or 'export M4=/path/to/gnu/m4'. solve: sudo apt-get install m4 (3)Error 3: Automake - aclocal: command not found solve: sudo apt-get install automake (4)Continue execution ./autogen.sh ./configure --prefix=/tmp/ortp --host=arm-hisiv300-linux make && make install
4. Go to the / tmp/ortp directory to view the migrated library and header files
3, RTP video transmission practice
1. Add rtp transmission code in the sample of the official SDK
(1) In Venc / sample_venc. C, add: s32ChnNum = 1;
Three channels change to one. We only send one channel of data.
(2) In common / sample_common_venc. C, we need to modify many things. I will provide you with the modified file:
To understand the following codes, refer to learning (required):
https://blog.csdn.net/tq08g2z/article/details/77528335
https://blog.csdn.net/zhang_danf/article/details/51037572
#define ORTP_ENABLE 1
#if ORTP_ENABLE
#include <ortp/ortp.h>
#include <signal.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/time.h>
#include <stdio.h>
#define Y_PLOAD_TYPE 96 //H.264
#define MAX_RTP_PKT_LENGTH 1400
#define DefaultTimestampIncrement 3600 //(90000/25)
uint32_t g_userts=0;
RtpSession *pRtpSession = NULL;
#Define local_host_ip "192.168.1.20" / / set according to your computer IP
/** initialization
*
* It is mainly used to initialize ortp and other parameters
* @param: char * ipStr Destination IP address description string
* @param: int port Destination RTP listening port
* @return: RtpSession * Returns a pointer to the RtpSession object. If it is NULL, initialization fails
* @note:
*/
RtpSession * rtpInit( char * ipStr, int port)
{
RtpSession *session;
char *ssrc;
printf("********oRTP for H.264 Init********\n");
ortp_init();
ortp_scheduler_init();
ortp_set_log_level_mask(ORTP_MESSAGE|ORTP_WARNING|ORTP_ERROR);
session=rtp_session_new(RTP_SESSION_SENDONLY);//Create session
rtp_session_set_scheduling_mode(session,1);
rtp_session_set_blocking_mode(session,0);
//rtp_session_set_connected_mode(session,TRUE);
rtp_session_set_remote_addr(session,ipStr,port);
rtp_session_set_payload_type(session,Y_PLOAD_TYPE);
ssrc=getenv("SSRC");//Get environment variables
if (ssrc!=NULL) {//There are many session transmissions in a line. Each session is distinguished by adding source flag bits
printf("using SSRC=%i.\n",atoi(ssrc));
// Set the SSRC of the output stream. If this step is not done, a random value will be given
rtp_session_set_ssrc(session,atoi(ssrc));
}
return session;
}
/** End the sending of ortp and release resources
*
* @param: RtpSession *session RTP Pointer to the session object
* @return: 0 Indicates success
* @note:
*/
int rtpExit(RtpSession *session)
{
printf("********oRTP for H.264 Exit********\n");
g_userts = 0;
rtp_session_destroy(session);
ortp_exit();
ortp_global_stats_display();
return 0;
}
/** Send rtp packet
*
* It is mainly used to send rtp packets
* @param: RtpSession *session RTP Pointer to the session object
* @param: const char *buffer The buffer address of the data to be sent
* @param: int len Length of data to send
* @return: int Number of packets actually sent
* @note: If the length of the data packet to be sent is greater than BYTES_PER_COUNT, subcontracting will be performed inside this function
*/
int rtpSend(RtpSession *session, char *buffer, int len)
{
int sendBytes = 0;
int status;
uint32_t valid_len=len-4;
unsigned char NALU=buffer[4];
//If the data is less than MAX_RTP_PKT_LENGTH bytes, send directly: single NAL unit mode
if(valid_len <= MAX_RTP_PKT_LENGTH)
{
sendBytes = rtp_session_send_with_ts(session,
&buffer[4],
valid_len,
g_userts);
}
else if (valid_len > MAX_RTP_PKT_LENGTH)
{
//It is divided into many packets and sent. The header should be processed before each packet, such as the first packet
valid_len -= 1;
int k=0,l=0;
k=valid_len/MAX_RTP_PKT_LENGTH;
l=valid_len%MAX_RTP_PKT_LENGTH;
int t=0;
int pos=5;
if(l!=0)
{
k=k+1;
}
while(t<k)//||(t==k&&l>0))
{
if(t<(k-1))//(t<k&&l!=0)||(t<(k-1))&&(l==0))//(0==t)||(t<k&&0!=l))
{
buffer[pos-2]=(NALU & 0x60)|28;
buffer[pos-1]=(NALU & 0x1f);
if(0==t)
{
buffer[pos-1]|=0x80;
}
sendBytes = rtp_session_send_with_ts(session,
&buffer[pos-2],
MAX_RTP_PKT_LENGTH+2,
g_userts);
t++;
pos+=MAX_RTP_PKT_LENGTH;
}
else //if((k==t&&l>0)||((t==k-1)&&l==0))
{
int iSendLen;
if(l>0)
{
iSendLen=valid_len-t*MAX_RTP_PKT_LENGTH;
}
else
iSendLen=MAX_RTP_PKT_LENGTH;
buffer[pos-2]=(NALU & 0x60)|28;
buffer[pos-1]=(NALU & 0x1f);
buffer[pos-1]|=0x40;
sendBytes = rtp_session_send_with_ts(session,
&buffer[pos-2],
iSendLen+2,
g_userts);
t++;
}
}
}
g_userts += DefaultTimestampIncrement;//timestamp increase
return len;
}
#endif
/******************************************************************************
* funciton : save H264 stream
******************************************************************************/
HI_S32 SAMPLE_COMM_VENC_SaveH264(FILE* fpH264File, VENC_STREAM_S *pstStream)
{
HI_S32 i;
for (i = 0; i < pstStream->u32PackCount; i++)
{
#if ORTP_ENABLE
rtpSend(pRtpSession,pstStream->pstPack[i].pu8Addr, pstStream->pstPack[i].u32Len);
#else
fwrite(pstStream->pstPack[i].pu8Addr+pstStream->pstPack[i].u32Offset,
pstStream->pstPack[i].u32Len-pstStream->pstPack[i].u32Offset, 1, fpH264File);
fflush(fpH264File);
#endif
}
return HI_SUCCESS;
}
#if ORTP_ENABLE
/***rtp init****/
pRtpSession = rtpInit( LOCAL_HOST_IP ,8080);
if (pRtpSession==NULL)
{
printf( "error rtpInit" );
exit(-1);
return 0;
}
#endif
Modified file:
Link: https://pan.baidu.com/s/1FNZ09-E2ERsjIVDmNxqOiw
Extraction code: dveu
– sharing from Baidu online disk super member V5
2. Recompile sample
(1) Copy the ortp header file because the code added above uses the source code in ortp. Copy the folder in the include directory generated by ortp compilation to the Hi3518E_SDK_V1.0.3.0/mpp/include/ directory
cp /tmp/ortp/include/ortp ~/sambashare/Hi3518E_SDK_V1.0.3.0/mpp/include/ -rf
(2) Modify Makefile in venc and add link support for libortp
(3) Compile with make
3. Deploy and run tests in the development board
(1) When the development board is powered on and mounted to the host through nfs, deploy libortp.so to the / usr/lib directory in the development board
(2) Check whether the original configuration of the development board is correct, such as whether the sensor corresponds to the actual configuration
(3) Get the new sample program test run through nfs mount
(4) Open the configured sdp file in VLC and see the real-time image to prove that the whole experiment is completed. The following is the link to this file:
Link: https://pan.baidu.com/s/1VRuPLWw4r8MWhGgTWGWSJw
Extraction code: gqhy
– sharing from Baidu online disk super member V5
m=video 8080 RTP/AVP 96 a=rtpmap:96 H264 a=framerate:25 c=IN IP4 192.168.1.20
Note: most of this material is compiled from the course notes of Mr. Zhu's Internet of things lecture hall, and some other people's blogs are cited. If there is infringement, please contact to delete it! The level is limited. If there are errors, you are welcome to communicate in the comment area.