preface
I have always wanted to be an unmanned car that can be monitored / operated remotely (deliver goods in the school to make money) (LE); Just because of my laziness, I didn't take action.
But my bike was stolen the other day; Angrily, I wrote a real-time pedestrian recognition with opencv. Raspberry sent a camera to identify the guy who rode my car in the dormitory parking shed.
But I encountered the first problem: online real-time monitoring.
Anyway, it's all real-time monitoring. The temporary conflict gave me motivation, and I took the first step in building a car.
(the vast majority of online blogs are sent or received as servers at the same time, isn't that ridiculous...)
Nonsense.
The above is the framework of car building technology I conceived a long time ago; At that time, limited to the narrow vision and the lack of knowledge reserve, I thought I could do the upper level control at most, so I was crazy about building a car.
It was not until I had a fever in my head and participated in all kinds of meaningless competitions (and almost ended in failure) that I found that I had already clicked on these technical points and was poor in improvement.
Looking back now, it seems that the meaning of those things learned for the competition is not so unclear, but there is still a sense of achievement
If you don't want to see me talk nonsense, you can jump right here
So what we want to achieve today is the communication between clients through the server:
The framework used is Django, and sending video is implemented in Python (easy to combine with ROS); Make do with HTML for client display first. It's too tired to write Qt or Swift
Although there is built-in DDS communication in ROS2, I haven't used this thing and don't understand how to communicate in the WAN, so I chose to build a Django server repeatedly.
Server
In order to realize two-way communication, I choose websocket; But the pain is that Django 3.0 doesn't support websocket when it goes up, so here I implement websocket through channels.
You can first run the official tutorial demo of channels: https://channels.readthedocs.io/en/latest/tutorial/part_1.html
I only wrote the first three tutorial, and my code is changed on this basis; So I won't start from 0 (I'll talk about it later when I have time);
And this demo realizes text transmission (chat room), which can realize command and status transmission without slight modification? How nice
Next, I default that you have finished the demo; Next, my python versions are all 3.8.
step
First, create an app again. I name it video:
python manage.py startapp video
And in MySite / setting Register in py:
INSTALLED_APPS = [
'channels',
'chat',
'video',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
]
At the same time in video / URLs Py and MySite / URLs Add route to py:
# mysite/urls.py
from django.conf.urls import include
from django.urls import path
from django.contrib import admin
urlpatterns = [
path('chat/', include('chat.urls')),
path('video/', include('video.urls')),
path('admin/', admin.site.urls),
]
# video/urls.py
from django.urls import path
from . import views
urlpatterns = [
path('', views.index, name='index'),
path('<str:v_name>/', views.v_name, name='v_name'),
]
Edit video / views py:
# video/views.py
from django.shortcuts import render
# Create your views here.
def index(request):
return render(request, 'video/index.html')
def v_name(request, v_name):
return render(request, 'video/video.html', {
'v_name': v_name
})
Add video / consumers py:
# video/consumers.py
import json
from channels.generic.websocket import AsyncWebsocketConsumer
class VideoConsumer(AsyncWebsocketConsumer):
async def connect(self):
self.room_name = self.scope['url_route']['kwargs']['v_name']
self.room_group_name = 'video_%s' % self.room_name
#print(self.room_name)
# Join room group
await self.channel_layer.group_add(
self.room_group_name,
self.channel_name
)
await self.accept()
async def disconnect(self, close_code):
# Leave room group
await self.channel_layer.group_discard(
self.room_group_name,
self.channel_name
)
# Receive message from WebSocket
async def receive(self, text_data):
# print(1)
# Send message to room group
await self.channel_layer.group_send(
self.room_group_name,
{
'type': 'video_message',
'message': text_data,
}
)
# Receive message from room group
async def video_message(self, event):
# print(1)
message = event['message']
# Send message to WebSocket
await self.send(text_data=json.dumps({
'message': message
}))
We'll talk about this document later. In fact, this file is basically a copy of chat / consumers Py, you should be able to see.
Then we create routing in mysite application py:
# chat/routing.py from django.urls import re_path import chat.consumers import video.consumers websocket_urlpatterns = [ re_path(r'chat/(?P<room_name>\w+)/$', chat.consumers.ChatConsumer.as_asgi()), re_path(r'video/(?P<v_name>\w+)/$', video.consumers.VideoConsumer.as_asgi()) ]
Also modify asgi py:
# mysite/asgi.py
import os
from channels.routing import ProtocolTypeRouter, URLRouter
from django.core.asgi import get_asgi_application
from channels.auth import AuthMiddlewareStack
# import chat.routing
# import video.routing
from . import routing
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'mysite.settings')
application = ProtocolTypeRouter({
"http": get_asgi_application(),
"websocket": AuthMiddlewareStack(
URLRouter(
routing.websocket_urlpatterns
)
),
})
This is done so that chat and video can work at the same time (transmitting video and command / status at the same time).
explain
Here I only explain video / consumers The rest of the operation principle of the code I modified in py is the same as that of chat.
See the receive() function:
async def receive(self, text_data):
await self.channel_layer.group_send(
self.room_group_name,
{
'type': 'video_message',
'message': text_data,
}
)
During transmission, I disassemble the video screen into pictures and encode the pictures into base64, so the received value here is text_data.
Sender
send_video.py:
# send_video.py
import asyncio
import websockets
import numpy as np
import json
import cv2
import base64
import time
capture = cv2.VideoCapture(0)
if not capture.isOpened():
print('quit')
quit()
ret, frame = capture.read()
encode_param=[int(cv2.IMWRITE_JPEG_QUALITY),95]
# Send video screenshots to the server in real time
async def send_video(websocket):
global ret, frame
# global cam
while True:
time.sleep(0.1)
result, imgencode = cv2.imencode('.jpg', frame, encode_param)
data = np.array(imgencode)
img = data.tobytes()
# base64 encoded transmission
img = base64.b64encode(img).decode()
await websocket.send("data:image/jpg;base64,"+ img)
ret, frame = capture.read()
async def main_logic():
async with websockets.connect('ws://127.0.0.1:8000/ws/video/wms/') as websocket:
await send_video(websocket)
asyncio.get_event_loop().run_until_complete(main_logic())
Read the camera through cv2 and transcode it; Here I default to the fixed connection ws://127.0.0.1:8000/ws/video/wms /, which is convenient for debugging.
receiving end
video.html:
<!--video.html-->
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Video</title>
</head>
<body>
<div>
<h1>Video</h1>
<div>
<div>
<img id="resImg" src="" />
</div>
<script src="http://apps.bdimg.com/libs/jquery/2.1.1/jquery.min.js" ></script>
<script>
const ws = new WebSocket(
'ws://'
+ window.location.host
+ '/ws/video/'
+ 'wms'
+ '/'
);
ws.onmessage = function(evt) {
v_data = JSON.parse(evt.data);
$("#resImg").attr("src", v_data.message);
//console.log( "Received Message: " + v_data.message);
// ws.close();
};
ws.onclose = function(evt) {
console.log("Connection closed.");
};
</script>
</body>
</html>
The receiver parses json data and decodes jpg image in reverse and displays it; After that, I may implement it with Swift (Qt I can't write well and runs too slowly).
function
Run in Django project:
python manage.py runserver
Then run the sender send_video.py:
python send_video.py
Finally, open video html: 
YADAZE
After the local test is completed, it is almost ready for deployment.
It's more than four o'clock again