Preface

Stepped on the pit for 5-6 hours, various tests. There's no one left. But I'm satisfied with the result.
This article is correct. https://segmentfault.com/a/11... Perfect and Preliminary BUG!

Dakeng 1 (scrapyd service bind_address):

I used Docker (I remember scrapyd's configuration file seemed to be the default 0.0.0 when I didn't use docker before)
But I found that the default in the Docker container was bind 127.0.0.1 (really f-k... I hadn't found it for hours).
If you use docker, first create a default_scrapyd.conf in the Docker peer directory
Then write the following configuration (without hesitation, copy it all in):

[scrapyd]
eggs_dir    = eggs
logs_dir    = logs
items_dir   =
jobs_to_keep = 5
dbs_dir     = dbs
max_proc    = 0
max_proc_per_cpu = 4
finished_to_keep = 100
poll_interval = 5.0
bind_address = 0.0.0.0
http_port   = 6800
debug       = off
runner      = scrapyd.runner
application = scrapyd.app.application
launcher    = scrapyd.launcher.Launcher
webroot     = scrapyd.website.Root

[services]
schedule.json     = scrapyd.webservice.Schedule
cancel.json       = scrapyd.webservice.Cancel
addversion.json   = scrapyd.webservice.AddVersion
listprojects.json = scrapyd.webservice.ListProjects
listversions.json = scrapyd.webservice.ListVersions
listspiders.json  = scrapyd.webservice.ListSpiders
delproject.json   = scrapyd.webservice.DeleteProject
delversion.json   = scrapyd.webservice.DeleteVersion
listjobs.json     = scrapyd.webservice.ListJobs
daemonstatus.json = scrapyd.webservice.DaemonStatus

Finally, write it at the end of your Docker file (depending on the situation). Everyone has a different directory structure.

COPY default_scrapyd.conf /usr/local/lib/python3.6/site-packages/scrapyd/default_scrapyd.conf

Dakeng 2 (docker-compose: command Multi-Command problem)

The scrapyd deployment requires two steps:

1. Start scrapyd service (scrapyd command is enough)
2. Then the crawler program is pushed to the scrapyd service through scrapyd-deploy

Obviously: 2 is dependent on 1.

Error resolution:

However, docker-compose command: Only one command can be run. I use N common sense ideas to support multiple commands:

1. Initial use&&
2. Using sh script
3. This order is the most absurd... The same blog on the internet. Eighty percent copied from each other. (Then I did use it.)

The most absurd command is as follows: (I have been delighted that I have found a solution, and have been debugging and testing on this command axis....)

#    command:
#      - /bin/bash 
#      - -c 
#      - |
#        scrapyd 
#        scrapyd-deploy Cython_lin -p Baidu

This command has two results (both results are also tm random). There is no love in life.

1. scrapyd is executed first and blocked directly. Later commands fail (even if you use & you can't solve the problem of random startup sequence)
2. Scrapyd-deploy Cython_lin-p Baidu executes first (direct error reporting, think scrapyd hasn't started yet)

So far:::::::::: I said above are the wrong way!!!!!!!!!!!!!!!
(Nearly strangled to death on a tree, since only one command can be run inside the docker-compose file. So let's run it outside!!!
(I have despair and rigidity of thought, this time I really see hope, dawn)

Correct solutions:

The docker-compose.yml file only says:

command: scrapyd

Then save the exit and execute:

docker-compose up -d
# We need to start scrapyd for a while before we can start scrapy-deploy (just wait a minute)

Then continue to implement:

docker-compose exec crawl  scrapyd-deploy Cython_lin -p Baidu


Special attention! Explain the following! (The docker-compose command has two ways:
    docker-compose exec is correct
    docker-compose run error (never use this outside (I mean my business, of course)

Concluding remarks

Maybe some mistakes are small, but when you doubt more, there will be a lot of problem-solving branches in your mind.
Then the mechanical row of BUG, and ultimately no love...
In fact, sometimes it's better to calm down first, then concentrate on it and finalize it with one hammer. Find out BUG!!!