Continuous integration
unit testing
karma
KarmayesNode.jsOfJavaScriptTest Execution Process Management Tool(Test Runner). The tool can be used to test all mainstreamsWebBrowsers can also be integrated intoCI(Continuous integration)Tools can also be used with other code editors.Interface-free browsers we tested
phantomjs. Test framework usagemochaandchai.The following are the main configuration information used in our project:
/** * Test Start Browser * Available browsers: https://npmjs.org/browse/keyword/karma-launcher */ browsers: ['PhantomJS'], /** * Test framework * Available framework: https://npmjs.org/browse/keyword/karma-adapter */ frameworks: ['mocha', 'chai'], /** * List of files that need to be loaded into the browser */ files: [ '../../src/dcv/plugins/jquery/jquery-1.8.1.min.js', '../../src/dcv/plugins/common/mock.min.js', '../../src/dcv/plugins/common/bluebird.min.js', '../../src/dcv/javascripts/uinv.js', '../../src/dcv/javascripts/uinv_util.js', '../../src/dcv/javascripts/browser/uinv_browser.js', 'specs/validators.js' ], /** * Excluded File List */ exclude: [ ], /** * Processing matching files before browsers are used * Available preprocessing: https://npmjs.org/browse/keyword/karma-preprocessor */ preprocessors: { //Report coverage "../../src/dcv/javascripts/**/*.js": ["coverage"] }, /** * Reporter using test results * Possible values:'dots','progress' * Available Reporters: https://npmjs.org/browse/keyword/karma-reporter */ reporters: ['spec', 'coverage'], /** * The type and directory of the report output when reporting is "coverage" */ coverageReporter: { type: 'html', dir: 'coverage/' }, /** * Service port number */ port: 9876, /** * Enable or disable colors in output reports or logs */ colors: true, /** * Log Level * Possible values: * config.LOG_DISABLE //No output information * config.LOG_ERROR //Output error information only * config.LOG_WARN //Output only warning information * config.LOG_INFO //Output all information * config.LOG_DEBUG //Output debugging information */ logLevel: config.LOG_INFO, /** * Enable or disable automatic detection of file changes for testing */ autoWatch: true, /** * Open or disable continuous integration mode * Setting it to true, Karma will open the browser, execute the test, and finally exit */ // singleRun: true, /** * Concurrency level (number of browsers started) */ concurrency: Infinity
Configuration in package.json is as follows:
"scripts": { "unit": "./node_modules/.bin/karma start test/unit/karma.conf.js --single-run" }
-- single-run means a single test run, which overrides the singleRun configuration item above. The html format report of test coverage will eventually be generated in the test/unit/coverage directory.
mocha
mocha is a unit testing framework for JavaScript, which can run in both browser and Node.js environments.
With mocha, we just need to focus on writing the unit test itself, and then let Mocha run all the tests automatically and give the test results.
The main characteristics of mocha are:
- You can test both simple JavaScript functions and asynchronous code, because asynchrony is one of the characteristics of JavaScript.
- It can run all tests automatically or only specific tests.
- You can support before, after, before and after Each to write initialization code.
describe denotes a test suite, which is a test of a sequence of related programs; it denotes unit test, which is the smallest unit of test. Example:
describe("Example", function () { it("deep usage", function () { expect({a: 1}).to.deep.equal({a: 1}); expect({a: 1}).to.not.equal({a: 1}); expect([{a: 1}]).to.deep.include({a: 1}); // expect([{a: 1}]).to.not.include({a: 1}); expect([{a: 1}]).to.be.include({a: 1}); }); });
There are four life hooks in mocha
before(): Execute before all test cases in this block
after(): executes after all test cases in the block
beforeEach(): Execute before each unit test
AfterEvery (): Executed after each unit test
Descri. skip allows you to skip tests without annotating large blocks of code; asynchronism requires only adding a done callback to the function. Example:
describe.skip('asynchronous beforeEach Example', function () { var foo = false; beforeEach(function (done) { setTimeout(function () { foo = true; done(); }, 50); }); it('Asynchronous modification of global variables should succeed', function () { expect(foo).to.be.equal(true); }); it('read book async', function (done) { book.read((err, result) => { expect(err).equal(null); expect(result).to.be.a('string'); done(); }) }); });
chai
chai is an assertion library, which can be understood as a comparison function, that is, whether the assertion function is consistent with expectations, if it is consistent, it means that the test passes, if it is not consistent, it means that the test fails.
mocha itself does not contain assertion library, so it is necessary to introduce third-party assertion library. At present, there are should.js, expect.js, chai, which are popular assertion library. Specific grammar rules need to be consulted.
Because chai has three styles: should, expect and assert, it has strong extensibility. The essence is the same, according to personal habits. See for details api
Here's a brief introduction to that style
should cases:
let num = 4+5 num.should.equal(9); num.should.not.equal(10); //boolean 'ok'.should.to.be.ok; false.should.to.not.be.ok; //type 'test'.should.to.be.a('string'); ({ foo: 'bar' }).should.to.be.an('object');
expect cases:
// equal or no equal let num = 4+5 expect(num).equal(9); expect(num).not.equal(10); //boolean expect('ok').to.be.ok; expect(false).to.not.be.ok; //type expect('test').to.be.a('string'); expect({ foo: 'bar' }).to.be.an('object');
assert example:
// equal or no equal let num = 4+5 assert.equal(num,9); //type assert.typeOf('test', 'string', 'test is a string');
End-to-end testing (e2e)
e2e(end to end) testing refers to end-to-end testing, also known as functional testing. From the user's point of view, using various functions and interactions is the simulation of users'real use scenarios.
At present, in the high-speed iteration of products, an automated test is an important guarantee for reconfiguration and iteration. For the web front-end, the main test is whether the form, animation, page jump, dom rendering, Ajax and so on are as expected.
e2e test is the highest level test to ensure the function, not focusing on the details of code implementation, focusing on whether the code can achieve the corresponding function. For our developers, the main focus of testing is whether the logic mapped to pages (usually stored variables) is correct.
We use nigthwatch for e2e testing
nightwatch
nightwatch is an e2e automated testing framework written by node JS of selenium or webdriver or phantomjs, which can easily write test cases to imitate user's operation to realize automated verification function.
The use of nightwatch is very simple. A nightwatch.json or nightwatch.config.js (the latter has a high priority) configuration file, using runner will automatically find the two files at the same level to obtain configuration information. You can also manually use -- config to determine the relative path of the configuration file.
selenium
Selenium is a powerful browser test platform, which supports the simulation test of firefox, chrome, edge and other browsers. The principle of selenium is to embed its JavaScript files into the web page when opening the browser. Then selenium's page is embedded into the target page through frame. In this way, you can use selenium's JavaScript object to control the target page.
The main configuration of nightwatch.config.js in the project is as follows:
{ "src_folders": ["test/e2e/specs"],//The folder where the test code is located "output_folder": "test/e2e/reports",//The test report is in the folder "globals_path": "test/e2e/global.js",//The folder where the global variables are located can be obtained by browser.globals.XX "custom_commands_path": ["node_modules/nightwatch-helpers/commands"],//Custom Extension Command "custom_assertions_path": ["node_modules/nightwatch-helpers/assertions"],//Custom extended assertions "selenium": { "start_process": true, "server_path": seleniumServer.path,//selenium's service address is usually a jar package "host": "127.0.0.1", "port": 4444, "cli_args": { "webdriver.chrome.driver": chromedriver.path,//Google Browser's drvier address is an exe file under windows "webdriver.firefox.profile": "", "webdriver.ie.driver": "", "webdriver.phantomjs.driver": phantomjsDriver.path } }, "test_settings": { "phantomjs": { "desiredCapabilities": { "browserName": "phantomjs", "marionette": true, "acceptSslCerts": true, "phantomjs.binary.path": phantomjsDriver.path, "phantomjs.cli.args": ["--ignore-ssl-errors=false"] } }, "chrome": { "desiredCapabilities": { "browserName": "chrome", "javascriptEnabled": true, "acceptSslCerts": true, 'chromeOptions': { 'args': [ // "start-fullscreen" // '- headless', // Open no interface // '--disable-gpu' ] } } }, "firefox": { "desiredCapabilities": { "browserName": "firefox", "javascriptEnabled": true, "acceptSslCerts": true } }, "ie": { "desiredCapabilities": { "browserName": "internet explorer", "javascriptEnabled": true, "acceptSslCerts": true } } } }
Configuration in package.json is as follows:
"scripts": { "e2e_ci": "node test/e2e/runner.js --env phantomjs", "e2e_parallel": "node test/e2e/runner.js --env phantomjs,chrome" }
The above two commands are executed in the runner.js file. The former configures an environment variable phantomjs, which will find phantomjs in test_settings; the latter is executed concurrently and tested with phantomjs and chrome browsers.
Test code
Any js file under the src_folders folder mentioned above will be considered test code and will execute the test. There are several ways to skip tests:
- @ disabled, so that the entire file skips the test
- @ tags tags, multiple files can tag the same tags. You can add -- tag manager to the command line so that only js files labeled manager will be tested, and others will be skipped.
- If you just want to skip a test method for the current file, you can convert function to a string, such as
module.exports = { 'step1': function (browser) { }, 'step2': "" + function (browser) { } }
Here is an example of a project that covers almost all kinds of operations. See for details http://nightwatchjs.org/api
var path = require("path"); module.exports = { //'@disabled': true, // does not execute this test module '@tags': ["manager"],//Label 'test manager': function (browser) { const batchFile = browser.globals.batchFile; const url = browser.globals.managerURL; browser .url(url) .getCookie("token", function (result) { if (result) { // browser.deleteCookie("token"); } else { this .waitForElementVisible('#loginCode', 50) .setValue('#loginCode', browser.globals.userName) .setValue("#loginPwd", browser.globals.password) .element("css selector", "#mntCode", function (res) {// Determine whether there are multiple tenants if (res.status != -1) { browser .click("#mntCode", function () { browser .assert.cssProperty("#MntList","display","block"// Show a multi-tenant list .assert.elementPresent("#mntList li[value=uinnova]"); }) .pause(500) .moveToElement("#mntList li[value=uinnova]", 0, 0, function () {// Move the mouse cursor to Youfu browser.click("#mntList li[value=uinnova]", function () { browser.assert.containsText("#mntCode, Superior Relief Technology; }); }); } }) .click("#fm-login-submit") .pause(50) .url(function (res) { if (res.value !== url) { //This command can be used for screenshots browser.saveScreenshot(browser.globals.imagePath + "login.png"); } }) .assert.urlContains(url, "Judge whether the jump is successful or not, otherwise the landing will fail."); .execute(function (param) { //Here you can execute the code in the page and get the parameters passed later. try { return uinv.data3("token"); } catch (e) { } }, ["param1"], function (res) { //Here you can get the return value of the above method }); } }) .maximizeWindow() //window maximizing .waitForElementVisible("#app", 1000) .pause(1000) .elements("css selector", ".data .clear li", function (res) { var nums = res.value.length - 1; //Get the number of scenarios in the manage.html page browser.expect.element('.data_num').text.to.equal('(' + nums + ')'); // Is the value in the sapn tag used to count the number of scenarios equal to the actual number of scenarios browser.pause(500); }) .click(".clear .last .add_data") .waitForElementPresent("#dcControlFrame") .frame("dcControlFrame", function () { //To locate iframe on the page, you need to fill in the id of iframe (no need to add #) browser .waitForElementPresent("#dataCenterId") .saveScreenshot(browser.globals.imagePath + "dcControlFrame.png") .setValue("#dataCenterId", browser.globals.sceneId) .setValue("#dataCenterName", browser.globals.sceneName) .setValue("#Data Center Text, Welcome .setValue("#Up_picture [type='file'], path. resolve (batchFile +'/ color. png')// Upload picture .click(".group-btn .save", function () { browser .pause(1000) .click(".layui-layer-btn0"); }) .waitForElementVisible("#dataCenterMenu3", 1000) .pause(1500) //Upload Scene .click("#dataCenterMenu3", function () { browser .setValue("#Img-3d-max-model input [type='file'], path. resolve (batchFile +'/20121115 uinnova DEMO.zip')// upload scenario file .waitForElementVisible(".layui-layer-btn0", 20000, function () { browser .click(".layui-layer-btn0"); }) .setValue("#Img-3d-max-layout input [type='file'], path.resolve (batchFile +'/DEMO 20140424-2016-01-14-17-48-17.js')//upload layout file .waitForElementVisible(".layui-layer-btn0", 5000, function () { browser .click(".layui-layer-btn0"); }); }) .pause(500) .saveScreenshot(browser.globals.imagePath + "frameParentBefore.png"); }) // FrameeParent ()// Back to the parent page of iframe; / / TODO has no interface, frame exit is problematic, so refresh is temporarily used to refresh the page. .refresh() .end(); } };
The following is a summary of XX students'use
- In some cases, pause is necessary, such as uploading pictures in the form operation, and clicking the Save button after the file upload is successful.
- Then the first one says that pause must pass in a fixed time millisecond value, which is too time-consuming and too small to be executed, requiring repeated testing. If you can, you can use the waitForElementVisible class's method, and it's okay to set it for a longer time.
- The return value in the callback function of the command method will be an object. Print out the object first to see the format, and then use the object.
- All assert s and commands end up with an optional parameter, command-line prompts when a custom test passes
appendix
phantomjs
Phantom JS is a JavaScript API based on webkit. It uses QtWebKit as its core browser function and WebKit to compile, interpret and execute JavaScript code. Anything you can do in a WebKit browser can be done. It is not only an invisible browser, but also provides such operations as CSS selector, supporting Web standards, DOM operations, JSON, HTML5, Canvas, SVG, etc. It also provides operations to process file I/O, so that you can read and write files to the operating system. Phantom JS has a wide range of applications, such as network monitoring, web screenshots, web testing without browsers, page access automation, etc.
Because phantomjs itself is not a node JS library, we are actually using the phantomjs-prebuilt package, which will download the driver package from the phantomjs official website according to the current operating system judgment.
Unfortunately, Vitaly Slobodin, one of the core developers of Phantom JS, recently announced that he had resigned as maintainer and no longer maintained the project.
According to Vitaly, Chrome 59 will support headless mode, and users will eventually turn to it. Chrome is faster and more stable than Phantom JS, and it won't eat memory like Phantom JS.
"I can't see the future of Phantom JS. It's like a bloody hell to develop Phantom JS 2 and 2.5 as a separate developer. Even though the recently released version of 2.5 Beta has a brand new and shining QtWebKit, I still can't really support the three platforms. We are not supported by other forces!"
With Vitaly's exit, only two core developers are left to maintain the project.
As mentioned above, the project has not been supported by resources, and it is very difficult to maintain such a large project even if two people are on duty.
defect
- Although Phantom.js is fully functional headless browser, it is quite different from real browsers and can not fully simulate real user operations. Many times, we found some problems in Phantom. js, but after debugging for half a day, we found that it was Phantom. js'own problem.
- Nearly 2k of the issue still needs to be repaired.
- Javascript's inherent single-threaded weakness requires asynchronous simulation of multithreading, and the resulting callback hell is painful for novices, but with the wide application of es6, we can use promise to solve the problem of multiple nested callback functions.
- Although webdriver supports htmlunit and phantomjs, because there is no interface, when we need to debug or reproduce problems, it is very troublesome.
Puppeteer
Puppeteer is a Node library officially produced by Google that controls headless Chrome through the DevTools protocol. You can use Puppeteer's api to directly control Chrome to simulate most user actions for UI Test s or to access pages as crawlers to collect data. A high-level api similar to webdriver helps us control interface-free Chrome through the DevTools protocol.
Before puppeteer, we need to use chrome-remote-interface to control chrome headless, but it is closer to low-level implementation than the Puppeteer API, and is more complex to read and write than puppeteer. There is no specific dom operation, especially if we want to simulate click events, input events and so on, it seems that we are unable to do so.
Let's compare the two libraries with the same two pieces of code.
Let's start with chrome-remote-interface
const chromeLauncher = require('chrome-launcher'); const CDP = require('chrome-remote-interface'); const fs = require('fs'); function launchChrome(headless=true) { return chromeLauncher.launch({ // port: 9222, // Uncomment to force a specific port of your choice. chromeFlags: [ '--window-size=412,732', '--disable-gpu', headless ? '--headless' : '' ] }); } (async function() { const chrome = await launchChrome(); const protocol = await CDP({port: chrome.port}); const {Page, Runtime} = protocol; await Promise.all([Page.enable(), Runtime.enable()]); Page.navigate({url: 'https://www.github.com/'}); await Page.loadEventFired( console.log("start") ); const {data} = await Page.captureScreenshot(); fs.writeFileSync('example.png', Buffer.from(data, 'base64')); // Wait for window.onload before doing stuff. protocol.close(); chrome.kill(); // Kill Chrome.
Let's look at puppeteer again.
const puppeteer = require('puppeteer'); (async () => { const browser = await puppeteer.launch(); const page = await browser.newPage(); await page.goto('https://www.github.com'); await page.screenshot({path: 'example.png'}); await browser.close(); })();
It's so short and concise that it's closer to natural language. Without callback, a few lines of code can do everything we need.
Another example of printing the pdf document of Ruan Yifeng's "Introduction to ECMAScript 6":
const puppeteer = require('puppeteer'); const getRootDir = require('root-directory'); (async () => { const rootDir = await getRootDir(); let pdfDir = rootDir + "/public/pdf/es6-pdf/"; const browser = await puppeteer.launch({ headless: false, devtools: true //Development, useful when headless is true }); let page = await browser.newPage(); await page.goto('http://es6.ruanyifeng.com/#README'); await page.waitFor(2000); const aTags = await page.evaluate(() => { let as = [...document.querySelectorAll('ol li a')]; return as.map((a) => { return { href: a.href.trim(), name: a.text }; }); }); if (!aTags) { browser.close(); return; } await page.pdf({path: pdfDir + `${aTags[0].name}.pdf`}); page.close(); // promise all can also be used here, but the cpu may be tight and prudent for (var i = 1; i < aTags.length; i++) { page = await browser.newPage(); var a = aTags[i]; await page.goto(a.href); await page.waitFor(2000); await page.pdf({path: pdfDir + `${a.name}.pdf`}); console.log(a.name); page.close(); } browser.close(); })();