Duplicate console log messages when conflicting reporters are defined

[eklundjoshua]

Description of the problem:

In our project we have both Mocha & Remap-Istanbul defined as reporters for our Karma Tests. It seems as though these two reporters both have similar functionality in that they both intercept console.log messages that are written out from the JavaScript so there are always two log messages for every console.log message that we write. I do not think there is a feature yet, but it would be neat to add functionality that basically disables console.log messages for one plugin but leaves them for another so that way you can have the functionality of both plugins but only allow 1 of them to output console.log messages.

Environment Details

Karma Version 0.13.22

Steps to reproduce the behaviour

1. Defined reporters: ["mocha", "coverage", "karma-remap-istanbul"] in the karma.conf.js for my angular app
2. Ran my tests
3. Console logs that were written in the code were duplicated for every test that had been executed.
4. Removed "karma-remap-istanbul" from the reporters list, reran the tests, and the duplication went away.
5. Added "karma-remap-istanbul" back to the reports list and removed "mocha", reran the tests, and the duplication went away.

[maksimr]

@eklundjoshua could you post karma config for karma-remap-istanbul?

Thanks!

[eklundjoshua]

@maksimr Thanks for the follow up! Here is the karma.conf.js that I'm currently using:

var path = require('path');
var webpackConfig = require('./webpack.config');
module.exports = function (config) {
    var _config = {
        // base path that will be used to resolve all patterns (eg. files, exclude)
        basePath: '',

        // frameworks to use
        // available frameworks: https://npmjs.org/browse/keyword/karma-adapter
        frameworks: ['jasmine'],

        // list of files / patterns to load in the browser
        files: [
            { pattern: './karma-shim.js', watched: false }
        ],

        // list of files to exclude
        exclude: [],

        // preprocess matching files before serving them to the browser
        // available preprocessors: https://npmjs.org/browse/keyword/karma-preprocessor
        preprocessors: {
            './karma-shim.js': ['coverage', 'webpack', 'sourcemap']
        },

        webpack: webpackConfig,

        webpackMiddleware: {
            // webpack-dev-middleware configuration
            // i. e.
            stats: 'errors-only',
            noInfo: true
        },

        coverageReporter: {
            dir: 'coverage/',
            reporters: [{
                type: 'json',
                dir: 'coverage',
                subdir: 'json',
                file: 'coverage-final.json'
            }, {
                type: 'html',
                dir: 'coverage',
                subdir: 'html'
            }]
        },

        remapIstanbulReporter: {
            src: 'coverage/json/coverage-final.json',
            reports: {
                lcovonly: 'coverage/json/lcov.info',
                html: 'coverage/html',
                'text': null
            },
            timeoutNotCreated: 1000, // default value
            timeoutNoMoreFiles: 1000 // default value
        },

        webpackServer: {
            noInfo: true // please don't spam the console when running in karma!
        },

        // test results reporter to use
        // possible values: 'dots', 'progress', 'mocha'
        // available reporters: https://npmjs.org/browse/keyword/karma-reporter
        reporters: ["mocha", "coverage", "karma-remap-istanbul", "html"],

        htmlReporter: {
            outputFile: 'reports/unit_tests/index.html',
            pageTitle: 'Unit Test Results',
            useCompactStyle: true,
            groupSuites: true
        },

        // web server port
        port: 9876,

        // enable / disable colors in the output (reporters and logs)
        colors: true,

        // level of logging
        // possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
        logLevel: config.LOG_INFO,

        // enable / disable watching file and executing tests whenever any file changes

        // start these browsers
        // available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
        browsers: ['PhantomJS'], // you can also use Chrome

        // Continuous Integration mode
        // if true, Karma captures browsers, runs the tests and exits
        singleRun: true
    };

    config.set(_config);
};

[maksimr]

@eklundjoshua

From remap-instanbul

Try remove 'text': null from remapIstanbulReporter reporters

Thanks

[wesleycho]

I've seen this issue in my app as well, it turns out that it doesn't matter what reporters you are using with remap-istanbul - I was able to isolate it as a karma coverage reporter issue.

Edit: Correction, I misspoke - this is very clearly a karma-remap-istanbul issue, not karma. It does not matter what reporters you send to it either.

Edit 2: Digging further into it, this seems to occur if I use karma-remap-istanbul or karma-junit-reporter on top of coverage. This may be a bug within karma itself with how it handles reporting, or something about the combination of plugins.

Edit 3: This does not occur if I exclusively use karma-coverage and karma-remap-istanbul. Exclusively using karma-coverage and karma-junit-reporter causes no output to be shown. Using dots, karma-coverage, and karma-remap-istanbul causes the log to output one in the log color in the shell, and one in just pure text.

I suspect that there is a mutation happening in the mocha, dots, and progress reporters that bleed over in karma-coverage. Will dig into the reporter area closer to understand how the system works & determine which plugin is the true culprit here.

[wesleycho]

After doing some chasing around, I am mostly convinced this is because karma-remap-istanbul is using baseReporterDecorator, so when writes are triggered in the base reporter, it propagates to the karma-mocha-reporter and karma-remap-istanbul.

I would recommend for fixing this that the writing of the logs be delegated to another reporter type that is not base, and have dots and progress use that. This would be a breaking change, and would need to be scheduled for a major release if this approach is adopted.

[eklundjoshua]

Thank you @maksimr. I tried your recommendation but still saw the duplication of logs. Thank you @wesleycho for your detailed findings as well!

[wesleycho]

Just so a solution is out in the open, we could either move to create another reporter type that can be decorated around, or we can try making a mixin pattern of sorts to do this. A mixin pattern would probably be more flexible than an inheritance pattern here, and not create problems in case we missed other features of the reporter we missed.

[JPMardelli]

I've got similar issues when running with the karma coverage reporter and the karma mocha reporter. Any of the config options to turn off console reporting do it for both, can't find a way to turn one of and not the other without rewriting them. Wish there was an option for which reporter to send console output to. Also unrelated, but I don't understand why the coverage reporter doesn't respect the color flag for console reporting.

[wesleycho]

Alternatively, we could add a new hook inside karma for running actions after reporters, since some of the things being done by reporters like the karma-junit-reporter and karma-remap-istanbul are actually doing post test processing. This would minimize pain to other reporters, and the reporters causing the multiple logging can then be refactored to avoid this. Maybe a postReporters config option that takes an array of plugins to run after the reporters are finished running.

Thoughts on this @dignifiedquire ?