Page MenuHomeWildfire Games

Linting with Coala
ClosedPublic

Authored by Itms on Mar 11 2017, 9:53 PM.

Details

Summary

This is a proposal for the basis of a linting suite using Coala.

This linting would happen before the build+tests pass in Jenkins.

Test Plan

Test the script.

Diff Detail

Repository
rP 0 A.D. Public Repository
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
elexis added inline comments.May 26 2017, 9:04 PM
build/jenkins/lint-config/eslintrc.json
11 ↗(On Diff #1551)

✓ "disallow labels for anything other than loops and switches" http://eslint.org/docs/rules/no-empty-label
if it was up to me, we can prohibit labels altogether

12 ↗(On Diff #1551)

Noooooooooooooooooooooo

http://eslint.org/docs/rules/no-extra-boolean-cast
It would prevent if (!!foo) {, but this is widely used to return a boolean if a (A) property doesn't exist, like x && x.y or to convert something to boolean that isn't a boolean yet (for example if the target function expects a boolean and treats something like a boolean but someone passes some class instance)

The trick is mostly used to return something truthy to true.

The linter already complained about files that contain this double negation.

If we were to prevent that, we would have to use "x ? true : false` in many cases or even `

And to check for existence of settings, we would have to test like this (while easily being able to oversee some typo):
else if (g_IsController && g_GameAttributes.settings.PlayerData[playerSlot].AI && g_GameAttributes.settings.PlayerData[playerSlot].AI != "")

The first proposal was posted here
https://wildfiregames.com/forum/index.php?/topic/22251-coding-conventions-updates/&_fromLogin=1

13 ↗(On Diff #1551)

✓ prevents unneeded parenthesis like `if ((x == y) && z) http://eslint.org/docs/rules/no-extra-parens ( ͡° ͜ʖ ͡°)

14 ↗(On Diff #1551)
15 ↗(On Diff #1551)

Looks correct. The only case when we really do foo = function(..) {.. is for local functions and for global objects that have functions in objects, like unitAI or gui http://eslint.org/docs/rules/no-func-assign

16 ↗(On Diff #1551)

prevents if(!key in object) {

This rule was deprecated in ESLint v3.3.0 and replaced by the no-unsafe-negation rule.

which isn't specified here, but I guess that has a reason.

✓ as long as we specify it at all

17 ↗(On Diff #1551)


Uh yes, who would ever call Math() or JSON()
http://eslint.org/docs/rules/no-obj-calls

18 ↗(On Diff #1551)

✓ useful one, prevents return; foo; http://eslint.org/docs/rules/no-unreachable

19 ↗(On Diff #1551)

✓ prevent if (foo == NaN) {, require isNaN(foo)

20 ↗(On Diff #1551)

✓ yes please! I have seen too many broken JS doc comments http://eslint.org/docs/rules/valid-jsdoc

21 ↗(On Diff #1551)

✓ prevents typos typeof foo === "strnig" http://eslint.org/docs/rules/valid-typeof

23 ↗(On Diff #1551)

ok, but but wai not
prevents

if (true) {
    var build = true;
} else {
    var build = false;
}

http://eslint.org/docs/rules/block-scoped-var

Might be ok to start with it since that is still present in the codebase in various places, but I'd discourage a code pattern like that. A variable ought to be defined in the broadest scope it's accessible.

If we use let in local scope everywhere, above pattern will throw error at JS-compile time afaik, which makes it easy to spot for devs to find instances where variables are used out of scope weirdly

24 ↗(On Diff #1551)

✓ must have, prevents if (x) return; else return X. should either never or always specify the return value. http://eslint.org/docs/rules/consistent-return

25 ↗(On Diff #1551)

Not convinced in particular for requireing default, (Not that I like switches neither), for example if we can guarantee that it will never occur.
http://eslint.org/docs/rules/default-case

26 ↗(On Diff #1551)

✓ prevents foo["bar"] requires foo.bar http://eslint.org/docs/rules/dot-notation

27 ↗(On Diff #1551)

✓ prevents if (x) return; else bar(); http://eslint.org/docs/rules/no-else-return since that else is entirely pointless, code that can't be executed.

28 ↗(On Diff #1551)

✓ didn't see someone trying this yet, definitely should prevent access to this if not inside a class http://eslint.org/docs/rules/no-invalid-this

29 ↗(On Diff #1551)

Not necessarily.

Prevents unneeded scopes.
D195, also might help structuring things sometimes, making some vars more local than before. Would only to argue for the D195, not really for any others. http://eslint.org/docs/rules/no-lone-blocks

30 ↗(On Diff #1551)

✓ We don't run into closure issues that often (only saw it once in menu.js iirc) so we violate this recommendation all over the place
http://eslint.org/docs/rules/no-loop-func

31 ↗(On Diff #1551)

✓ prevents duplicate spaces if(foo === "bar") {} I do this all the time without noticing until the linter complains http://eslint.org/docs/rules/no-multi-spaces

32 ↗(On Diff #1551)

✓ Never seen the Symbol class being used, so I guess it's ok to prevent doing wrong things with it http://eslint.org/docs/rules/no-new-symbol

33 ↗(On Diff #1551)

✓ We have many occurances of that I assume, but they will be easy to fix
prevents

var a = 3;
var a = 10;
34 ↗(On Diff #1551)

✓ prevents return x = z which is useless and misleading http://eslint.org/docs/rules/no-return-assign

35 ↗(On Diff #1551)
36 ↗(On Diff #1551)
37 ↗(On Diff #1551)

Maybe.

prevents

while (node)
    doSomething(node);

assuming that node never changes.
Can have it enabled until we meet the first false positive.
http://eslint.org/docs/rules/no-unmodified-loop-condition

38 ↗(On Diff #1551)

✓ prevents useless code

function increment() { i += 1; }
increment(); // return value is unused, but i changed as a side effect

http://eslint.org/docs/rules/no-unused-expressions

39 ↗(On Diff #1551)

✓ prevents unused labels (for GOTOs), definitely cut them off. http://eslint.org/docs/rules/no-unused-labels

40 ↗(On Diff #1551)

Maybe. Prevents "x" + "y".

There might be occurances where having it this way might be more logical, as those might be 2 logically separate strings that might be replaced individually sometime.

Can be kept until further notice.

http://eslint.org/docs/rules/no-useless-concat

42 ↗(On Diff #1551)

✓prevents delete x since delete only works on properties http://eslint.org/docs/rules/no-delete-var

43 ↗(On Diff #1551)

✓ prevents variables that are label names http://eslint.org/docs/rules/no-label-var

44 ↗(On Diff #1551)

✓ prevents overwriting defined keywords like undefined http://eslint.org/docs/rules/no-shadow-restricted-names

45 ↗(On Diff #1551)

Wrong?

prevents overshadowing of ones own variables:

var a = 3;
function b() {
    var a = 10;
}

Sounds good to me, is there a reason not to do it?

46 ↗(On Diff #1551)

✓ definitely each variable referenced should be declared at some point. However

47 ↗(On Diff #1551)

✓ Meh fine (as discussed in the linting topic). Prevents var foo = undefined; Will fix accoring code sometime.

48 ↗(On Diff #1551)

Any reason for ignoring this?

Prevents unused variables (never read from).
http://eslint.org/docs/rules/no-unused-vars

50 ↗(On Diff #1551)

Why not?

Prevents lowercase prototype names. At least for the sim+AI directory everything looks fine on first sight
http://eslint.org/docs/rules/new-cap

51 ↗(On Diff #1551)

✓ requires parenthesis for ctors, i.e. prevents var person = new Person;

52 ↗(On Diff #1551)


prevents `var isYes = answer === 1 ? true : false`;
(Depends on above "noooooooooooo" thing)

54 ↗(On Diff #1551)


prevents overwriting of class definitions:

class A { }
A = 0;
55 ↗(On Diff #1551)

✓ prevents overwriting consts:

const a = 0;
a = 1;
56 ↗(On Diff #1551)

✓ prevents duplicate class members:

class Foo {
  bar() { console.log("hello"); }
  bar() { console.log("goodbye"); }
}
57 ↗(On Diff #1551)

✓ requires const if the value isn't changed.
totally dont want this rule so that our code and mods can modify globals in different files externally
http://eslint.org/docs/rules/prefer-const

leper edited edge metadata.May 27 2017, 12:52 AM
In D213#22793, @elexis wrote:

This could also be split into one patch for each proposal in case we can't find anyone soon for python and perl

IMO we don't have enough python or perl around to really worry about those, and we can always fix things later.

That said the perl config is missing. @Itms

Itms updated this revision to Diff 2232.May 27 2017, 11:02 AM
  • Addressed elexis and Imarok's suggestions (with an answer on the patch when I didn't agree)
  • Added the Perl critic configuration as pointed out by leper
  • Made the script not fail when lint issues were discovered, else Phabricator write 'Those changes have failed to build', which is not entirely true and annoying.
build/jenkins/lint-config/eslintrc.json
29 ↗(On Diff #1551)

Take a closer look at the docs, it detects only blocks that are useless (enclosing a var or a function, but not a let for instance).

48 ↗(On Diff #1551)

Yes, a lot of things are defined and used in other files, so a lot of false positives.

Why not split python and perl to a separate diff before we start mixing reviews of all 4 linters in one page?

RE eslint:

build/jenkins/lint-config/eslintrc.json
25 ↗(On Diff #1551)

Still not convinced. Why would we require default in switch statements?

29 ↗(On Diff #1551)

I see. Then we can set this to 1

45 ↗(On Diff #1551)

no-shadow should become 1, no?

48 ↗(On Diff #1551)

oh, obviously.

50 ↗(On Diff #1551)

✓ until we find a good reason not to

Build is green

Updating workspaces.
Build (release)...
Build (debug)...
Running release tests...
Running cxxtest tests (306 tests)..................................................................................................................................................................................................................................................................................................................OK!
Running debug tests...
Running cxxtest tests (306 tests)..................................................................................................................................................................................................................................................................................................................OK!

http://jw:8080/job/phabricator/1371/ for more details.

Imarok added inline comments.May 27 2017, 1:49 PM
build/jenkins/lint-config/eslintrc.json
50 ↗(On Diff #1551)

I think here is the reason: https://code.wildfiregames.com/D516#22933

Itms added a comment.May 27 2017, 1:56 PM
In D213#22926, @elexis wrote:

Why not split python and perl to a separate diff before we start mixing reviews of all 4 linters in one page?

I could split indeed, but I don't think we have anybody with strong opinions on the good ways to write scripts in Python/Perl. Also we only use them as side tools, our scripts don't directly impact the quality of the game. So I think we should just follow the widespread and recommended configuration which is in those files, and tweak them when we update the scripts.

build/jenkins/lint-config/eslintrc.json
25 ↗(On Diff #1551)

Because that's the proper way of writing a switch. You should always handle all cases, you can issue a warning if the default case shouldn't happen but does.

45 ↗(On Diff #1551)

Sorry, I had missed it indeed.

50 ↗(On Diff #1551)

I tried to change that in the previous iteration and it produced things like in D516, so that's a good reason to have it to zero ?

Itms updated this revision to Diff 2239.May 27 2017, 1:58 PM

Update

In D213#22956, @Itms wrote:
In D213#22926, @elexis wrote:

Why not split python and perl to a separate diff before we start mixing reviews of all 4 linters in one page?

I could split indeed, but I don't think we have anybody with strong opinions on the good ways to write scripts in Python/Perl.

Then why is this uploaded for review? I don't feel comfortable with accepting two linters reviewed to the bone and two linters not even read. If we don't really care about these two linters (works for me) then I propose to commit sh file, pylint and perlcritic without review as a placeholder and update this differential revision proposal for the reviewed part.

build/jenkins/lint-config/pylintrc
1 ↗(On Diff #2239)

In case of review https://docs.pylint.org/en/1.6.0/features.html and I'd ask for the source of this file (to see whether those are actually defaults)

Executing section Default...
Executing section Source...
Executing section JS...
Executing section XML GUI...
Executing section Python...
Executing section Perl...

http://jw:8080/job/phabricator_lint/57/ for more details.

Build is green

Updating workspaces.
Build (release)...
Build (debug)...
Running release tests...
Running cxxtest tests (306 tests)..................................................................................................................................................................................................................................................................................................................OK!
Running debug tests...
Running cxxtest tests (306 tests)..................................................................................................................................................................................................................................................................................................................OK!
Checking XML files...

http://jw:8080/job/phabricator/1374/ for more details.

Imarok added inline comments.Jun 1 2017, 12:01 PM
build/jenkins/lint-config/jshintrc.json
6 ↗(On Diff #2239)

Trying this on my on device, it seems loopfunc has to be true to not get this Don't make functions within a loop. error.
(Sorry for suggesting the wrong change)
When set to true, these options will make JSHint produce fewer warnings about your code.

There are still warnings of the type Use '===' to compare with '0'.left. So either add "-W041": false or wait for the next release, which will remove this warning (https://github.com/jshint/jshint/commit/376fa624d7eacc8f7b85c721bcd48a07ecf2643b)

Imarok added a comment.Jun 1 2017, 2:23 PM

ESLint complains about such things as colorizeHotkey("%(hotkey)s" + " ", "session.kill") which is done to make people changing this line not forgetting about this space.
So we should set no-useless-concatto 0.

I also get this error, when executing coala with the .coafile specified here:

[WARNING][18:11:18] Implicit 'Default' section inheritance is deprecated. It will be removed soon. To silence this warning remove settings in the 'Default' section from your coafile. You can use dots to specify inheritance: the section 'all.python' will inherit all settings from 'all'.

I fixed it by removing the Default section. (But I don't know, if this fix is correct)

Imarok added a comment.EditedJun 22 2017, 12:25 AM

For eslint we should also add:

"space-in-parens": 1,
"space-before-function-paren": ["warn", "never"],
"space-infix-ops": 1,
"space-unary-ops": 1,
"spaced-comment": ["warn", "always"],
"comma-spacing": 1,
"semi-spacing": 1,
"brace-style": ["warn", "allman", { "allowSingleLine": true }],
"no-multi-assign": 1,
"no-trailing-spaces": 1,
"object-curly-spacing": ["warn", "always"],
"operator-assignment": 1,
"operator-linebreak": ["warn", "after"],
"semi": 1,
"yoda": 1,
"curly": ["warn", "multi"],
"quote-props": 1,
"indent": ["warn", "tab"],
"no-mixed-spaces-and-tabs": ["warn", "smart-tabs"],
"key-spacing": 1

We should also use http://eslint.org/docs/rules/linebreak-style.
Maybe we should use "brace-style": ["allman", { "allowSingleLine": true }] for enforcing braces on the new line.
Though this would error out for:

bar.filter(el => {
    doo;
    something;
}

This plugin: https://github.com/jrsearles/eslint-plugin-brace-rules can enforce different brace styles for each block type (forin, forof, try, switch, arrow func,...), so we just have to discuss which exceptions we want. (see https://wildfiregames.com/forum/index.php?/topic/21949-linting/#comment-328472) (I tested the plugin and it works quite well)

Edit: Fixed operator-linebreak
Edit2: added no-mixed-spaces-and-tabs
Edit3: added indent rule
Edit4: the ident rule bugs for switch cases with ESLint < 4.0, so we need the most recent ESLint version.
Edit5: added link for brace style
Edit6: tested the plugin
Edit7: key-spacing

Imarok added a comment.Jul 7 2017, 3:54 PM

We should change "no-multi-spaces": 1 to "no-multi-spaces": ["warn", { "ignoreEOLComments": true }], to allow aligning end of line comments like:

let foo;   // init foo
foo += 32; // add 32

here is the config part for the curly option:

"brace-rules/brace-on-same-line": ["warn", {

 "FunctionDeclaration": "never",
 "FunctionExpression": "ignore",
 "ArrowFunctionExpression": "always",
 "IfStatement": "never",
 "TryStatement": "ignore",
 "CatchClause": "ignore",
 "DoWhileStatement": "never",
 "WhileStatement": "never",
 "ForStatement": "never",
 "ForInStatement": "never",
 "ForOfStatement": "never",
 "SwitchStatement": "never"
}, { "allowSingleLine": true }],

We also need to add

"plugins": [
    "brace-rules"
],

after

"parserOptions": {
    "ecmaVersion": 6
},

and install the plugin brace-rules

The Linter and Build Bot should use spoilers, as they can be seen in https://code.wildfiregames.com/D615#32677

Itms updated this revision to Diff 3841.Oct 3 2017, 8:44 PM
Itms added a reviewer: Imarok.

I followed Imarok's suggestions except for a thing: space-infix-ops does not help us because we often write things like foo[i+1] in case where foo[i + 1] hurts readability.

I also agree with elexis that it's not good to have those unreviewed Python rules (and Perl ones for the same reason). On top of that our Python scripts are so badly written that the linter would be extremely noisy. I'll try to devise rules while fixing the scripts themselves.

Vulcan added a comment.Oct 3 2017, 8:45 PM
Executing section Default...
Executing section Source...
Executing section JS...
Executing section XML GUI...

http://jenkins-master:8080/job/phabricator_lint/574/ for more details.

Vulcan added a comment.Oct 3 2017, 8:48 PM

Build is green

Updating workspaces...
Build (release)...
Build (debug)...
Running release tests...
Running cxxtest tests (307 tests)...................................................................................................................................................................................................................................................................................................................OK!
Running debug tests...
Running cxxtest tests (307 tests)...................................................................................................................................................................................................................................................................................................................OK!
Checking XML files...

http://jenkins-master:8080/job/phabricator/2101/ for more details.

Imarok added a comment.Oct 9 2017, 3:11 PM
In D213#36986, @Itms wrote:

I followed Imarok's suggestions except for a thing: space-infix-ops does not help us because we often write things like foo[i+1] in case where foo[i + 1] hurts readability.

I also agree with elexis that it's not good to have those unreviewed Python rules (and Perl ones for the same reason). On top of that our Python scripts are so badly written that the linter would be extremely noisy. I'll try to devise rules while fixing the scripts themselves.

Good.
Already applied that?

Itms added a comment.Oct 12 2017, 3:54 PM
In D213#37171, @Imarok wrote:

Already applied that?

Sorry, yes I did when updating the patch. If things are looking good for you guys I can commit the script and the configurations.

Imarok accepted this revision.Oct 20 2017, 3:15 PM

Looks good (Only talking about js linting).
I assume Jenkins has at least jshint 2.9.5 installed. (Else you should add the -W041 option)

This revision is now accepted and ready to land.Oct 20 2017, 3:15 PM
In D213#37560, @Imarok wrote:

Looks good (Only talking about js linting).
I assume Jenkins has at least jshint 2.9.5 installed. (Else you should add the -W041 option)

Judging from https://code.wildfiregames.com/D429#37573 jshint is older than 2.9.5.

Itms added a comment.Oct 28 2017, 4:24 PM
In D213#37703, @Imarok wrote:

Judging from https://code.wildfiregames.com/D429#37573 jshint is older than 2.9.5.

I updated jshint and I am now going to commit this script. Thanks a lot for your help with this!

This revision was automatically updated to reflect the committed changes.