Replace CodeClimate with SticklerCI?


#1

I know you all love the lovely CodeClimate errors, like Your method has 7 lines (exceeds 6 allowed) :slight_smile: I was also annoyed by the fact that whenever it finds lint issues, you can either ignore them or have to push another commit for really inconsequential issues like “File not ending with a blank line”.

With that in mind I started searching for an automated solution and landed on SticklerCI which is based on the Open Source lint-review project.

This is the pull request that adds support for it:

I also did a dummy commit with a Flake8 violation, to trigger its fix:

It seems to do its job, but what bothered me here was how silent it was – there was no visible feedback on what actually failed aside from the follow up commit.

For the above reason, I decided to wait with merging it into our repository and wait for more comments… So what do you think?


#2

(There is, of course, the option of using them both, but…)


#3

Up till now, CodeClimate has shown me 0 benefit and ∞ annoyance. So I say ditch it.

Stickler - Does it offer anything other than lints which we already run locally?
It’s Auto-fix feature is redundant if we implement a git pre-push test+lint hook, which imo is better.


#4

@arikfr take a look at DangerJS. It allows some features we talked about regarding automatic community PR enforcements like “max PR size” and “test required”.


#5

:+1: So no matter what we will switch to a different tool that will do only linting.

No, but it’s still important because not everyone (specially “adhoc” contributors) have the same setup. Therefore it’s important to have automatic lints done regardless of local setup. Just like we run automatic tests.

I remember seeing it in the past, I will check it again. :+1:


#6

Therefore it’s important to have automatic lints done regardless of local setup

No doubt, wasn’t implying otherwise.
But I wrongly assumed running lints on PRs won’t require third-party software.


#7

After some additional thought I realized that while the autofix feature is really nice, it has the potential to create confusion for contributors:

  1. They push some code.
  2. Stickler fixes lint issues.
  3. They try to push again and get an error (because remote changed).

At this point, if they aren’t familiar with Git enough, they might get confused/frustrated. The value we get from the auto fixes doesn’t seem to worth this friction point.

:wave: Stickler.

I will still put some effort in either replacing CodeClimate with a less noisy tool or adjust its settings.