Research and setup code coverage tool for our community projects

[isabelcosta]

Description

Research code coverage report tool to apply to one of our projects and then set up at least one project with that (e.g.: perhaps mentorship backend 🙃 )

For the research have into account the existing tech stack projects in the community (e.g.: kotlin, python, java, ..).

• Documentation options - Code Coverage sheet

Note: you might need help from @anitab-org/admins with this setup.

Tasks

• Research code coverage report tools to be used in PR analysis
• Select and communicate to the community the option selected
• Setup the selected tool on a project

[Tlazypanda]

@isabelcosta @mayburgos After going through the Options here is what I think we should go for - Code Climate. The reasons and comparisons are as follows:-

• The problem with Codacy is that it wont support flutter.
• The problem with Codecov is that the team size< 5 to be free for OSS.
• Coveralls and Code climate both support flutter(dart) but the support for Code climate is better in terms of support, it is maintained more actively, user interface is better.

Although same may argue that Coveralls has been around for longer but Code Climate although a relatively new tool is now gaining popularity among Opensource projects even to the likes of Bootstrap and Angular using it as well.

Also adding to this conversation, Travis CI can be used along with Code Climate(they both work together great!) for creating the perfect QA atmosphere.

Can you kindly let me know if I should go ahead with this? Thanks ✌️

[meenakshi-dhanani]

We need coverage tools for the Android front end and the backend, correct? So we might require different for flutter and different for the python backend?

…

On Wed, 20 May, 2020, 3:59 AM Sneha Mishra, ***@***.***> wrote: @isabelcosta <https://github.com/isabelcosta> @mayburgos <https://github.com/mayburgos> After going through the Options here is what I think we should go for - Code Climate. The reasons and comparisons are as follows:- - The problem with Codacy is that it wont support flutter. - The problem with Codecov is that the team size< 5 to be free for OSS. - Coveralls and Code climate both support flutter(dart) but the support for Code climate is better in terms of support, it is maintained more actively, user interface is better. Although same may argue that Coveralls has been around for longer but Code Climate although a relatively new tool is now gaining popularity among Opensource projects even to the likes of Bootstrap and Angular using it as well. Also adding to this conversation, Travis CI can be used along with Code Climate(they both work together great!) for creating the perfect QA atmosphere.. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADM6DFOAUNQJAVQEKEZZM3LRSMB4DANCNFSM4NFGUMAQ> .

[SanketDG]

I'll put in my two cents.

• It's fine if we use two completely different tools for two different projects, I think it's more important to use the right tool with the right language.
• Codecov's 5 user limit is probably being misrepresented here. There are lots of large open source (Python) orgs using codecov for their coverage report.
• Travis CI should ideally work with any coverage provider, I don't think it gains any advantage from supporting a single tool well, especially in the spirit of Open Source, unless it's officially documented.
• We can always replace one tool for another very easily (unless we are paid haha), so comparing too exhaustively can be just unproductive.
• We should focus on making sure we set up the coverage metrics right, provide a great developer experience (in PRs!) and setup a great testing/coverage culture across the organization.
• Also, this initiative is great!

[Tlazypanda]

Hey @SanketDG! Thanks for the review 💯

Just adding to what I meant by some of these -

• Yeah absolutely agree with this, I meant for achieving uniformity across all repositories this might be a good solution.
• Not sure about this ...their website mentioned this so might be wrong on this one 😅 There is a GitHub app for Codecov that is in use which allows for limited features for free.
• Oops what I meant here is that Code climate being a relatively new tool still works well with Travis CI (not favoring any tool above the other xD) so I meant ading them together can create a good atmosphere.(key takeaway being we need to add a CI tool in unison with this)
• Yeah actually we had a discussion about this in the QA open session and decided to do a comparison to get a rough idea of where to start with.
• Agree!! 💯 Let's hope we get that done.
• ikr!!

[Tlazypanda]

@isabelcosta @mayburgos So should I go ahead with this? Thanks ✌️

coverage bot + travis ci + codeclimate

[Tlazypanda]

We need coverage tools for the Android front end and the backend, correct? So we might require different for flutter and different for the python backend?
…
On Wed, 20 May, 2020, 3:59 AM Sneha Mishra, @.***> wrote: @isabelcosta https://github.com/isabelcosta @mayburgos https://github.com/mayburgos After going through the Options here is what I think we should go for - Code Climate. The reasons and comparisons are as follows:- - The problem with Codacy is that it wont support flutter. - The problem with Codecov is that the team size< 5 to be free for OSS. - Coveralls and Code climate both support flutter(dart) but the support for Code climate is better in terms of support, it is maintained more actively, user interface is better. Although same may argue that Coveralls has been around for longer but Code Climate although a relatively new tool is now gaining popularity among Opensource projects even to the likes of Bootstrap and Angular using it as well. Also adding to this conversation, Travis CI can be used along with Code Climate(they both work together great!) for creating the perfect QA atmosphere.. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADM6DFOAUNQJAVQEKEZZM3LRSMB4DANCNFSM4NFGUMAQ .

@meenakshi-dhanani So sorry I missed your message 😅 codeclimate has support for both dart and python so that shouldn't be a problem. But yeah as @SanketDG pointed out other tools may work better for specific languages... I did some digging and found blogs for flutter used with codeclimate so I think it should work fine.

[nandini45]

@Tlazypanda correct me if i am missing anything..In the documentation generators HeaderDoc is missing the possibility of extended customization??

[Tlazypanda]

Hey @nandini45 Not quite sure I understand what you refer to here 😅 can you elaborate a bit on what you are referring to? Thanks ✌️

[nandini45]

oh i couldn't go through the whole doc. i went only through the documentation generators. i thought headerdoc was missing some content but now i am rechecking it the content has been update or i think i missed the column.. @Tlazypanda

you didn't add the anitab-org.io repo that is under construction in the existing repo?

[Tlazypanda]

@nandini45 Umm... are you sure this is relevant to this issue 😅 I am not able to comprehend anything and am extremely confused 😕

[nandini45]

tell me is this what the issue about'
you are talking about all the community projects and setting up their tools right?
what can we use for coding and all related stuff right?
@Tlazypanda

[Tlazypanda]

Ohh are you referring to the doc that @isabelcosta linked in the issue ...yeah it's a reference doc from couple of years back that she shared so that I can research faster 😅

[nandini45]

and on basis of that you have to create a new doc
right?

[Tlazypanda]

no doc isnt required I summarized my findings in the comment above 😅

[nandini45]

oh ok i thought the doc is updated and i couldn't fond one project so i was wronged. my bad😅

[Tlazypanda]

Ohh no issues 😅 Even I got confused as to what it referred to ...Thanks for your review ❤️

[isabelcosta]

@Tlazypanda I agree with your choices, I think we should start implementing these as soon as possible :)
I think you have started a conversation on Zulip already, can you please tag me there, I want to help you get what you need to start implementing this in one of the repositories

[mayburgos]

@Tlazypanda I have no clue if you've started. @sidvenu brought up today that we can use GitHub Actions for the Travis CI work. Have you looked into those automation tools provided in GitHub Actions, Sneha?

[meenakshi-dhanani]

Hello there hasn't been any comment on this PR lately. We think coverage is important for us to go to production. Wanted to follow up on this? @Tlazypanda any updates on the status?

[Tlazypanda]

Hey @meenakshi-dhanani this has been setup in my fork for a demo, please refer this https://anitab-org.zulipchat.com/#narrow/stream/216325-quality-assurance/topic/Setup.20coverage.20tools/near/202120525 conversation where I have included links of the demo. Any feedback appreciated 🎉

[isabelcosta]

This was already successfully done here https://github.com/anitab-org/mentorship-backend
@Tlazypanda should we close this issue?